forked from qiubaiying/qiubaiying.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
atom.xml
527 lines (270 loc) · 138 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>辉</title>
<subtitle>[object Object]</subtitle>
<link href="/atom.xml" rel="self"/>
<link href="https://blog.collmall.com/"/>
<updated>2019-04-19T08:50:20.232Z</updated>
<id>https://blog.collmall.com/</id>
<author>
<name>John Doe</name>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>2019-09-20-Typora For Markdown 语法</title>
<link href="https://blog.collmall.com/2020/ck91gq88m0007ygta5vvn5pcp/"/>
<id>https://blog.collmall.com/2020/ck91gq88m0007ygta5vvn5pcp/</id>
<published>2020-04-15T10:32:04.074Z</published>
<updated>2019-04-19T08:50:20.232Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><p>#Typora For Markdown 语法</p><p>[TOC]</p><p>###数学表达式</p><p>要启用这个功能，首先到<code>Preference</code>-><code>Editor</code>中启用。然后使用<code>$</code>符号包裹Tex命令，例如：<code>$lim_{x \to \infty} \ exp(-x)=0$</code>将产生如下的数学表达式：</p><p>$\lim_{x \to \infty} \exp(-x)=0$</p><p>###下标</p><p>下标使用<code>~</code>包裹，例如：<code>H~2~O</code>将产生H<del>2</del>O, 即水的分子式。</p><p>###上标</p><p>上标使用<code>^</code>包裹，例如：<code>y^2^=4</code>将产生表达式y^2^ = 4</p><p>###插入表情:happy:</p><p>使用<code>:happy:</code>输入表情:happy:,使用<code>:sad:</code>输入表情:sad:,使用<code>:cry:</code>输入表情:cry:等。以此类推！</p><h3 id="下划线"><a href="#下划线" class="headerlink" title="下划线"></a>下划线</h3><p>用HTML的语法<code><u>Underline</u></code>将产生下划线<u>Underline</u>.</p><h3 id="删除线"><a href="#删除线" class="headerlink" title="删除线"></a>删除线</h3><p>GFM添加了删除文本的语法，这是标准的Markdown语法木有的。使用<code>~~</code>包裹的文本将会具有删除的样式，例如<code>~删除文本~</code>将产生<del>删除文本</del>的样式。</p><h3 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h3><ul><li>使用`包裹的内容将会以代码样式显示，例如</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">使用`printf()`</span><br></pre></td></tr></table></figure><p>则会产生<code>printf()</code>样式。</p><ul><li><p>输入<code>~~~</code>或者```然后回车，可以输入代码块，并且可以选择代码的语言。例如：</p></li><li><pre><code>```javapublic Class HelloWorld{ System.out.println("Hello World!");}```<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">将会产生</span><br><span class="line"></span><br><span class="line">~~~java</span><br><span class="line">public Class HelloWorld{</span><br><span class="line"> System.out.println("Hello World!");</span><br><span class="line">}</span><br></pre></td></tr></table></figure>### 强调使用两个*号或者两个_包裹的内容将会被强调。例如<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">**使用两个*号强调内容**</span><br><span class="line">__使用两个下划线强调内容__</span><br></pre></td></tr></table></figure>将会输出**使用两个*号强调内容**__使用两个下划线强调内容__Typroa 推荐使用两个*号。### 斜体在标准的Markdown语法中，*和_包裹的内容会是斜体显示，但是GFM下划线一般用来分隔人名和代码变量名，因此我们推荐是用星号来包裹斜体内容。如果要显示星号，则使用转义：<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">\*</span><br></pre></td></tr></table></figure>### 插入图片我们可以通过拖拉的方式，将本地文件夹中的图片或者网络上的图片插入。</code></pre></li></ul><h3 id="插入URL连接"><a href="#插入URL连接" class="headerlink" title="插入URL连接"></a>插入URL连接</h3><p>使用尖括号包裹的url将产生一个连接，例如：<code><www.baidu.com></code>将产生连接:<<a href="http://www.baidu.com>" target="_blank" rel="noopener">www.baidu.com></a>.</p><p>如果是标准的url，则会自动产生连接，例如:<a href="http://www.google.com" target="_blank" rel="noopener">www.google.com</a></p><h3 id="目录列表Table-of-Contents（TOC）"><a href="#目录列表Table-of-Contents（TOC）" class="headerlink" title="目录列表Table of Contents（TOC）"></a>目录列表Table of Contents（TOC）</h3><p>输入[toc]然后回车，将会产生一个目录，这个目录抽取了文章的所有标题，自动更新内容。</p><h3 id="水平分割线"><a href="#水平分割线" class="headerlink" title="水平分割线"></a>水平分割线</h3><p>使用<code>***</code>或者<code>---</code>，然后回车，来产生水平分割线。</p><hr><h3 id="标注"><a href="#标注" class="headerlink" title="标注"></a>标注</h3><p>我们可以对某一个词语进行标注。例如</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">某些人用过了才知道[^注释]</span><br><span class="line">[^注释]:Somebody that I used to know.</span><br></pre></td></tr></table></figure><p>将产生：</p><p>某些人用过了才知道[^注释]<br>[^注释]: Somebody that I used to know.</p><p>把鼠标放在<code>注释</code>上，将会有提示内容。</p><h3 id="表格"><a href="#表格" class="headerlink" title="表格"></a>表格</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">|姓名|性别|毕业学校|工资|</span><br><span class="line">|:---|:---:|:---:|---:|</span><br><span class="line">|杨洋|男|重庆交通大学|3200|</span><br><span class="line">|峰哥|男|贵州大学|5000|</span><br><span class="line">|坑货|女|北京大学|2000|</span><br></pre></td></tr></table></figure><p>将产生:</p><table><thead><tr><th align="left">姓名</th><th align="center">性别</th><th align="center">毕业学校</th><th align="right">工资</th></tr></thead><tbody><tr><td align="left">杨洋</td><td align="center">男</td><td align="center">重庆交通大学</td><td align="right">3200</td></tr><tr><td align="left">峰哥</td><td align="center">男</td><td align="center">贵州大学</td><td align="right">5000</td></tr><tr><td align="left">坑货</td><td align="center">女</td><td align="center">北京大学</td><td align="right">2000</td></tr><tr><td align="left">其中代码的第二行指定对齐的方式，第一个是左对齐，第二个和第三个是居中，最后一个是右对齐。</td><td align="center"></td><td align="center"></td><td align="right"></td></tr></tbody></table><h3 id="数学表达式块"><a href="#数学表达式块" class="headerlink" title="数学表达式块"></a>数学表达式块</h3><p>输入两个美元符号，然后回车，就可以输入数学表达式块了。例如：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$$\mathbf{V}_1 \times \mathbf{V}_2 = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\\frac{\partial X}{\partial u} & \frac{\partial Y}{\partial u} & 0 \\\frac{\partial X}{\partial v} & \frac{\partial Y}{\partial v} & 0 \\\end{vmatrix}$$</span><br></pre></td></tr></table></figure><p>将会产生:</p><p> $$\mathbf{V}_1 \times \mathbf{V}_2 = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\frac{\partial X}{\partial u} & \frac{\partial Y}{\partial u} & 0 \\frac{\partial X}{\partial v} & \frac{\partial Y}{\partial v} & 0 \\end{vmatrix}$$</p><h3 id="任务列表"><a href="#任务列表" class="headerlink" title="任务列表"></a>任务列表</h3><p>使用如下的代码创建任务列表，在[]中输入x表示完成，也可以通过点击选择完成或者没完成。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">- [ ] 吃饭</span><br><span class="line">- [ ] 逛街</span><br><span class="line">- [ ] 看电影</span><br><span class="line">- [ ] 约泡</span><br></pre></td></tr></table></figure><ul><li><input checked="" disabled="" type="checkbox"> <p>吃饭</p><pre><code></code></pre></li><li><input checked="" disabled="" type="checkbox"> <p>逛街</p><pre><code></code></pre></li><li><input checked="" disabled="" type="checkbox"> <p>看电影</p><pre><code></code></pre></li><li><input checked="" disabled="" type="checkbox"> <p>约泡</p></li></ul><h3 id="列表"><a href="#列表" class="headerlink" title="列表"></a>列表</h3><p>输入+, -, *,创建无序的列表，使用任意数字开头，创建有序列表，例如：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">**无序的列表**</span><br><span class="line">* tfboys</span><br><span class="line">* 杨洋</span><br><span class="line">* 我爱你</span><br></pre></td></tr></table></figure><p><strong>无序的列表</strong></p><ul><li>tfboys</li><li>杨洋</li><li>我爱你</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">**有序的列表**</span><br><span class="line">1. 苹果</span><br><span class="line">6. 香蕉</span><br><span class="line">10. 我都不喜欢</span><br></pre></td></tr></table></figure><p><strong>有序的列表</strong></p><ol><li>苹果</li><li>香蕉</li><li>我都不喜欢</li></ol><h3 id="块引用"><a href="#块引用" class="headerlink" title="块引用"></a>块引用</h3><p>使用>来插入块引用。例如：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">>这是一个块引用！</span><br></pre></td></tr></table></figure><p>将产生：</p><blockquote><p>这是一个块引用！</p></blockquote><h3 id="标题"><a href="#标题" class="headerlink" title="标题"></a>标题</h3><p>使用#表示一级标题，##表示二级标题，以此类推，有6个标题。</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
</entry>
<entry>
<title>2019-02-27-idea下的配置springboot的热加载</title>
<link href="https://blog.collmall.com/2020/ck91gq88i0004ygta82i7aua3/"/>
<id>https://blog.collmall.com/2020/ck91gq88i0004ygta82i7aua3/</id>
<published>2020-04-15T10:32:04.067Z</published>
<updated>2019-03-04T06:48:09.729Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h2 id="idea下配置springboot热加载"><a href="#idea下配置springboot热加载" class="headerlink" title="idea下配置springboot热加载"></a>idea下配置springboot热加载</h2><h4 id="1-添加pom依赖"><a href="#1-添加pom依赖" class="headerlink" title="1.添加pom依赖"></a>1.添加pom依赖</h4><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"><!-- SpringBoot自带热加载开发工具 --></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-devtools<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>runtime<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure><h4 id="2-添加配置"><a href="#2-添加配置" class="headerlink" title="2. 添加配置"></a>2. 添加配置</h4><p>打开 File –> Settings –> Build-Execution-Deployment –> Compiler<code>，将</code>Build project automatically.勾上。</p><p><img src="../img/post-idea-setting-1.png" alt="post-idea-setting-1"></p><h4 id="3"><a href="#3" class="headerlink" title="3 ."></a>3 .</h4><p>快捷键 ctrl + alt + shift + / 打开 Maintenance 找到 <code>Registry...</code>，将 其中的 <code>compiler.automake.allow.when.app.running</code>勾上。</p><p><img src="../img/post-idea-setting-2.png" alt="post-idea-setting-2"></p><p><img src="../img/post-idea-setting-3.png" alt="post-idea-setting-3"></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
</entry>
<entry>
<title>Hello World</title>
<link href="https://blog.collmall.com/2020/ck91gq8bo006sygtagvg51lar/"/>
<id>https://blog.collmall.com/2020/ck91gq8bo006sygtagvg51lar/</id>
<published>2020-04-15T09:05:27.263Z</published>
<updated>2020-04-14T15:53:19.223Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><p>Welcome to <a href="https://hexo.io/" target="_blank" rel="noopener">Hexo</a>! This is your very first post. Check <a href="https://hexo.io/docs/" target="_blank" rel="noopener">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a href="https://hexo.io/docs/troubleshooting.html" target="_blank" rel="noopener">troubleshooting</a> or you can ask me on <a href="https://github.com/hexojs/hexo/issues" target="_blank" rel="noopener">GitHub</a>.</p><h2 id="Quick-Start"><a href="#Quick-Start" class="headerlink" title="Quick Start"></a>Quick Start</h2><h3 id="Create-a-new-post"><a href="#Create-a-new-post" class="headerlink" title="Create a new post"></a>Create a new post</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">"My New Post"</span></span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/writing.html" target="_blank" rel="noopener">Writing</a></p><h3 id="Run-server"><a href="#Run-server" class="headerlink" title="Run server"></a>Run server</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo server</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/server.html" target="_blank" rel="noopener">Server</a></p><h3 id="Generate-static-files"><a href="#Generate-static-files" class="headerlink" title="Generate static files"></a>Generate static files</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo generate</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/generating.html" target="_blank" rel="noopener">Generating</a></p><h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerlink" title="Deploy to remote sites"></a>Deploy to remote sites</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/one-command-deployment.html" target="_blank" rel="noopener">Deployment</a></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
</entry>
<entry>
<title>Markdown测试</title>
<link href="https://blog.collmall.com/2020/ck91gq8bs0070ygta80dv8xuu/"/>
<id>https://blog.collmall.com/2020/ck91gq8bs0070ygta80dv8xuu/</id>
<published>2020-04-05T09:07:11.000Z</published>
<updated>2020-04-15T13:07:10.637Z</updated>
<content type="html">< 为文字增加外链接。</p><p>示例：</p><p>这是去往 <a href="http://Shen-Yu.github.com" target="_blank" rel="noopener">本人博客</a> 的链接。</p><p><img src="https://n1-q.mafengwo.net/s15/M00/65/BB/CoUBGV3brtOAMKOdAAsHBZ8nyQk791.jpg?imageMogr2%2Fthumbnail%2F1360x%2Fstrip%2Fquality%2F90" alt=""></p><h3 id="4-无序列表"><a href="#4-无序列表" class="headerlink" title="4. 无序列表"></a>4. 无序列表</h3><p>使用 *，+，- 表示无序列表。</p><p>示例：</p><ul><li>无序列表项 一</li><li>无序列表项 二</li><li>无序列表项 三</li></ul><h3 id="5-有序列表"><a href="#5-有序列表" class="headerlink" title="5. 有序列表"></a>5. 有序列表</h3><p>使用数字和点表示有序列表。</p><p>示例：</p><ol><li>有序列表项 一</li><li>有序列表项 二</li><li>有序列表项 三</li></ol><h3 id="6-文字引用"><a href="#6-文字引用" class="headerlink" title="6. 文字引用"></a>6. 文字引用</h3><p>使用 > 表示文字引用。</p><p>示例：</p><blockquote><p>野火烧不尽，春风吹又生。</p></blockquote><h3 id="7-行内代码块"><a href="#7-行内代码块" class="headerlink" title="7. 行内代码块"></a>7. 行内代码块</h3><p>使用 `代码` 表示行内代码块。</p><p>示例：</p><p>让我们聊聊 <code>html</code>。</p><h3 id="8-代码块"><a href="#8-代码块" class="headerlink" title="8. 代码块"></a>8. 代码块</h3><p>使用 四个缩进空格 表示代码块。</p><p>示例：</p><pre><code>这是一个代码块，此行左侧有四个不可见的空格。</code></pre><h3 id="9-插入图像"><a href="#9-插入图像" class="headerlink" title="9. 插入图像"></a>9. 插入图像</h3><p>使用  插入图像。</p><p>示例：</p><p><img src="https://www.zybuluo.com/static/img/my_head.jpg" alt="我的头像"></p><h1 id="Cmd-Markdown-高阶语法手册"><a href="#Cmd-Markdown-高阶语法手册" class="headerlink" title="Cmd Markdown 高阶语法手册"></a>Cmd Markdown 高阶语法手册</h1><h3 id="1-内容目录"><a href="#1-内容目录" class="headerlink" title="1. 内容目录"></a>1. 内容目录</h3><p>在段落中填写 <code>[TOC]</code> 以显示全文内容的目录结构。</p><p>[TOC]</p><h3 id="2-标签分类"><a href="#2-标签分类" class="headerlink" title="2. 标签分类"></a>2. 标签分类</h3><p>在编辑区任意行的列首位置输入以下代码给文稿标签：</p><p>标签： 数学 英语 Markdown</p><p>或者</p><p>Tags： 数学 英语 Markdown</p><h3 id="3-删除线"><a href="#3-删除线" class="headerlink" title="3. 删除线"></a>3. 删除线</h3><p>使用 ~~ 表示删除线。</p><p><del>这是一段错误的文本。</del></p><h3 id="4-注脚"><a href="#4-注脚" class="headerlink" title="4. 注脚"></a>4. 注脚</h3><p>使用 [^keyword] 表示注脚。</p><p>这是一个注脚[^footnote]的样例。</p><p>这是第二个注脚[^footnote2]的样例。</p><h3 id="5-LaTeX-公式"><a href="#5-LaTeX-公式" class="headerlink" title="5. LaTeX 公式"></a>5. LaTeX 公式</h3><p>$ 表示行内公式： </p><p>质能守恒方程可以用一个很简洁的方程式 $E=mc^2$ 来表达。</p><p>$$ 表示整行公式：</p><p>$$\sum_{i=1}^n a_i=0$$</p><p>$$f(x_1,x_x,\ldots,x_n) = x_1^2 + x_2^2 + \cdots + x_n^2 $$</p><p>$$\sum^{j-1}<em>{k=0}{\widehat{\gamma}</em>{kj} z_k}$$</p><p>访问 <a href="http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference" target="_blank" rel="noopener">MathJax</a> 参考更多使用方法。</p><h3 id="6-加强的代码块"><a href="#6-加强的代码块" class="headerlink" title="6. 加强的代码块"></a>6. 加强的代码块</h3><p>支持四十一种编程语言的语法高亮的显示，行号显示。</p><p>非代码示例：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-get install vim-gnome</span><br></pre></td></tr></table></figure><p>Python 示例：</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@requires_authorization</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">somefunc</span><span class="params">(param1=<span class="string">''</span>, param2=<span class="number">0</span>)</span>:</span></span><br><span class="line"> <span class="string">'''A docstring'''</span></span><br><span class="line"> <span class="keyword">if</span> param1 > param2: <span class="comment"># interesting</span></span><br><span class="line"> <span class="keyword">print</span> <span class="string">'Greater'</span></span><br><span class="line"> <span class="keyword">return</span> (param2 - param1 + <span class="number">1</span>) <span class="keyword">or</span> <span class="literal">None</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">SomeClass</span>:</span></span><br><span class="line"> <span class="keyword">pass</span></span><br><span class="line"></span><br><span class="line"><span class="meta">>>> </span>message = <span class="string">'''interpreter</span></span><br><span class="line"><span class="string"><span class="meta">... </span>prompt'''</span></span><br></pre></td></tr></table></figure><p>JavaScript 示例：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">* nth element in the fibonacci series.</span></span><br><span class="line"><span class="comment">* <span class="doctag">@param </span>n >= 0</span></span><br><span class="line"><span class="comment">* <span class="doctag">@return </span>the nth element, >= 0.</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">fib</span>(<span class="params">n</span>) </span>{</span><br><span class="line"> <span class="keyword">var</span> a = <span class="number">1</span>, b = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">var</span> tmp;</span><br><span class="line"> <span class="keyword">while</span> (--n >= <span class="number">0</span>) {</span><br><span class="line"> tmp = a;</span><br><span class="line"> a += b;</span><br><span class="line"> b = tmp;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> a;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="built_in">document</span>.write(fib(<span class="number">10</span>));</span><br></pre></td></tr></table></figure><h3 id="7-流程图"><a href="#7-流程图" class="headerlink" title="7. 流程图"></a>7. 流程图</h3><h4 id="示例"><a href="#示例" class="headerlink" title="示例"></a>示例</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">st=>start: Start:>https://www.zybuluo.com</span><br><span class="line">io=>inputoutput: verification</span><br><span class="line">op=>operation: Your Operation</span><br><span class="line">cond=>condition: Yes or No?</span><br><span class="line">sub=>subroutine: Your Subroutine</span><br><span class="line">e=>end</span><br><span class="line"></span><br><span class="line">st->io->op->cond</span><br><span class="line">cond(yes)->e</span><br><span class="line">cond(no)->sub->io</span><br></pre></td></tr></table></figure><h4 id="更多语法参考：流程图语法参考"><a href="#更多语法参考：流程图语法参考" class="headerlink" title="更多语法参考：流程图语法参考"></a>更多语法参考：<a href="http://adrai.github.io/flowchart.js/" target="_blank" rel="noopener">流程图语法参考</a></h4><h3 id="8-序列图"><a href="#8-序列图" class="headerlink" title="8. 序列图"></a>8. 序列图</h3><h4 id="示例-1"><a href="#示例-1" class="headerlink" title="示例 1"></a>示例 1</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">Alice->Bob: Hello Bob, how are you?</span><br><span class="line">Note right of Bob: Bob thinks</span><br><span class="line">Bob-->Alice: I am good thanks!</span><br></pre></td></tr></table></figure><h4 id="示例-2"><a href="#示例-2" class="headerlink" title="示例 2"></a>示例 2</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Title: Here is a title</span><br><span class="line">A->B: Normal line</span><br><span class="line">B-->C: Dashed line</span><br><span class="line">C->>D: Open arrow</span><br><span class="line">D-->>A: Dashed open arrow</span><br></pre></td></tr></table></figure><h4 id="更多语法参考：序列图语法参考"><a href="#更多语法参考：序列图语法参考" class="headerlink" title="更多语法参考：序列图语法参考"></a>更多语法参考：<a href="http://bramp.github.io/js-sequence-diagrams/" target="_blank" rel="noopener">序列图语法参考</a></h4><h3 id="9-甘特图"><a href="#9-甘特图" class="headerlink" title="9. 甘特图"></a>9. 甘特图</h3><p>甘特图内在思想简单。基本是一条线条图，横轴表示时间，纵轴表示活动（项目），线条表示在整个期间上计划和实际的活动完成情况。它直观地表明任务计划在什么时候进行，及实际进展与计划要求的对比。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">title 项目开发流程</span><br><span class="line">section 项目确定</span><br><span class="line"> 需求分析 :a1, 2016-06-22, 3d</span><br><span class="line"> 可行性报告 :after a1, 5d</span><br><span class="line"> 概念验证 : 5d</span><br><span class="line">section 项目实施</span><br><span class="line"> 概要设计 :2016-07-05 , 5d</span><br><span class="line"> 详细设计 :2016-07-08, 10d</span><br><span class="line"> 编码 :2016-07-15, 10d</span><br><span class="line"> 测试 :2016-07-22, 5d</span><br><span class="line">section 发布验收</span><br><span class="line"> 发布: 2d</span><br><span class="line"> 验收: 3d</span><br></pre></td></tr></table></figure><h4 id="更多语法参考：甘特图语法参考"><a href="#更多语法参考：甘特图语法参考" class="headerlink" title="更多语法参考：甘特图语法参考"></a>更多语法参考：<a href="https://knsv.github.io/mermaid/#gant-diagrams" target="_blank" rel="noopener">甘特图语法参考</a></h4><h3 id="10-Mermaid-流程图"><a href="#10-Mermaid-流程图" class="headerlink" title="10. Mermaid 流程图"></a>10. Mermaid 流程图</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">A[Hard edge] -->|Link text| B(Round edge)</span><br><span class="line">B --> C{Decision}</span><br><span class="line">C -->|One| D[Result one]</span><br><span class="line">C -->|Two| E[Result two]</span><br></pre></td></tr></table></figure><h4 id="更多语法参考：Mermaid-流程图语法参考"><a href="#更多语法参考：Mermaid-流程图语法参考" class="headerlink" title="更多语法参考：Mermaid 流程图语法参考"></a>更多语法参考：<a href="https://knsv.github.io/mermaid/#flowcharts-basic-syntax" target="_blank" rel="noopener">Mermaid 流程图语法参考</a></h4><h3 id="11-Mermaid-序列图"><a href="#11-Mermaid-序列图" class="headerlink" title="11. Mermaid 序列图"></a>11. Mermaid 序列图</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Alice->John: Hello John, how are you?</span><br><span class="line">loop every minute</span><br><span class="line"> John-->Alice: Great!</span><br><span class="line">end</span><br></pre></td></tr></table></figure><h4 id="更多语法参考：Mermaid-序列图语法参考"><a href="#更多语法参考：Mermaid-序列图语法参考" class="headerlink" title="更多语法参考：Mermaid 序列图语法参考"></a>更多语法参考：<a href="https://knsv.github.io/mermaid/#sequence-diagrams" target="_blank" rel="noopener">Mermaid 序列图语法参考</a></h4><h3 id="12-表格支持"><a href="#12-表格支持" class="headerlink" title="12. 表格支持"></a>12. 表格支持</h3><table><thead><tr><th>项目</th><th align="right">价格</th><th align="center">数量</th></tr></thead><tbody><tr><td>计算机</td><td align="right">$1600</td><td align="center">5</td></tr><tr><td>手机</td><td align="right">$12</td><td align="center">12</td></tr><tr><td>管线</td><td align="right">$1</td><td align="center">234</td></tr></tbody></table><h3 id="13-定义型列表"><a href="#13-定义型列表" class="headerlink" title="13. 定义型列表"></a>13. 定义型列表</h3><dl><dt>名词 1</dt><dd>定义 1（左侧有一个可见的冒号和四个不可见的空格）</dd></dl><dl><dt>代码块 2</dt><dd>这是代码块的定义（左侧有一个可见的冒号和四个不可见的空格）</dd></dl><pre><code>代码块（左侧有八个不可见的空格）</code></pre><h3 id="14-Html-标签"><a href="#14-Html-标签" class="headerlink" title="14. Html 标签"></a>14. Html 标签</h3><p>本站支持在 Markdown 语法中嵌套 Html 标签，譬如，你可以用 Html 写一个纵跨两行的表格：</p><pre><code><table> <tr> <th rowspan="2">值班人员</th> <th>星期一</th> <th>星期二</th> <th>星期三</th> </tr> <tr> <td>李强</td> <td>张明</td> <td>王平</td> </tr></table></code></pre><table> <tr> <th rowspan="2">值班人员</th> <th>星期一</th> <th>星期二</th> <th>星期三</th> </tr> <tr> <td>李强</td> <td>张明</td> <td>王平</td> </tr></table><h3 id="15-内嵌图标"><a href="#15-内嵌图标" class="headerlink" title="15. 内嵌图标"></a>15. 内嵌图标</h3><p>本站的图标系统对外开放，在文档中输入</p><pre><code><i class="icon-weibo"></i></code></pre><p>即显示微博的图标： <i class="icon-weibo icon-2x"></i></p><p>替换 上述 <code>i 标签</code> 内的 <code>icon-weibo</code> 以显示不同的图标，例如：</p><pre><code><i class="icon-renren"></i></code></pre><p>即显示人人的图标： <i class="icon-renren icon-2x"></i></p><p>更多的图标和玩法可以参看 <a href="http://fortawesome.github.io/Font-Awesome/3.2.1/icons/" target="_blank" rel="noopener">font-awesome</a> 官方网站。</p><h3 id="16-待办事宜-Todo-列表"><a href="#16-待办事宜-Todo-列表" class="headerlink" title="16. 待办事宜 Todo 列表"></a>16. 待办事宜 Todo 列表</h3><p>使用带有 [ ] 或 [x] （未完成或已完成）项的列表语法撰写一个待办事宜列表，并且支持子列表嵌套以及混用Markdown语法，例如：</p><pre><code>- [ ] **Cmd Markdown 开发** - [ ] 改进 Cmd 渲染算法，使用局部渲染技术提高渲染效率 - [ ] 支持以 PDF 格式导出文稿 - [x] 新增Todo列表功能 [语法参考](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments) - [x] 改进 LaTex 功能 - [x] 修复 LaTex 公式渲染问题 - [x] 新增 LaTex 公式编号功能 [语法参考](http://docs.mathjax.org/en/latest/tex.html#tex-eq-numbers)- [ ] **七月旅行准备** - [ ] 准备邮轮上需要携带的物品 - [ ] 浏览日本免税店的物品 - [x] 购买蓝宝石公主号七月一日的船票</code></pre><p>对应显示如下待办事宜 Todo 列表：<br> </p><ul><li><input disabled="" type="checkbox"> <strong>Cmd Markdown 开发</strong><ul><li><input disabled="" type="checkbox"> 改进 Cmd 渲染算法，使用局部渲染技术提高渲染效率</li><li><input disabled="" type="checkbox"> 支持以 PDF 格式导出文稿</li><li><input checked="" disabled="" type="checkbox"> 新增Todo列表功能 <a href="https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments" target="_blank" rel="noopener">语法参考</a></li><li><input checked="" disabled="" type="checkbox"> 改进 LaTex 功能<ul><li><input checked="" disabled="" type="checkbox"> 修复 LaTex 公式渲染问题</li><li><input checked="" disabled="" type="checkbox"> 新增 LaTex 公式编号功能 <a href="http://docs.mathjax.org/en/latest/tex.html#tex-eq-numbers" target="_blank" rel="noopener">语法参考</a></li></ul></li></ul></li><li><input disabled="" type="checkbox"> <strong>七月旅行准备</strong><ul><li><input disabled="" type="checkbox"> 准备邮轮上需要携带的物品</li><li><input disabled="" type="checkbox"> 浏览日本免税店的物品</li><li><input checked="" disabled="" type="checkbox"> 购买蓝宝石公主号七月一日的船票</li></ul></li></ul><p>[^footnote]: 这是一个 <em>注脚</em> 的 <strong>文本</strong>。</p><p>[^footnote2]: 这是另一个 <em>注脚</em> 的 <strong>文本</strong>。</p>]]></content>
<summary type="html">
<h1 id="一级标题"><a href="#一级标题" class="headerlink" title="一级标题"></a>一级标题</h1><hr>
<h3 id="1-斜体和粗体"><a href="#1-斜体和粗体" class="headerlink" title="1. 斜体和粗体"></a>1. 斜体和粗体</h3><p>使用 * 和 ** 表示斜体和粗体。</p>
<p>示例：</p>
<p>这是 <em>斜体</em>，这是 <strong>粗体</strong>。</p>
</summary>
<category term="测试" scheme="https://blog.collmall.com/categories/%E6%B5%8B%E8%AF%95/"/>
<category term="测试" scheme="https://blog.collmall.com/tags/%E6%B5%8B%E8%AF%95/"/>
</entry>
<entry>
<title>基于Java对员工信息进行聚合分析</title>
<link href="https://blog.collmall.com/2020/ck91gq8bn006pygtacabg5u2y/"/>
<id>https://blog.collmall.com/2020/ck91gq8bn006pygtacabg5u2y/</id>
<published>2020-03-07T09:07:11.000Z</published>
<updated>2020-04-15T13:31:28.919Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="基于Java对员工信息进行聚合分析"><a href="#基于Java对员工信息进行聚合分析" class="headerlink" title="基于Java对员工信息进行聚合分析"></a>基于Java对员工信息进行聚合分析</h4><p>API 语法:</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">SearchResponse sr = node.client().prepareSearch()</span><br><span class="line"> .addAggregation(</span><br><span class="line"> AggregationBuilders.terms(<span class="string">"by_country"</span>).field(<span class="string">"country"</span>)</span><br><span class="line"> .subAggregation(AggregationBuilders.dateHistogram(<span class="string">"by_year"</span>)</span><br><span class="line"> .field(<span class="string">"dateOfBirth"</span>)</span><br><span class="line"> .dateHistogramInterval(DateHistogramInterval.YEAR)</span><br><span class="line"> .subAggregation(AggregationBuilders.avg(<span class="string">"avg_children"</span>).field(<span class="string">"children"</span>))</span><br><span class="line"> )</span><br><span class="line"> )</span><br><span class="line"> .execute().actionGet();</span><br></pre></td></tr></table></figure><p>我们先给个需求：</p><p>（1）首先按照country国家来进行分组<br>（2）然后在每个country分组内，再按照入职年限进行分组<br>（3）最后计算每个分组内的平均薪资</p><p>PUT /company<br>{<br> “mappings”: {<br> “employee”: {<br> “properties”: {<br> “age”: {<br> “type”: “long”<br> },<br> “country”: {<br> “type”: “text”,<br> “fields”: {<br> “keyword”: {<br> “type”: “keyword”,<br> “ignore_above”: 256<br> }<br> },<br> “fielddata”: true<br> },<br> “join_date”: {<br> “type”: “date”<br> },<br> “name”: {<br> “type”: “text”,<br> “fields”: {<br> “keyword”: {<br> “type”: “keyword”,<br> “ignore_above”: 256<br> }<br> }<br> },<br> “position”: {<br> “type”: “text”,<br> “fields”: {<br> “keyword”: {<br> “type”: “keyword”,<br> “ignore_above”: 256<br> }<br> }<br> },<br> “salary”: {<br> “type”: “long”<br> }<br> }<br> }<br> }<br>}</p><p>GET /company/employee/_search<br>{<br> “size”: 0,<br> “aggs”: {<br> “group_by_country”: {<br> “terms”: {<br> “field”: “country”<br> },<br> “aggs”: {<br> “group_by_join_date”: {<br> “date_histogram”: {<br> “field”: “join_date”,<br> “interval”: “year”<br> },<br> “aggs”: {<br> “avg_salary”: {<br> “avg”: {<br> “field”: “salary”<br> }<br> }<br> }<br> }<br> }<br> }<br> }<br>}</p><p> </p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">Map<String, Aggregation> aggrMap = searchResponse.getAggregations().asMap();</span><br><span class="line">StringTerms groupByCountry = (StringTerms) aggrMap.get(<span class="string">"group_by_country"</span>);</span><br><span class="line">Iterator<Bucket> groupByCountryBucketIterator = groupByCountry.getBuckets().iterator();</span><br><span class="line"><span class="keyword">while</span>(groupByCountryBucketIterator.hasNext()) {</span><br><span class="line">Bucket groupByCountryBucket = groupByCountryBucketIterator.next();</span><br><span class="line"></span><br><span class="line">System.out.println(groupByCountryBucket.getKey() + <span class="string">"\t"</span> + groupByCountryBucket.getDocCount()); </span><br><span class="line"></span><br><span class="line">Histogram groupByJoinDate = (Histogram) groupByCountryBucket.getAggregations().asMap().get(<span class="string">"group_by_join_date"</span>); </span><br><span class="line">Iterator<org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket> groupByJoinDateBucketIterator = groupByJoinDate.getBuckets().iterator();</span><br><span class="line"> </span><br><span class="line"><span class="keyword">while</span>(groupByJoinDateBucketIterator.hasNext()) {</span><br><span class="line">org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket groupByJoinDateBucket = groupByJoinDateBucketIterator.next();</span><br><span class="line"></span><br><span class="line">System.out.println(groupByJoinDateBucket.getKey() + <span class="string">"\t"</span> + groupByJoinDateBucket.getDocCount()); </span><br><span class="line"></span><br><span class="line">Avg avgSalary = (Avg) groupByJoinDateBucket.getAggregations().asMap().get(<span class="string">"avg_salary"</span>);</span><br><span class="line">System.out.println(avgSalary.getValue()); </span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">client.close();</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>代码示例:</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.net.InetAddress;</span><br><span class="line"><span class="keyword">import</span> java.util.Iterator;</span><br><span class="line"><span class="keyword">import</span> java.util.Map;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.search.SearchResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.client.transport.TransportClient;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.settings.Settings;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.transport.InetSocketTransportAddress;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.Aggregation;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.AggregationBuilders;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramInterval;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.bucket.histogram.Histogram;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.bucket.terms.StringTerms;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.bucket.terms.Terms.Bucket;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.aggregations.metrics.avg.Avg;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.transport.client.PreBuiltTransportClient;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 员工聚合分析应用程序</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> Administrator</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">EmployeeAggrApp</span> </span>{</span><br><span class="line"></span><br><span class="line"><span class="meta">@SuppressWarnings</span>({ <span class="string">"unchecked"</span>, <span class="string">"resource"</span> })</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">Settings settings = Settings.builder()</span><br><span class="line">.put(<span class="string">"cluster.name"</span>, <span class="string">"elasticsearch"</span>)</span><br><span class="line">.build();</span><br><span class="line"></span><br><span class="line">TransportClient client = <span class="keyword">new</span> PreBuiltTransportClient(settings)</span><br><span class="line">.addTransportAddress(<span class="keyword">new</span> InetSocketTransportAddress(InetAddress.getByName(<span class="string">"localhost"</span>), <span class="number">9300</span>)); </span><br><span class="line"></span><br><span class="line">SearchResponse searchResponse = client.prepareSearch(<span class="string">"company"</span>) </span><br><span class="line">.addAggregation(AggregationBuilders.terms(<span class="string">"group_by_country"</span>).field(<span class="string">"country"</span>)</span><br><span class="line">.subAggregation(AggregationBuilders</span><br><span class="line">.dateHistogram(<span class="string">"group_by_join_date"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>)</span><br><span class="line">.dateHistogramInterval(DateHistogramInterval.YEAR)</span><br><span class="line">.subAggregation(AggregationBuilders.avg(<span class="string">"avg_salary"</span>).field(<span class="string">"salary"</span>)))</span><br><span class="line">)</span><br><span class="line">.execute().actionGet();</span><br><span class="line"></span><br><span class="line">Map<String, Aggregation> aggrMap = searchResponse.getAggregations().asMap();</span><br><span class="line"></span><br><span class="line">StringTerms groupByCountry = (StringTerms) aggrMap.get(<span class="string">"group_by_country"</span>);</span><br><span class="line">Iterator<Bucket> groupByCountryBucketIterator = groupByCountry.getBuckets().iterator();</span><br><span class="line"><span class="keyword">while</span>(groupByCountryBucketIterator.hasNext()) {</span><br><span class="line">Bucket groupByCountryBucket = groupByCountryBucketIterator.next();</span><br><span class="line">System.out.println(groupByCountryBucket.getKey() + <span class="string">":"</span> + groupByCountryBucket.getDocCount()); </span><br><span class="line"></span><br><span class="line">Histogram groupByJoinDate = (Histogram) groupByCountryBucket.getAggregations().asMap().get(<span class="string">"group_by_join_date"</span>);</span><br><span class="line">Iterator<org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket> groupByJoinDateBucketIterator = groupByJoinDate.getBuckets().iterator();</span><br><span class="line"><span class="keyword">while</span>(groupByJoinDateBucketIterator.hasNext()) {</span><br><span class="line">org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket groupByJoinDateBucket = groupByJoinDateBucketIterator.next();</span><br><span class="line">System.out.println(groupByJoinDateBucket.getKey() + <span class="string">":"</span> +groupByJoinDateBucket.getDocCount()); </span><br><span class="line"></span><br><span class="line">Avg avg = (Avg) groupByJoinDateBucket.getAggregations().asMap().get(<span class="string">"avg_salary"</span>); </span><br><span class="line">System.out.println(avg.getValue()); </span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">client.close();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>基于Java对员工信息进行复杂的搜索操作</title>
<link href="https://blog.collmall.com/2020/ck91gq8bk006jygta83ho0w3n/"/>
<id>https://blog.collmall.com/2020/ck91gq8bk006jygta83ho0w3n/</id>
<published>2020-03-06T09:07:11.000Z</published>
<updated>2020-04-15T13:30:57.379Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="基于Java对员工信息进行复杂的搜索操作"><a href="#基于Java对员工信息进行复杂的搜索操作" class="headerlink" title="基于Java对员工信息进行复杂的搜索操作"></a>基于Java对员工信息进行复杂的搜索操作</h4><p>API:</p><p>SearchResponse response = client.prepareSearch(“index1”, “index2”)<br> .setTypes(“type1”, “type2”)<br> .setQuery(QueryBuilders.termQuery(“multi”, “test”)) // Query<br> .setPostFilter(QueryBuilders.rangeQuery(“age”).from(12).to(18)) // Filter<br> .setFrom(0).setSize(60)<br> .get();</p><p>需求：</p><p>（1）搜索职位中包含technique的员工<br>（2）同时要求age在30到40岁之间<br>（3）分页查询，查找第一页</p><p>GET /company/employee/_search<br>{<br> “query”: {<br> “bool”: {<br> “must”: [<br> {<br> “match”: {<br> “position”: “technique”<br> }<br> }<br> ],<br> “filter”: {<br> “range”: {<br> “age”: {<br> “gte”: 30,<br> “lte”: 40<br> }<br> }<br> }<br> }<br> },<br> “from”: 0,<br> “size”: 1<br>}</p><p>刚才一边运行创建document，一边搜索什么都没搜索到？？？？</p><p>近实时！！！</p><p>默认是1秒以后，写入es的数据，才能被搜索到。很明显刚才，写入数据不到一秒，所以就搜索不到。</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.net.InetAddress;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.search.SearchResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.client.transport.TransportClient;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.settings.Settings;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.transport.InetSocketTransportAddress;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.xcontent.XContentFactory;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.index.query.QueryBuilders;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.search.SearchHit;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.transport.client.PreBuiltTransportClient;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 员工搜索应用程序</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> Administrator</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">EmployeeSearchApp</span> </span>{</span><br><span class="line"></span><br><span class="line"><span class="meta">@SuppressWarnings</span>({ <span class="string">"unchecked"</span>, <span class="string">"resource"</span> })</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">Settings settings = Settings.builder()</span><br><span class="line">.put(<span class="string">"cluster.name"</span>, <span class="string">"elasticsearch"</span>)</span><br><span class="line">.build();</span><br><span class="line"></span><br><span class="line">TransportClient client = <span class="keyword">new</span> PreBuiltTransportClient(settings)</span><br><span class="line">.addTransportAddress(<span class="keyword">new</span> InetSocketTransportAddress(InetAddress.getByName(<span class="string">"localhost"</span>), <span class="number">9300</span>)); </span><br><span class="line"></span><br><span class="line"><span class="comment">//prepareData(client);</span></span><br><span class="line">executeSearch(client);</span><br><span class="line"></span><br><span class="line">client.close();</span><br><span class="line"></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 执行搜索操作</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">executeSearch</span><span class="params">(TransportClient client)</span> </span>{</span><br><span class="line">SearchResponse response = client.prepareSearch(<span class="string">"company"</span>)</span><br><span class="line">.setTypes(<span class="string">"employee"</span>)</span><br><span class="line">.setQuery(QueryBuilders.matchQuery(<span class="string">"position"</span>, <span class="string">"technique"</span>))</span><br><span class="line">.setPostFilter(QueryBuilders.rangeQuery(<span class="string">"age"</span>).from(<span class="number">30</span>).to(<span class="number">40</span>))</span><br><span class="line">.setFrom(<span class="number">0</span>).setSize(<span class="number">1</span>)</span><br><span class="line">.get();</span><br><span class="line"></span><br><span class="line">SearchHit[] searchHits = response.getHits().getHits();</span><br><span class="line"><span class="keyword">for</span>(<span class="keyword">int</span> i = <span class="number">0</span>; i < searchHits.length; i++) {</span><br><span class="line">System.out.println(searchHits[i].getSourceAsString()); </span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 准备数据</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">prepareData</span><span class="params">(TransportClient client)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"1"</span>) </span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"jack"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">27</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"technique software"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"china"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2017-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">10000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line"></span><br><span class="line">client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"2"</span>) </span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"marry"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">35</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"technique manager"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"china"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2017-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">12000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line"></span><br><span class="line">client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"3"</span>) </span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"tom"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">32</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"senior technique software"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"china"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2016-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">11000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line"></span><br><span class="line">client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"4"</span>) </span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"jen"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">25</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"junior finance"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"usa"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2016-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">7000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line"></span><br><span class="line">client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"5"</span>) </span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"mike"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">37</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"finance manager"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"usa"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2015-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">15000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>基于Java实现员工信息的增删改查</title>
<link href="https://blog.collmall.com/2020/ck91gq8bm006mygtae9l7bmm6/"/>
<id>https://blog.collmall.com/2020/ck91gq8bm006mygtae9l7bmm6/</id>
<published>2020-03-05T09:07:11.000Z</published>
<updated>2020-04-15T13:30:21.998Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="基于Java实现员工信息的增删改查"><a href="#基于Java实现员工信息的增删改查" class="headerlink" title="基于Java实现员工信息的增删改查"></a>基于Java实现员工信息的增删改查</h4><h5 id="1、maven依赖"><a href="#1、maven依赖" class="headerlink" title="1、maven依赖"></a>1、maven依赖</h5><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><dependency></span><br><span class="line"> <groupId>org.elasticsearch.client</groupId></span><br><span class="line"> <artifactId>transport</artifactId></span><br><span class="line"> <version>5.2.2</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.apache.logging.log4j</groupId></span><br><span class="line"> <artifactId>log4j-api</artifactId></span><br><span class="line"> <version>2.7</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.apache.logging.log4j</groupId></span><br><span class="line"> <artifactId>log4j-core</artifactId></span><br><span class="line"> <version>2.7</version></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><p>创建log4j2的配置文件</p><p>log4j2.properties</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">appender.console.type = Console</span><br><span class="line">appender.console.name = console</span><br><span class="line">appender.console.layout.type = PatternLayout</span><br><span class="line"></span><br><span class="line">rootLogger.level = info</span><br><span class="line">rootLogger.appenderRef.console.ref = console</span><br></pre></td></tr></table></figure><h5 id="2、构建client"><a href="#2、构建client" class="headerlink" title="2、构建client"></a>2、构建client</h5><p>Settings settings = Settings.builder()<br> .put(“cluster.name”, “myClusterName”).build();<br>TransportClient client = new PreBuiltTransportClient(settings);</p><p>TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)<br> .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(“host1”), 9300))<br> .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(“host2”), 9300));</p><p>client.close();</p><h5 id="3、创建document"><a href="#3、创建document" class="headerlink" title="3、创建document"></a>3、创建document</h5><p>IndexResponse response = client.prepareIndex(“index”, “type”, “1”)<br> .setSource(jsonBuilder()<br> .startObject()<br> .field(“user”, “kimchy”)<br> .field(“postDate”, new Date())<br> .field(“message”, “trying out Elasticsearch”)<br> .endObject()<br> )<br> .get();</p><h5 id="4、查询document"><a href="#4、查询document" class="headerlink" title="4、查询document"></a>4、查询document</h5><p>GetResponse response = client.prepareGet(“index”, “type”, “1”).get();</p><h5 id="5、修改document"><a href="#5、修改document" class="headerlink" title="5、修改document"></a>5、修改document</h5><p>client.prepareUpdate(“index”, “type”, “1”)<br> .setDoc(jsonBuilder()<br> .startObject()<br> .field(“gender”, “male”)<br> .endObject())<br> .get();</p><h5 id="6、删除document"><a href="#6、删除document" class="headerlink" title="6、删除document"></a>6、删除document</h5><p>DeleteResponse response = client.prepareDelete(“index”, “type”, “1”).get();</p><p>eg:</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.net.InetAddress;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.delete.DeleteResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.get.GetResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.index.IndexResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.action.update.UpdateResponse;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.client.transport.TransportClient;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.settings.Settings;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.transport.InetSocketTransportAddress;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.common.xcontent.XContentFactory;</span><br><span class="line"><span class="keyword">import</span> org.elasticsearch.transport.client.PreBuiltTransportClient;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 员工增删改查的应用程序</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> Administrator</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">EmployeeCRUDApp</span> </span>{</span><br><span class="line"></span><br><span class="line"><span class="meta">@SuppressWarnings</span>({ <span class="string">"unchecked"</span>, <span class="string">"resource"</span> })</span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line"><span class="comment">// 先构建client</span></span><br><span class="line">Settings settings = Settings.builder()</span><br><span class="line">.put(<span class="string">"cluster.name"</span>, <span class="string">"elasticsearch"</span>)</span><br><span class="line">.build();</span><br><span class="line"></span><br><span class="line">TransportClient client = <span class="keyword">new</span> PreBuiltTransportClient(settings)</span><br><span class="line">.addTransportAddress(<span class="keyword">new</span> InetSocketTransportAddress(InetAddress.getByName(<span class="string">"localhost"</span>), <span class="number">9300</span>));</span><br><span class="line"></span><br><span class="line"><span class="comment">//createEmployee(client); </span></span><br><span class="line"><span class="comment">//getEmployee(client);</span></span><br><span class="line"><span class="comment">//updateEmployee(client); </span></span><br><span class="line"><span class="comment">//deleteEmployee(client); </span></span><br><span class="line"></span><br><span class="line">client.close();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 创建员工信息（创建一个document）</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">createEmployee</span><span class="params">(TransportClient client)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">IndexResponse response = client.prepareIndex(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"1"</span>)</span><br><span class="line">.setSource(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"name"</span>, <span class="string">"jack"</span>)</span><br><span class="line">.field(<span class="string">"age"</span>, <span class="number">27</span>)</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"technique"</span>)</span><br><span class="line">.field(<span class="string">"country"</span>, <span class="string">"china"</span>)</span><br><span class="line">.field(<span class="string">"join_date"</span>, <span class="string">"2017-01-01"</span>)</span><br><span class="line">.field(<span class="string">"salary"</span>, <span class="number">10000</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line">System.out.println(response.getResult()); </span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 获取员工信息</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> Exception</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">getEmployee</span><span class="params">(TransportClient client)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">GetResponse response = client.prepareGet(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"1"</span>).get();</span><br><span class="line">System.out.println(response.getSourceAsString()); </span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 修改员工信息</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> Exception</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">updateEmployee</span><span class="params">(TransportClient client)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">UpdateResponse response = client.prepareUpdate(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"1"</span>) </span><br><span class="line">.setDoc(XContentFactory.jsonBuilder()</span><br><span class="line">.startObject()</span><br><span class="line">.field(<span class="string">"position"</span>, <span class="string">"technique manager"</span>)</span><br><span class="line">.endObject())</span><br><span class="line">.get();</span><br><span class="line">System.out.println(response.getResult()); </span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 删除 员工信息</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> client</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@throws</span> Exception</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">deleteEmployee</span><span class="params">(TransportClient client)</span> <span class="keyword">throws</span> Exception </span>{</span><br><span class="line">DeleteResponse response = client.prepareDelete(<span class="string">"company"</span>, <span class="string">"employee"</span>, <span class="string">"1"</span>).get();</span><br><span class="line">System.out.println(response.getResult()); </span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>写入流程实现海量磁盘文件合并（segment merge，optimize）</title>
<link href="https://blog.collmall.com/2020/ck91gq8bi006dygtagjxs1jjg/"/>
<id>https://blog.collmall.com/2020/ck91gq8bi006dygtagjxs1jjg/</id>
<published>2020-03-04T09:07:11.000Z</published>
<updated>2020-04-15T13:29:53.313Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="写入流程实现海量磁盘文件合并（segment-merge，optimize）"><a href="#写入流程实现海量磁盘文件合并（segment-merge，optimize）" class="headerlink" title="写入流程实现海量磁盘文件合并（segment merge，optimize）"></a>写入流程实现海量磁盘文件合并（segment merge，optimize）</h4><p>每秒一个segment file，文件过多，而且每次search都要搜索所有的segment，很耗时</p><p>默认会在后台执行segment merge操作，在merge的时候，被标记为deleted的document也会被彻底物理删除</p><p>每次merge操作的执行流程</p><p>（1）选择一些有相似大小的segment，merge成一个大的segment<br>（2）将新的segment flush到磁盘上去<br>（3）写一个新的commit point，包括了新的segment，并且排除旧的那些segment<br>（4）将新的segment打开供搜索<br>（5）将旧的segment删除</p><p>POST /my_index/_optimize?max_num_segments=1，尽量不要手动执行，让它自动默认执行就可以了</p><p><img src="https://guanyuoss.oss-cn-qingdao.aliyuncs.com/prod/work_order/ynUf-eiQ8O8.png" alt=""></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>实现durability可靠存储（translog，flush）</title>
<link href="https://blog.collmall.com/2020/ck91gq8bj006gygta7ssrd1db/"/>
<id>https://blog.collmall.com/2020/ck91gq8bj006gygta7ssrd1db/</id>
<published>2020-03-03T09:07:11.000Z</published>
<updated>2020-04-15T13:29:34.185Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="实现durability可靠存储（translog，flush）"><a href="#实现durability可靠存储（translog，flush）" class="headerlink" title="实现durability可靠存储（translog，flush）"></a>实现durability可靠存储（translog，flush）</h4><p>再次优化的写入流程</p><p>（1）数据写入buffer缓冲和translog日志文件<br>（2）每隔一秒钟，buffer中的数据被写入新的segment file，并进入os cache，此时segment被打开并供search使用<br>（3）buffer被清空<br>（4）重复1~3，新的segment不断添加，buffer不断被清空，而translog中的数据不断累加<br>（5）当translog长度达到一定程度的时候，commit操作发生<br> （5-1）buffer中的所有数据写入一个新的segment，并写入os cache，打开供使用<br> （5-2）buffer被清空<br> （5-3）一个commit ponit被写入磁盘，标明了所有的index segment<br> （5-4）filesystem cache中的所有index segment file缓存数据，被fsync强行刷到磁盘上<br> （5-5）现有的translog被清空，创建一个新的translog</p><p>基于translog和commit point，如何进行数据恢复</p><p>fsync+清空translog，就是flush，默认每隔30分钟flush一次，或者当translog过大的时候，也会触发flush操作</p><p>POST /my_index/_flush，一般来说别手动flush，让ES自动执行就可以了</p><p>translog，每隔5秒被fsync一次到磁盘上。在一次增删改操作之后，当fsync在primary shard和replica shard都成功之后，那次增删改操作才会成功</p><p>但是这种在一次增删改时强行fsync translog可能会导致部分操作比较耗时，也可以允许部分数据丢失，设置异步fsync translog</p><p>PUT /my_index/_settings<br>{<br> “index.translog.durability”: “async”,<br> “index.translog.sync_interval”: “5s”<br>}</p><p><img src="https://guanyuoss.oss-cn-qingdao.aliyuncs.com/prod/work_order/fdnQ4UZNY3M.png" alt=""></p><p>终极版本:</p><p><img src="https://guanyuoss.oss-cn-qingdao.aliyuncs.com/prod/work_order/ITIUgWavU08.png" alt=""></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>优化写入流程实现NRT近实时（filesystem cache，refresh）</title>
<link href="https://blog.collmall.com/2020/ck91gq8bi006aygta30p83ny5/"/>
<id>https://blog.collmall.com/2020/ck91gq8bi006aygta30p83ny5/</id>
<published>2020-03-02T09:07:11.000Z</published>
<updated>2020-04-15T13:28:53.186Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="优化写入流程实现NRT近实时（filesystem-cache，refresh）"><a href="#优化写入流程实现NRT近实时（filesystem-cache，refresh）" class="headerlink" title="优化写入流程实现NRT近实时（filesystem cache，refresh）"></a>优化写入流程实现NRT近实时（filesystem cache，refresh）</h4><p>现有流程的问题，每次都必须等待fsync将segment刷入磁盘，才能将segment打开供search使用，这样的话，从一个document写入，到它可以被搜索，可能会超过1分钟！！！这就不是近实时的搜索了！！！主要瓶颈在于fsync实际发生磁盘IO写数据进磁盘，是很耗时的。</p><p>写入流程别改进如下：</p><p>（1）数据写入buffer<br>（2）每隔一定时间，buffer中的数据被写入segment文件，但是先写入os cache<br>（3）只要segment写入os cache，那就直接打开供search使用，不立即执行commit</p><p>数据写入os cache，并被打开供搜索的过程，叫做refresh，默认是每隔1秒refresh一次。也就是说，每隔一秒就会将buffer中的数据写入一个新的index segment file，先写入os cache中。所以，es是近实时的，数据写入到可以被搜索，默认是1秒。</p><p>POST /my_index/_refresh，可以手动refresh，一般不需要手动执行，让es自己搞就可以了</p><p>比如说，我们现在的时效性要求，比较低，只要求一条数据写入es，一分钟以后才让我们搜索到就可以了，那么就可以调整refresh interval</p><p>PUT /my_index<br>{<br> “settings”: {<br> “refresh_interval”: “30s”<br> }<br>}</p><p><img src="https://guanyuoss.oss-cn-qingdao.aliyuncs.com/prod/work_order/9DvuSHDWyeA.png" alt=""></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>剖析document写入原理（buffer，segment，commit）</title>
<link href="https://blog.collmall.com/2020/ck91gq8bg0064ygta7o6fget7/"/>
<id>https://blog.collmall.com/2020/ck91gq8bg0064ygta7o6fget7/</id>
<published>2020-03-01T09:07:11.000Z</published>
<updated>2020-04-15T13:28:34.198Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="剖析document写入原理（buffer，segment，commit）"><a href="#剖析document写入原理（buffer，segment，commit）" class="headerlink" title="剖析document写入原理（buffer，segment，commit）"></a>剖析document写入原理（buffer，segment，commit）</h4><p>（1）数据写入buffer<br>（2）commit point<br>（3）buffer中的数据写入新的index segment<br>（4）等待在os cache中的index segment被fsync强制刷到磁盘上<br>（5）新的index sgement被打开，供search使用<br>（6）buffer被清空</p><p>每次commit point时，会有一个.del文件，标记了哪些segment中的哪些document被标记为deleted了<br>搜索的时候，会依次查询所有的segment，从旧的到新的，比如被修改过的document，在旧的segment中，会标记为deleted，在新的segment中会有其新的数据</p><p><img src="https://guanyuoss.oss-cn-qingdao.aliyuncs.com/prod/work_order/FMr1GIRdH14.png" alt=""></p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>倒排索引组成结构以及其索引不可变原因</title>
<link href="https://blog.collmall.com/2020/ck91gq8be0061ygta0j0l40dy/"/>
<id>https://blog.collmall.com/2020/ck91gq8be0061ygta0j0l40dy/</id>
<published>2020-02-29T09:07:11.000Z</published>
<updated>2020-04-15T13:27:49.248Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="倒排索引组成结构以及其索引不可变原因"><a href="#倒排索引组成结构以及其索引不可变原因" class="headerlink" title="倒排索引组成结构以及其索引不可变原因"></a>倒排索引组成结构以及其索引不可变原因</h4><p>倒排索引，是适合用于进行搜索的</p><p>倒排索引的结构</p><p>（1）包含这个关键词的document list<br>（2）包含这个关键词的所有document的数量：IDF（inverse document frequency）<br>（3）这个关键词在每个document中出现的次数：TF（term frequency）<br>（4）这个关键词在这个document中的次序<br>（5）每个document的长度：length norm<br>（6）包含这个关键词的所有document的平均长度</p><p>倒排索引不可变的好处</p><p>（1）不需要锁，提升并发能力，避免锁的问题<br>（2）数据不变，一直保存在os cache中，只要cache内存足够<br>（3）filter cache一直驻留在内存，因为数据不变<br>（4）可以压缩，节省cpu和io开销</p><p>倒排索引不可变的坏处：每次都要重新构建整个索引</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>基于scoll+bulk+索引别名实现零停机重建索引</title>
<link href="https://blog.collmall.com/2020/ck91gq8bh0067ygtadwy4a07f/"/>
<id>https://blog.collmall.com/2020/ck91gq8bh0067ygtadwy4a07f/</id>
<published>2020-02-28T09:07:11.000Z</published>
<updated>2020-04-15T13:27:25.001Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="基于scoll-bulk-索引别名实现零停机重建索引"><a href="#基于scoll-bulk-索引别名实现零停机重建索引" class="headerlink" title="基于scoll+bulk+索引别名实现零停机重建索引"></a>基于scoll+bulk+索引别名实现零停机重建索引</h4><h6 id="1、重建索引"><a href="#1、重建索引" class="headerlink" title="1、重建索引"></a>1、重建索引</h6><p>一个field的设置是不能被修改的，如果要修改一个Field，那么应该重新按照新的mapping，建立一个index，然后将数据批量查询出来，重新用bulk api写入index中</p><p>批量查询的时候，建议采用scroll api，并且采用多线程并发的方式来reindex数据，每次scoll就查询指定日期的一段数据，交给一个线程即可</p><p>（1）一开始，依靠dynamic mapping，插入数据，但是不小心有些数据是2017-01-01这种日期格式的，所以title这种field被自动映射为了date类型，实际上它应该是string类型的</p><p>PUT /my_index/my_type/3<br>{<br> “title”: “2017-01-03”<br>}</p><p>此时该index的类型映射为 date</p><p>{<br> “my_index”: {<br> “mappings”: {<br> “my_type”: {<br> “properties”: {<br> “title”: {<br> “type”: “date”<br> }<br> }<br> }<br> }<br> }<br>}</p><p>（2）当后期向索引中加入string类型的title值的时候，就会报错</p><p>PUT /my_index/my_type/4<br>{<br> “title”: “my first article”<br>}</p><p>此时会报错</p><p>{<br> “error”: {<br> “root_cause”: [<br> {<br> “type”: “mapper_parsing_exception”,<br> “reason”: “failed to parse [title]”<br> }<br> ],<br> “type”: “mapper_parsing_exception”,<br> “reason”: “failed to parse [title]”,<br> “caused_by”: {<br> “type”: “illegal_argument_exception”,<br> “reason”: “Invalid format: "my first article"“<br> }<br> },<br> “status”: 400<br>}</p><p>（3）如果此时想修改title的类型，是不可能的</p><p>PUT /my_index/_mapping/my_type<br>{<br> “properties”: {<br> “title”: {<br> “type”: “text”<br> }<br> }<br>}</p><p>{<br> “error”: {<br> “root_cause”: [<br> {<br> “type”: “illegal_argument_exception”,<br> “reason”: “mapper [title] of different type, current_type [date], merged_type [text]”<br> }<br> ],<br> “type”: “illegal_argument_exception”,<br> “reason”: “mapper [title] of different type, current_type [date], merged_type [text]”<br> },<br> “status”: 400<br>}</p><p>（4）此时，唯一的办法，就是进行reindex，也就是说，重新建立一个索引，将旧索引的数据查询出来，再导入新索引</p><p>（5）如果说旧索引的名字，是old_index，新索引的名字是new_index，终端java应用，已经在使用old_index在操作了，难道还要去停止java应用，修改使用的index为new_index，才重新启动java应用吗？这个过程中，就会导致java应用停机，可用性降低</p><p>（6）所以说，给java应用一个别名，这个别名是指向旧索引的，java应用先用着，java应用先用goods_index alias来操作，此时实际指向的是旧的my_index</p><p>PUT /my_index/_alias/goods_index</p><p>（7）新建一个index，调整其title的类型为string</p><p>PUT /my_index_new<br>{<br> “mappings”: {<br> “my_type”: {<br> “properties”: {<br> “title”: {<br> “type”: “text”<br> }<br> }<br> }<br> }<br>}</p><p>（8）使用scroll api将数据批量查询出来</p><p>GET /my_index/_search?scroll=1m<br>{<br> “query”: {<br> “match_all”: {}<br> },<br> “sort”: [“_doc”],<br> “size”: 1<br>}</p><p>{<br> “_scroll_id”: “DnF1ZXJ5VGhlbkZldGNoBQAAAAAAADpAFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAA6QRY0b25zVFlWWlRqR3ZJajlfc3BXejJ3AAAAAAAAOkIWNG9uc1RZVlpUakd2SWo5X3NwV3oydwAAAAAAADpDFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAA6RBY0b25zVFlWWlRqR3ZJajlfc3BXejJ3”,<br> “took”: 1,<br> “timed_out”: false,<br> “_shards”: {<br> “total”: 5,<br> “successful”: 5,<br> “failed”: 0<br> },<br> “hits”: {<br> “total”: 3,<br> “max_score”: null,<br> “hits”: [<br> {<br> “_index”: “my_index”,<br> “_type”: “my_type”,<br> “_id”: “2”,<br> “_score”: null,<br> “_source”: {<br> “title”: “2017-01-02”<br> },<br> “sort”: [<br> 0<br> ]<br> }<br> ]<br> }<br>}</p><p>（9）采用bulk api将scoll查出来的一批数据，批量写入新索引</p><p>POST /_bulk<br>{ “index”: { “_index”: “my_index_new”, “_type”: “my_type”, “_id”: “2” }}<br>{ “title”: “2017-01-02” }</p><p>（10）反复循环多次，查询一批又一批的数据出来，采取bulk api将每一批数据批量写入新索引</p><p>（11）将goods_index alias切换到my_index_new上去，java应用会直接通过index别名使用新的索引中的数据，java应用程序不需要停机，零提交，高可用</p><p>POST /_aliases<br>{<br> “actions”: [<br> { “remove”: { “index”: “my_index”, “alias”: “goods_index” }},<br> { “add”: { “index”: “my_index_new”, “alias”: “goods_index” }}<br> ]<br>}</p><p>（12）直接通过goods_index别名来查询，是否ok</p><p>GET /goods_index/my_type/_search</p><h6 id="2、基于alias对client透明切换index"><a href="#2、基于alias对client透明切换index" class="headerlink" title="2、基于alias对client透明切换index"></a>2、基于alias对client透明切换index</h6><p>PUT /my_index_v1/_alias/my_index</p><p>client对my_index进行操作</p><p>reindex操作，完成之后，切换v1到v2</p><p>POST /_aliases<br>{<br> “actions”: [<br> { “remove”: { “index”: “my_index_v1”, “alias”: “my_index” }},<br> { “add”: { “index”: “my_index_v2”, “alias”: “my_index” }}<br> ]<br>}</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>自定义dynamic mapping策略</title>
<link href="https://blog.collmall.com/2020/ck91gq8bd005vygta3vbsc8en/"/>
<id>https://blog.collmall.com/2020/ck91gq8bd005vygta3vbsc8en/</id>
<published>2020-02-27T09:07:11.000Z</published>
<updated>2020-04-15T13:26:56.983Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="自定义dynamic-mapping策略"><a href="#自定义dynamic-mapping策略" class="headerlink" title="自定义dynamic mapping策略"></a>自定义dynamic mapping策略</h4><h6 id="1、定制dynamic策略"><a href="#1、定制dynamic策略" class="headerlink" title="1、定制dynamic策略"></a>1、定制dynamic策略</h6><p> true：遇到陌生字段，就进行dynamic mapping<br>false：遇到陌生字段，就忽略<br>strict：遇到陌生字段，就报错</p><p>PUT /my_index<br>{<br> “mappings”: {<br> “my_type”: {<br> “dynamic”: “strict”, // 设置属性<br> “properties”: {<br> “title”: {<br> “type”: “text”<br> },<br> “address”: {<br> “type”: “object”,<br> “dynamic”: “true”<br> }<br> }<br> }<br> }<br>}</p><p>设置和类型不同的field 时</p><p>PUT /my_index/my_type/1<br>{<br> “title”: “my article”,<br> “content”: “this is my article”,<br> “address”: {<br> “province”: “guangdong”,<br> “city”: “guangzhou”<br> }<br>}</p><p>会报以下错误:</p><p>{<br> “error”: {<br> “root_cause”: [<br> {<br> “type”: “strict_dynamic_mapping_exception”,<br> “reason”: “mapping set to strict, dynamic introduction of [content] within [my_type] is not allowed”<br> }<br> ],<br> “type”: “strict_dynamic_mapping_exception”,<br> “reason”: “mapping set to strict, dynamic introduction of [content] within [my_type] is not allowed”<br> },<br> “status”: 400<br>}</p><p>增减一条数据:</p><p>PUT /my_index/my_type/1<br>{<br> “title”: “my article”,<br> “address”: {<br> “province”: “guangdong”,<br> “city”: “guangzhou”<br> }<br>}</p><p>查看这个</p><p>GET /my_index/_mapping/my_type</p><p>{<br> “my_index”: {<br> “mappings”: {<br> “my_type”: {<br> “dynamic”: “strict”,<br> “properties”: {<br> “address”: {<br> “dynamic”: “true”,<br> “properties”: {<br> “city”: {<br> “type”: “text”,<br> “fields”: {<br> “keyword”: {<br> “type”: “keyword”,<br> “ignore_above”: 256<br> }<br> }<br> },<br> “province”: {<br> “type”: “text”,<br> “fields”: {<br> “keyword”: {<br> “type”: “keyword”,<br> “ignore_above”: 256<br> }<br> }<br> }<br> }<br> },<br> “title”: {<br> “type”: “text”<br> }<br> }<br> }<br> }<br> }<br>}</p><h6 id="2、定制dynamic-mapping策略"><a href="#2、定制dynamic-mapping策略" class="headerlink" title="2、定制dynamic mapping策略"></a>2、定制dynamic mapping策略</h6><p>（1）date_detection</p><p>默认会按照一定格式识别date，比如yyyy-MM-dd。但是如果某个field先过来一个2017-01-01的值，就会被自动dynamic mapping成date，后面如果再来一个”hello world”之类的值，就会报错。可以手动关闭某个type的date_detection，如果有需要，自己手动指定某个field为date类型。</p><p>PUT /my_index/_mapping/my_type<br>{<br> “date_detection”: false<br>}</p><p>（2）定制自己的dynamic mapping template（type level）</p><p>PUT /my_index<br>{<br> “mappings”: {<br> “my_type”: {<br> “dynamic_templates”: [<br> { “en”: {<br> “match”: “*_en”,<br> “match_mapping_type”: “string”,<br> “mapping”: {<br> “type”: “string”,<br> “analyzer”: “english”<br> }<br> }}<br> ]<br>}}}</p><p>PUT /my_index/my_type/1<br>{<br> “title”: “this is my first article”<br>}</p><p>PUT /my_index/my_type/2<br>{<br> “title_en”: “this is my first article”<br>}</p><p>title没有匹配到任何的dynamic模板，默认就是standard分词器，不会过滤停用词，is会进入倒排索引，用is来搜索是可以搜索到的<br>title_en匹配到了dynamic模板，就是english分词器，会过滤停用词，is这种停用词就会被过滤掉，用is来搜索就搜索不到了</p><p>（3）定制自己的default mapping template（index level）</p><p>PUT /my_index<br>{<br> “mappings”: {<br> “<em>default</em>“: {<br> “_all”: { “enabled”: false }<br> },<br> “blog”: {<br> “_all”: { “enabled”: true }<br> }<br> }<br>}</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>mapping root object深入剖析</title>
<link href="https://blog.collmall.com/2020/ck91gq8bd005yygtadbeo22sd/"/>
<id>https://blog.collmall.com/2020/ck91gq8bd005yygtadbeo22sd/</id>
<published>2020-02-26T09:07:11.000Z</published>
<updated>2020-04-15T13:26:28.530Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="mapping-root-object深入剖析"><a href="#mapping-root-object深入剖析" class="headerlink" title="mapping root object深入剖析"></a>mapping root object深入剖析</h4><h6 id="1、root-object"><a href="#1、root-object" class="headerlink" title="1、root object"></a>1、root object</h6><p>就是某个type对应的mapping json，包括了properties，metadata（_id，_source，_type），settings（analyzer），其他settings（比如include_in_all）</p><p>PUT /my_index<br>{<br> “mappings”: {<br> “my_type”: {<br> “properties”: {}<br> }<br> }<br>}</p><h6 id="2、properties"><a href="#2、properties" class="headerlink" title="2、properties"></a>2、properties</h6><p>type，index，analyzer</p><p>PUT /my_index/_mapping/my_type<br>{<br> “properties”: {<br> “title”: {<br> “type”: “text”<br> }<br> }<br>}</p><h6 id="3、-source"><a href="#3、-source" class="headerlink" title="3、_source"></a>3、_source</h6><p>好处</p><p>（1）查询的时候，直接可以拿到完整的document，不需要先拿document id，再发送一次请求拿document<br>（2）partial update基于_source实现<br>（3）reindex时，直接基于_source实现，不需要从数据库（或者其他外部存储）查询数据再修改<br>（4）可以基于_source定制返回field<br>（5）debug query更容易，因为可以直接看到_source</p><p>如果不需要上述功能，可以禁用_source</p><p>禁用语法:</p><p>PUT /my_index/_mapping/my_type2<br>{<br> “_source”: {“enabled”: false}<br>}</p><h6 id="4、-all"><a href="#4、-all" class="headerlink" title="4、_all"></a>4、_all</h6><p>将所有field打包在一起，作为一个_all field，建立索引。没指定任何 field 进行搜索时，就是使用_all field在搜索。</p><p>PUT /my_index/_mapping/my_type3<br>{<br> “_all”: {“enabled”: false}<br>}</p><p>也可以在field级别设置include_in_all field，设置是否要将field的值包含在_all field中</p><p>PUT /my_index/_mapping/my_type4<br>{<br> “properties”: {<br> “my_field”: {<br> “type”: “text”,<br> “include_in_all”: false<br> }<br> }<br>}</p><h6 id="5、标识性metadata"><a href="#5、标识性metadata" class="headerlink" title="5、标识性metadata"></a>5、标识性metadata</h6><p>_index，_type，_id</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>type的底层数据结构</title>
<link href="https://blog.collmall.com/2020/ck91gq8ba005pygtaeba8339w/"/>
<id>https://blog.collmall.com/2020/ck91gq8ba005pygtaeba8339w/</id>
<published>2020-02-25T09:07:11.000Z</published>
<updated>2020-04-15T13:26:02.056Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h5 id="type的底层数据结构"><a href="#type的底层数据结构" class="headerlink" title="type的底层数据结构"></a>type的底层数据结构</h5><p>type，是一个index中用来区分类似的数据的，但是可能有不同的fields，而且有不同的属性来控制索引建立、分词器<br>field的value，在底层的lucene中建立索引的时候，全部是opaque bytes类型，不区分类型的<br>也就是说lucene是没有type的概念的，在document中，实际上将type作为一个document的field来存储，即_type，es通过_type来进行type的过滤和筛选<br>一个index中的多个type，实际上是放在一起存储的，因此一个index下，不能有多个type重名，而类型或者其他设置不同的，因为那样是无法处理的</p><p>eg: type的数据结构</p><p>{<br> “ecommerce”: {<br> “mappings”: {<br> “elactronic_goods”: {<br> “properties”: {<br> “name”: {<br> “type”: “string”,<br> },<br> “price”: {<br> “type”: “double”<br> },<br> “service_period”: {<br> “type”: “string”<br> }<br> }<br> },<br> “fresh_goods”: {<br> “properties”: {<br> “name”: {<br> “type”: “string”,<br> },<br> “price”: {<br> “type”: “double”<br> },<br> “eat_period”: {<br> “type”: “string”<br> }<br> }<br> }<br> }<br> }<br>}</p><p>以下两个type 在底层存储的时候</p><p>{<br> “name”: “geli kongtiao”,<br> “price”: 1999.0,<br> “service_period”: “one year”<br>}</p><p>{<br> “name”: “aozhou dalongxia”,<br> “price”: 199.0,<br> “eat_period”: “one week”<br>}</p><p>同一type在底层的存储是这样子的,会把共有的东西糅合在一起</p><p>{<br> “ecommerce”: {<br> “mappings”: {<br> “_type”: {<br> “type”: “string”,<br> “index”: “not_analyzed”<br> },<br> “name”: {<br> “type”: “string”<br> }<br> “price”: {<br> “type”: “double”<br> }<br> “service_period”: {<br> “type”: “string”<br> }<br> “eat_period”: {<br> “type”: “string”<br> }<br> }<br> }<br>}</p><p>{<br> “_type”: “elactronic_goods”,<br> “name”: “geli kongtiao”,<br> “price”: 1999.0,<br> “service_period”: “one year”,<br> “eat_period”: “”<br>}</p><p>{<br> “_type”: “fresh_goods”,<br> “name”: “aozhou dalongxia”,<br> “price”: 199.0,<br> “service_period”: “”,<br> “eat_period”: “one week”<br>}</p><p>最佳实践，将类似结构的type放在一个index下，这些type应该有多个field是相同的<br>假如说，你将两个type的field完全不同，放在一个index下，那么就每条数据都至少有一半的field在底层的lucene中是空值，会有严重的性能问题, 和占用空间</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>修改分词器以及自定义分词器</title>
<link href="https://blog.collmall.com/2020/ck91gq8bb005sygtacrt557pv/"/>
<id>https://blog.collmall.com/2020/ck91gq8bb005sygtacrt557pv/</id>
<published>2020-02-24T09:07:11.000Z</published>
<updated>2020-04-15T13:25:32.239Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="修改分词器以及自定义分词器"><a href="#修改分词器以及自定义分词器" class="headerlink" title="修改分词器以及自定义分词器"></a>修改分词器以及自定义分词器</h4><h6 id="1、默认的分词器简介"><a href="#1、默认的分词器简介" class="headerlink" title="1、默认的分词器简介"></a>1、默认的分词器简介</h6><p>standard</p><p>standard tokenizer：以单词边界进行切分<br>standard token filter：什么都不做<br>lowercase token filter：将所有字母转换为小写<br>stop token filer（默认被禁用）：移除停用词，比如a the it等等</p><h6 id="2、修改分词器的设置"><a href="#2、修改分词器的设置" class="headerlink" title="2、修改分词器的设置"></a>2、修改分词器的设置</h6><p>启用english停用词token filter</p><p>PUT /my_index<br>{<br> “settings”: {<br> “analysis”: {<br> “analyzer”: {<br> “es_std”: { // 自己起的名字<br> “type”: “standard”,<br> “stopwords”: “<em>english</em>“<br> }<br> }<br> }<br> }<br>}</p><p>GET /my_index/_analyze<br>{<br> “analyzer”: “standard”,<br> “text”: “a dog is in the house”<br>}</p><p>GET /my_index/_analyze<br>{<br> “analyzer”: “es_std”,<br> “text”:”a dog is in the house”<br>}</p><h6 id="3、定制化自己的分词器"><a href="#3、定制化自己的分词器" class="headerlink" title="3、定制化自己的分词器"></a>3、定制化自己的分词器</h6><p>PUT /my_index<br>{<br> “settings”: {<br> “analysis”: {<br> “char_filter”: {<br> “&_to_and”: {<br> “type”: “mapping”,<br> “mappings”: [“&=> and”]<br> }<br> },<br> “filter”: {<br> “my_stopwords”: {<br> “type”: “stop”,<br> “stopwords”: [“the”, “a”]<br> }<br> },<br> “analyzer”: {<br> “my_analyzer”: {<br> “type”: “custom”,<br> “char_filter”: [“html_strip”, “&_to_and”],<br> “tokenizer”: “standard”,<br> “filter”: [“lowercase”, “my_stopwords”]<br> }<br> }<br> }<br> }<br>}</p><p>GET /my_index/_analyze<br>{<br> “text”: “tom&jerry are a friend in the house, <a>, HAHA!!”,<br> “analyzer”: “my_analyzer”<br>}</p><p>PUT /my_index/_mapping/my_type<br>{<br> “properties”: {<br> “content”: {<br> “type”: “text”,<br> “analyzer”: “my_analyzer”<br> }<br> }<br>}</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>创建、修改以及删除索引</title>
<link href="https://blog.collmall.com/2020/ck91gq8b9005mygta6m3t8a0s/"/>
<id>https://blog.collmall.com/2020/ck91gq8b9005mygta6m3t8a0s/</id>
<published>2020-02-23T09:07:11.000Z</published>
<updated>2020-04-15T13:24:59.809Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="创建、修改以及删除索引"><a href="#创建、修改以及删除索引" class="headerlink" title="创建、修改以及删除索引"></a>创建、修改以及删除索引</h4><h5 id="1、创建索引"><a href="#1、创建索引" class="headerlink" title="1、创建索引"></a>1、创建索引</h5><p>创建索引的语法</p><p>PUT /my_index<br>{<br> “settings”: { … any settings … },<br> “mappings”: {<br> “type_one”: { … any mappings … },<br> “type_two”: { … any mappings … },<br> …<br> }<br>}</p><p>创建索引的示例</p><p>PUT /my_index<br>{<br> “settings”: {<br> “number_of_shards”: 1,<br> “number_of_replicas”: 0<br> },<br> “mappings”: {<br> “my_type”: {<br> “properties”: {<br> “my_field”: {<br> “type”: “text”<br> }<br> }<br> }<br> }<br>}</p><h5 id="2、修改索引"><a href="#2、修改索引" class="headerlink" title="2、修改索引"></a>2、修改索引</h5><p>PUT /my_index/_settings<br>{<br> “number_of_replicas”: 1<br>}</p><h5 id="3、删除索引"><a href="#3、删除索引" class="headerlink" title="3、删除索引"></a>3、删除索引</h5><p>DELETE /my_index<br>DELETE /index_one,index_two<br>DELETE /index_*<br>DELETE /_all</p><p>可以在elasticsearch.yml中配置<br>action.destructive_requires_name: true </p><p>禁用DELETE /_all 操作,只能指定操作</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>scoll滚动搜索</title>
<link href="https://blog.collmall.com/2020/ck91gq8b3005jygta7mlyao5t/"/>
<id>https://blog.collmall.com/2020/ck91gq8b3005jygta7mlyao5t/</id>
<published>2020-02-21T09:07:11.000Z</published>
<updated>2020-04-15T13:24:34.614Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="scoll滚动搜索"><a href="#scoll滚动搜索" class="headerlink" title="scoll滚动搜索"></a>scoll滚动搜索</h4><p>eg : 如果一次性要查出来比如10万条数据，那么性能会很差，此时一般会采取用scoll滚动查询，一批一批的查，直到所有数据都查询完处理完</p><p>使用scoll滚动搜索，可以先搜索一批数据，然后下次再搜索一批数据，以此类推，直到搜索出全部的数据来<br>scoll搜索会在第一次搜索的时候，保存一个当时的视图快照，之后只会基于该旧的视图快照提供数据搜索，如果这个期间数据变更，是不会让用户看到的<br>采用基于_doc进行排序的方式，性能较高<br>每次发送scroll请求，我们还需要指定一个scoll参数，指定一个时间窗口，每次搜索请求只要在这个时间窗口内能完成就可以了</p><p>GET /test_index/test_type/_search?scroll=1m<br>{<br> “query”: {<br> “match_all”: {}<br> },<br> “sort”: [ “_doc” ],<br> “size”: 3<br>}</p><p>结果示例:</p><p>{<br> “_scroll_id”: “DnF1ZXJ5VGhlbkZldGNoBQAAAAAAACxeFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAAsYBY0b25zVFlWWlRqR3ZJajlfc3BXejJ3AAAAAAAALF8WNG9uc1RZVlpUakd2SWo5X3NwV3oydwAAAAAAACxhFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAAsYhY0b25zVFlWWlRqR3ZJajlfc3BXejJ3”,<br> “took”: 5,<br> “timed_out”: false,<br> “_shards”: {<br> “total”: 5,<br> “successful”: 5,<br> “failed”: 0<br> },<br> “hits”: {<br> “total”: 10,<br> “max_score”: null,<br> “hits”: [<br> {<br> “_index”: “test_index”,<br> “_type”: “test_type”,<br> “_id”: “8”,<br> “_score”: null,<br> “_source”: {<br> “test_field”: “test client 2”<br> },<br> “sort”: [<br> 0<br> ]<br> },<br> {<br> “_index”: “test_index”,<br> “_type”: “test_type”,<br> “_id”: “6”,<br> “_score”: null,<br> “_source”: {<br> “test_field”: “tes test”<br> },<br> “sort”: [<br> 0<br> ]<br> },<br> {<br> “_index”: “test_index”,<br> “_type”: “test_type”,<br> “_id”: “AVp4RN0bhjxldOOnBxaE”,<br> “_score”: null,<br> “_source”: {<br> “test_content”: “my test”<br> },<br> “sort”: [<br> 0<br> ]<br> }<br> ]<br> }<br>}</p><p>获得的结果会有一个scoll_id，下一次再发送scoll请求的时候，必须带上上一个的scoll_id</p><p>GET /_search/scroll<br>{<br> “scroll”: “1m”,<br> “scroll_id” : “DnF1ZXJ5VGhlbkZldGNoBQAAAAAAACxeFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAAsYBY0b25zVFlWWlRqR3ZJajlfc3BXejJ3AAAAAAAALF8WNG9uc1RZVlpUakd2SWo5X3NwV3oydwAAAAAAACxhFjRvbnNUWVZaVGpHdklqOV9zcFd6MncAAAAAAAAsYhY0b25zVFlWWlRqR3ZJajlfc3BXejJ3”<br>}</p><p>11,4,7<br>3,2,1<br>20</p><p>scoll，看起来挺像分页的，但是其实使用场景不一样。分页主要是用来一页一页搜索，给用户看的；scoll主要是用来一批一批检索数据，让系统进行处理的</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
<entry>
<title>bouncing results问题解决方案</title>
<link href="https://blog.collmall.com/2020/ck91gq8b2005gygtacpnr6djk/"/>
<id>https://blog.collmall.com/2020/ck91gq8b2005gygtacpnr6djk/</id>
<published>2020-02-21T09:07:11.000Z</published>
<updated>2020-04-15T13:24:11.120Z</updated>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" class="aplayer-secondary-script-marker"></script><h4 id="bouncing-results问题解决方案"><a href="#bouncing-results问题解决方案" class="headerlink" title="bouncing results问题解决方案"></a>bouncing results问题解决方案</h4><h5 id="1、preference"><a href="#1、preference" class="headerlink" title="1、preference"></a>1、preference</h5><p>决定了哪些shard会被用来执行搜索操作</p><p>_primary, _primary_first, _local, _only_node:xyz, _prefer_node:xyz, _shards:2,3</p><p>bouncing results问题，两个document排序，field值相同；不同的shard上，可能排序不同；每次请求轮询打到不同的replica shard上；每次页面上看到的搜索结果的排序都不一样。这就是bouncing result，也就是跳跃的结果。</p><p>搜索的时候，是轮询将搜索请求发送到每一个replica shard（primary shard），但是在不同的shard上，可能document的排序不同</p><p>解决方案就是将preference设置为一个字符串，比如说user_id，让每个user每次搜索的时候，都使用同一个replica shard去执行，就不会看到bouncing results了</p><h5 id="2、timeout-主要就是限定在一定时间内，将部分获取到的数据直接返回，避免查询耗时过长"><a href="#2、timeout-主要就是限定在一定时间内，将部分获取到的数据直接返回，避免查询耗时过长" class="headerlink" title="2、timeout: 主要就是限定在一定时间内，将部分获取到的数据直接返回，避免查询耗时过长"></a>2、timeout: 主要就是限定在一定时间内，将部分获取到的数据直接返回，避免查询耗时过长</h5><h5 id="3、routing，document文档路由，-id路由，routing-user-id，这样的话可以让同一个user对应的数据到一个shard上去"><a href="#3、routing，document文档路由，-id路由，routing-user-id，这样的话可以让同一个user对应的数据到一个shard上去" class="headerlink" title="3、routing，document文档路由，_id路由，routing=user_id，这样的话可以让同一个user对应的数据到一个shard上去"></a>3、routing，document文档路由，_id路由，routing=user_id，这样的话可以让同一个user对应的数据到一个shard上去</h5><h5 id="4、search-type"><a href="#4、search-type" class="headerlink" title="4、search_type"></a>4、search_type</h5><p>default：query_then_fetch<br>dfs_query_then_fetch，可以提升revelance sort精准度</p>]]></content>
<summary type="html">
<link rel="stylesheet" class="aplayer-secondary-style-marker" href="\assets\css\APlayer.min.css"><script src="\assets\js\APlayer.min.js" cla
</summary>
<category term="Elasticsearch" scheme="https://blog.collmall.com/categories/Elasticsearch/"/>
<category term="Elasticsearch" scheme="https://blog.collmall.com/tags/Elasticsearch/"/>
</entry>
</feed>