Skip to content

Changes for OpenSearch #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 32 commits into
base: 7.10
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
597b924
update elasticsearch to 7.10.2
Feb 11, 2021
d900694
update to ES 7.10.2
jprante Jan 29, 2021
bc06891
changes for opensearch 1.0
Jul 11, 2022
2bb4a5b
changes for opensearch 1.3.3
Jul 11, 2022
d078348
changes for opensearch 2.0.1
Jul 12, 2022
5fc4e6e
changes for opensearch 2.1.0
Jul 12, 2022
8bd7e12
cleanup readme
Jul 13, 2022
85718e0
Merge pull request #1 from jprante/7.10
grmblfrz Jul 13, 2022
2884f73
Merge pull request #2 from grmblfrz/7.10
grmblfrz Jul 13, 2022
2fed8b4
fix plugin name
Jul 14, 2022
6c2855b
changes for opensearch 2.2.0
Aug 22, 2022
51d29ae
changes for opensearch 2.3.0
Nov 14, 2022
3b15959
anges for opensearch 2.4.0
Nov 16, 2022
a7530d7
changes for opensearch 2.4.1
Jan 25, 2023
fb8bf11
changes for opensearch 2.5.0
Jan 25, 2023
4863954
changes for opensearch 2.6.0
Mar 28, 2023
0c48842
changes for opensearch 2.7.0
Jul 13, 2023
fe6b47d
changes for opensearch 2.8.0
Jul 13, 2023
c492529
changes for opensearch 2.9.0
Sep 7, 2023
867ca27
changes for opensearch 2.10.0
Nov 8, 2023
6f36fe1
changes for opensearch 2.11.0
Nov 8, 2023
ec61bda
changes for opensearch 2.12.0
Apr 12, 2024
ca9e672
changes for opensearch 2.13.0
Apr 12, 2024
e022b33
changes for opensearch 2.14.0
Mar 5, 2025
936175f
changes for opensearch 2.15.0
Mar 5, 2025
24d2a12
changes for opensearch 2.16.0
Mar 5, 2025
9ee7bfc
changes for opensearch 2.17.0
Mar 5, 2025
1e22d40
changes for opensearch 2.18.0
Mar 5, 2025
bbfa46b
changes for opensearch 2.19.0
Mar 5, 2025
b38e9d8
changes for opensearch 2.19.1
Mar 5, 2025
e104a35
changes for opensearch 3.0.0
May 27, 2025
f2d1828
changes for opensearch 3.1.0
Jun 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
110 changes: 27 additions & 83 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# A plugin bundle for Elastisearch
# A plugin bundle for Opensearch

image:https://api.travis-ci.org/jprante/elasticsearch-plugin-bundle.svg[title="Build status", link="https://travis-ci.org/jprante/elasticsearch-plugin-bundle/"]
image:https://img.shields.io/sonar/http/nemo.sonarqube.com/org.xbib.elasticsearch.plugin%3Aelasticsearch-plugin-bundle/coverage.svg?style=flat-square[title="Coverage", link="https://sonarqube.com/dashboard/index?id=org.xbib.elasticsearch.plugin%3Aelasticsearch-plugin-bundle"]
Expand All @@ -9,101 +9,45 @@ image:https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif[title="PayPa

This plugin is the combination of the following plugins:

- elasticsearch-analysis-autophrase
- elasticsearch-analysis-baseform
- elasticsearch-analysis-concat
- elasticsearch-analysis-decompound
- elasticsearch-analysis-german
- elasticsearch-analysis-hyphen
- elasticsearch-analysis-icu
- elasticsearch-analysis-naturalsort
- elasticsearch-analysis-reference
- elasticsearch-analysis-sortform
- elasticsearch-analysis-standardnumber
- elasticsearch-analysis-symbolname
- elasticsearch-analysis-worddelimiter
- elasticsearch-analysis-year
- elasticsearch-mapper-crypt
- elasticsearch-mapper-langdetect
- opensearch-analysis-autophrase
- opensearch-analysis-baseform
- opensearch-analysis-concat
- opensearch-analysis-decompound
- opensearch-analysis-german
- opensearch-analysis-hyphen
- opensearch-analysis-icu
- opensearch-analysis-naturalsort
- opensearch-analysis-reference
- opensearch-analysis-sortform
- opensearch-analysis-standardnumber
- opensearch-analysis-symbolname
- opensearch-analysis-worddelimiter
- opensearch-analysis-year
- opensearch-mapper-crypt
- opensearch-mapper-langdetect

The plugin code in each plugin is equivalent to the code in this combined bundle plugin.

.Compatibility matrix
[frame="all"]
|===
| Plugin version | Elasticsearch version | Release date
| 6.3.2.2 | 6.3.2 | Oct 2, 2018
| 5.4.1.0 | 5.4.0 | Jun 1, 2017
| 5.4.0.1 | 5.4.0 | May 12, 2017
| 5.4.0.0 | 5.4.0 | May 4, 2017
| 5.3.1.0 | 5.3.1 | Apr 25, 2017
| 5.3.0.0 | 5.3.0 | Apr 4, 2017
| 5.2.2.0 | 5.2.2 | Mar 2, 2017
| 5.2.1.0 | 5.2.1 | Feb 27, 2017
| 5.1.1.2 | 5.1.1 | Feb 27, 2017
| 5.1.1.0 | 5.1.1 | Dec 31, 2016
| 2.3.4.0 | 2.3.4 | Jul 30, 2016
| 2.3.3.0 | 2.3.3 | May 23, 2016
| 2.3.2.0 | 2.3.2 | May 11, 2016
| 2.2.0.6 | 2.2.0 | Mar 25, 2016
| 2.2.0.3 | 2.2.0 | Mar 6, 2016
| 2.2.0.2 | 2.2.0 | Mar 3, 2016
| 2.2.0.1 | 2.2.0 | Feb 22, 2016
| 2.2.0.0 | 2.2.0 | Feb 8, 2016
| 2.1.1.2 | 2.1.1 | Dec 30, 2015
| 2.1.1.0 | 2.1.1 | Dec 21, 2015
| 2.1.0.0 | 2.1.0 | Nov 27, 2015
| 2.0.0.0 | 2.0.0 | Oct 28, 2015
| 1.6.0.0 | 1.6.0 | Jun 30, 2015
| 1.5.2.1 | 1.5.2 | Jun 30, 2015
| 1.5.2.0 | 1.5.2 | Apr 27, 2015
| 1.5.1.0 | 1.5.1 | Apr 23, 2015
| 1.5.0.0 | 1.5.0 | Mar 31, 2015
| 1.4.4.0 | 1.4.4 | Apr 26, 2015
| 1.4.0.6 | 1.4.0 | Feb 23, 2015
| 1.4.0.5 | 1.4.0 | Jan 28, 2015
| 1.4.0.4 | 1.4.0 | Jan 19, 2015
| 1.4.0.3 | 1.4.0 | Dec 16, 2014
| 1.4.0.1 | 1.4.0 | Nov 10, 2014
| Plugin version | Opensearch version | Release date
| 2.1.0.0 | 2.1.0 | Jul 12, 2022
| 2.0.1.0 | 2.0.1 | Jul 12, 2022
| 1.3.3.0 | 1.3.3 | Jul 12, 2022
| 1.0.0.0 | 1.0.0 | Jul 12, 2022
|===


## Installation

### Elasticsearch 5.x
### Opensearch 1.x, 2.x

[source]
----
./bin/elasticsearch-plugin install http://search.maven.org/remotecontent?filepath=org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/5.4.0.0/elasticsearch-plugin-bundle-5.4.0-plugin.zip
opensearch-plugin install -b https://github.com/grmblfrz/elasticsearch-plugin-bundle/releases/download/opensearch-2.1.0/opensearch-plugin-bundle-2.1.0.0.zip
----

or

[source]
----
./bin/elasticsearch-plugin install http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/5.3.0.0/elasticsearch-plugin-bundle-5.3.0.0-plugin.zip
----
Do not forget to restart the node after installing.

### Elasticsearch 2.x

[source]
----
./bin/plugin install 'http://search.maven.org/remotecontent?filepath=org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/2.3.3.0/elasticsearch-plugin-bundle-2.3.3.0-plugin.zip'
----
or
[source]
----
./bin/plugin install 'http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/2.3.4.0/elasticsearch-plugin-bundle-2.3.4.0-plugin.zip'
----
Do not forget to restart the node after installing.

### Elasticsearch 1.x

[source]
----
./bin/plugin -install bundle -url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/1.6.0.0/elasticsearch-plugin-bundle-1.6.0.0-plugin.zip
----
Do not forget to restart the node after installing.

# Documentation
Expand All @@ -130,7 +74,7 @@ More to come.

## German normalizer

The `german_normalizer` is equivalent to Elasticsearch `german_normalization`. It performs umlaut treatment
The `german_normalizer` is equivalent to OpenSearch `german_normalization`. It performs umlaut treatment
with vocal expansion which is typical for german language.

[source]
Expand Down Expand Up @@ -617,7 +561,7 @@ GET _analyze

# Decompound

This is an implementation of a word decompounder plugin for link:http://github.com/elasticsearch/elasticsearch[Elasticsearch].
This is an implementation of a word decompounder plugin for link:http://github.com/opensearch-project/OpenSearch[OpenSearch].

Compounding several words into one word is a property not all languages share.
Compounding is used in German, Scandinavian Languages, Finnish and Korean.
Expand Down Expand Up @@ -673,7 +617,7 @@ In the mapping, use a token filter of type "decompound"::
"Die Jahresfeier der Rechtsanwaltskanzleien auf dem Donaudampfschiff hat viel Ökosteuer gekostet" will be tokenized into
"Die", "Die", "Jahresfeier", "Jahr", "feier", "der", "der", "Rechtsanwaltskanzleien", "Recht", "anwalt", "kanzlei", "auf", "auf", "dem", "dem", "Donaudampfschiff", "Donau", "dampf", "schiff", "hat", "hat", "viel", "viel", "Ökosteuer", "Ökosteuer", "gekostet", "gekosten"

It is recommended to add the `Unique token filter <http://www.elasticsearch.org/guide/reference/index-modules/analysis/unique-tokenfilter.html>`_ to skip tokens that occur more than once.
It is recommended to add the `Unique token filter https://www.elastic.co/guide/en/elasticsearch/reference/7.10/analysis-unique-tokenfilter.html to skip tokens that occur more than once.

Also the Lucene german normalization token filter is provided::

Expand Down
88 changes: 60 additions & 28 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,30 @@ import java.time.ZonedDateTime
buildscript {
repositories {
mavenCentral()
jcenter()
gradlePluginPortal()
}
dependencies {
classpath "org.elasticsearch.gradle:build-tools:${project.property('elasticsearch.version')}"
classpath "org.opensearch.gradle:build-tools:${project.property('opensearch.version')}"
}
}

plugins {
id "org.sonarqube" version "2.8"
id "io.codearte.nexus-staging" version "0.21.1"
id "org.xbib.gradle.plugin.jflex" version "1.4.0"
id "org.xbib.gradle.plugin.asciidoctor" version "1.5.6.0.1"
id "org.xbib.gradle.plugin.jflex" version "3.0.2"
id "org.xbib.gradle.plugin.asciidoctor" version "3.0.2"
}

apply plugin: 'java-library'
apply plugin: 'maven'
apply plugin: 'elasticsearch.esplugin'
apply plugin: 'maven-publish'
apply plugin: 'opensearch.opensearchplugin'
apply plugin: 'org.xbib.gradle.plugin.jflex'
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'opensearch.rest-resources'

ext {
pluginName = 'bundle'
pluginClassname = 'org.xbib.elasticsearch.plugin.bundle.BundlePlugin'
pluginDescription = 'A bundle of plugins for Elasticsearch'
pluginClassname = 'org.xbib.opensearch.plugin.bundle.BundlePlugin'
pluginDescription = 'A bundle of plugins for Opensearch'
user = 'jprante'
name = 'elasticsearch-plugin-bundle'
name = 'opensearch-plugin-bundle'
scmUrl = 'https://github.com/' + user + '/' + name
scmConnection = 'scm:git:git://github.com/' + user + '/' + name + '.git'
scmDeveloperConnection = 'scm:git:git://github.com/' + user + '/' + name + '.git'
Expand All @@ -45,30 +43,53 @@ sourceSets {
}
}

esplugin {
description 'Plugin bundle for Elasticsearch'
name 'elasticsearch-plugin-bundle'
classname 'org.xbib.elasticsearch.plugin.bundle.BundlePlugin'
opensearchplugin {
description = 'Plugin bundle for Opensearch'
name = 'opensearch-plugin-bundle'
classname = 'org.xbib.opensearch.plugin.bundle.BundlePlugin'
}

configurations {
agent
}

task prepareAgent(type: Copy) {
from(configurations.agent)
into "$buildDir/agent"
}

dependencies {
// we must use "compile" and not "implementation" otherwise ES build does not detect dependencies
compile "com.ibm.icu:icu4j:${project.property('icu4j.version')}"
compile "org.xbib:standardnumber:${project.property('standardnumber.version')}"
testImplementation "org.codelibs.elasticsearch.module:analysis-common:${project.property('elasticsearch.version')}"
implementation "com.ibm.icu:icu4j:${project.property('icu4j.version')}"
implementation "org.xbib:standardnumber:${project.property('standardnumber.version')}"
testImplementation "org.opensearch.plugin:analysis-common:${project.property('opensearch.version')}"
agent "org.opensearch:opensearch-agent-bootstrap:${project.property('opensearch.version')}"
agent "org.opensearch:opensearch-agent:${project.property('opensearch.version')}"
agent "net.bytebuddy:byte-buddy:${project.property('bytebuddy.version')}"
}


tasks.withType(Test) {
dependsOn prepareAgent
jvmArgs += ["-javaagent:" + project.layout.buildDirectory.file("agent/opensearch-agent-${project.property('opensearch.version')}.jar").get()]
}

java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(21))
}
}

compileJava {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
options.compilerArgs = [
"-Xlint:-try"
]
}

compileTestJava {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
options.compilerArgs = [
"-Xlint:-cast,-deprecation,-rawtypes,-try,-unchecked",
"-proc:none"
Expand All @@ -78,6 +99,7 @@ compileTestJava {
javadoc {
enabled = true
options.addStringOption('Xdoclint:none', '-quiet')
dependsOn('generateJflex')
}

jar {
Expand All @@ -90,9 +112,9 @@ jar {
'Build-Java-Version': System.getProperty("java.version"),
'X-Compile-Source-JDK': compileJava.targetCompatibility,
'X-Compile-Target-JDK': compileJava.sourceCompatibility,
'X-Compile-Elasticsearch-Version': project.property('elasticsearch.version'),
'X-Compile-Opensearch-Version': project.property('opensearch.version'),
'X-Compile-Lucene-Version': project.property('lucene.version'),
'X-Compile-Elasticsearch-Snapshot': false
'X-Compile-Opensearch-Snapshot': false
)
}
}
Expand Down Expand Up @@ -195,7 +217,6 @@ task xbibUpload(type: Upload) {
nexusStaging {
packageGroup = "org.xbib"
}
*/

sonarqube {
properties {
Expand All @@ -207,6 +228,7 @@ sonarqube {
property "sonar.junit.reportsPath", "build/test-results/test/"
}
}
*/

asciidoctor {
attributes toc: 'left',
Expand All @@ -220,7 +242,7 @@ asciidoctor {
'source-highlighter': 'coderay'
}

checkstyleMain.source = 'src/main/java'
//checkstyleMain.source = 'src/main/java'
// we use @Ignore
forbiddenApisTest.enabled = false
forbiddenPatterns.exclude('**/*.txt', '**/*.brk', '**/*.fst', '**/*.tree', '**/*.nrm')
Expand All @@ -230,3 +252,13 @@ thirdPartyAudit.enabled = false
loggerUsageCheck.enabled = false
testingConventions.enabled = false
validateNebulaPom.enabled = false

tasks.named('sourcesJar').configure {
dependsOn('generateJflex')
}
tasks.named('filepermissions').configure {
dependsOn('generateJflex')
}
tasks.named('forbiddenPatterns').configure {
dependsOn('generateJflex')
}
23 changes: 12 additions & 11 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
group = org.xbib.elasticsearch
name = elasticsearch-plugin-bundle
version = 7.10.2.0
group = org.xbib.opensearch
name = opensearch-plugin-bundle
version = 3.1.0.0

elasticsearch.version = 7.10.2
lucene.version = 8.7.0
opensearch.version = 3.1.0
lucene.version = 10.2.1

icu4j.version = 67.1
standardnumber.version = 1.0.1
jackson.version = 2.9.10
icu4j.version = 77.1
standardnumber.version = 1.2.0
jackson.version = 2.17.2
spatial4j.version = 0.7
jts.version = 1.15.1
jna.version = 4.5.1
jna.version = 5.16.0
checkstyle.version = 8.13
wagon.version = 3.0.0

log4j.version = 2.13.1
junit.version = 4.13
log4j.version = 2.21.0
junit.version = 4.13.2
bytebuddy.version = 1.17.5
Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
4 changes: 3 additions & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Loading