August 2023 Sprint (#385)

* [Issue 328] Resource not found exceptions (#329) * Return null to indicate not found * Add file not found exception handler * Add external asset directory config example * Update README.md * Add index collection name configuration and replace constant * Add initial index health indicator * [Issue 237] Add initial GitHub templates. (#342) * [Issue 347] Add configuration to disable individual document create on batch fail (#348) Co-authored-by: Rincy Mathew <[email protected]> * Add configuration to enable/disable schematizing (#356) * Remove over encapsulation and correct separation of concern (#361) The helper classes in discovery package are violating common principles and are not providing abstraction to multiple implementations of their use. * Refactor research/er age with academic age (#363) * Add initial data and analytics view (#366) The additional properties required for data and analytics UI/UX has yet to be determined. * Add organization id to theme (#365) The organization could be removed and wherever rendered can be updated to fetch the organization by id. Note the non default constructor is for testing purposes. * [Issue 368] Refactor relationship date time interval properties (#369) * Refactor relationship date time interval properties The field types conflicted with property of the same name on Process. * Add eof newline to template file * [Issue 370] Add additional facet entity properties to control UI behavior (#371) * Add expandable/collapsible properties to facet entity * Support open dialog for facets * Update facet test for use dialog getters/setters * Use property paths where applicable in views (#374) * Add sub organization type to organization (#375) * Upgrade Jena lib, remove SDB triplestore support, and minor refactoring (#377) * Remove SDB triplestore support * Upgrade jena libs * Add caffeine as a dependency * Minor refactor and replace deprecation * Remove redundant field type creation * Cleanup triplestore harvester * [Issue 357] Individual medium for harvest/index (#378) * Individual medium for harvest/index and remove field binding from discovery model Note that Individual still uses Solr Java Object binding to SolrInputDocument for indexing. * Remove unused test dependency injection * [Issue 380] Data and analytic view container type (#381) * Add container type to data and analytics view * Update individual model resources * Update tests * Add eof newline to data and analytics default yaml * Update export view names * Fix analytics endpoint mapping for academicAge * Remove authorize for now * Fix copy field duplication (#384) * Docx exporter regression (#388) * Use individual for serialization instead of content * Minor refactor document to individual * Add facets to data and analytics views (#389) * Remove broken workflows (#392) * Upgrade some dependencies (#393) * Use liquibase (#394) * Use liquibase * Fix updating defaults * Update authorizations for data and analytics (#395) * Update authorizations for data and analytics * Add some missing ending newlines * Upgrade to solr docker to 9.3 * Use public tamu smtp as default * Now using stomp without sockjs * Upgrade spring security config * Minor cleanup and expose content disposition header * Fix zip exporter authorization * Prevent template error when type is not present (#397) --------- Co-authored-by: William Welling <[email protected]> Co-authored-by: William Welling <[email protected]> Co-authored-by: Rincy Mathew <[email protected]> Co-authored-by: wwoops <[email protected]> Co-authored-by: William Welling <[email protected]>
TAMULib · Oct 12, 2023 · b24b6db · b24b6db
1 parent 538ce10
commit b24b6db
Show file tree

Hide file tree

Showing 115 changed files with 5,499 additions and 1,660 deletions.
diff --git a/.github/ISSUE_TEMPLATE/feature.md b/.github/ISSUE_TEMPLATE/feature.md
@@ -0,0 +1,25 @@
+---
+name: Tracking feature
+about: Use this template for tracking new features.
+title: "[DATE]: [FEATURE NAME]"
+labels: tracking feature, needs planning
+---
+
+1. Purpose
+    > What use case is this for?
+
+
+2. Scope
+    > What is the scope of the feature?
+
+
+3. Overview
+    > Please provide as much detail as necessary to describe the feature.
+
+
+4. Reference Material
+    > Please provide only links.
+
+
+5. Definitions and Acronyms (optional)
+    > Provide a list of definitions and acronyms if any are used above.
diff --git a/.github/ISSUE_TEMPLATE/issue.md b/.github/ISSUE_TEMPLATE/issue.md
@@ -0,0 +1,23 @@
+---
+name: Tracking issue
+about: Use this template for tracking complaints.
+title: "[DATE]: [ISSUE NAME]"
+labels: tracking issue, needs triage
+---
+
+1. What is the type of this issue?
+    - [ ] Client
+
+        - [ ] Documentation: documentation is missing, incomplete, incorrect, ambiguous, or incomprehensible.
+        - [ ] Functional: the expected or intended functionality is not provided.
+        - [ ] Behavioral: something happened that is not expected.
+
+        > Please limit issues to a single type.
+
+    - [ ] Developer
+
+        > Please include any and all links from this repository suspected of causing the issue.
+
+    - [ ] Operational
+
+        > Please describe what changes are required.
diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md
@@ -0,0 +1,8 @@
+---
+name: Tracking question
+about: Use this template for asking questions.
+title: "[DATE]: [QUESTION NAME]"
+labels: tracking question, needs answer
+---
+
+> Please limit to one question and only supporting statements necessary to convey.
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -0,0 +1,15 @@
+# Description
+> Please provide only as much detail as needed to justify the changes.
+
+# Supporting Material
+> Please provide the links necessary to support the claims made in description above.
+
+# Additional Comments
+> Please provide any additional feedback here.
+
+# Helper Checklist (optional)
+> Please ensure the following has been completed.
+
+- [ ] Changes reviewed and are acceptable to the best of your understanding.
+- [ ] Changes are documented where necessary.
+- [ ] Contacted code owners to evaluate.
diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
diff --git a/README.md b/README.md
@@ -38,6 +38,22 @@ Here is a list of some dependencies used:
 
 The basic Spring Boot application configuration can be found at [src/main/resources/application.yml](https://github.com/vivo-community/scholars-discovery/blob/master/src/main/resources/application.yml). Here you be able to configure basic server and spring configuration as well as custom configuration for Scholars Discovery. There are several configuration POJOs to represent configurations. They can be found in [src/main/java/edu/tamu/scholars/middleware/config/model](https://github.com/vivo-community/scholars-discovery/tree/master/src/main/java/edu/tamu/scholars/middleware/config/model), and [src/main/java/edu/tamu/scholars/middleware/auth/config](https://github.com/vivo-community/scholars-discovery/tree/master/src/main/java/edu/tamu/scholars/middleware/auth/config).
 
+## Assets
+
+Assets are hosted at `/file/:id/:filename` and configured location `middleware.assets-location`.
+
+Tested options are
+
+Assets stored in src/main/resources/assets
+```
+middleware.assets-location: classpath:/assets
+```
+
+Assets stored in externally
+```
+middleware.assets-location: file:/scholars/assets
+```
+
 ### Harvesting
 
 Harvesting can be configured via ```middleware.harvesters``` and represented with [HarvesterConfig](https://github.com/vivo-community/scholars-discovery/blob/master/src/main/java/edu/tamu/scholars/middleware/config/model/HarvesterConfig.java). For each harvester, a bean will be created in which specifies the type of harvester and which document types it maps to. The reference implementation is the local triplestore harvester.
@@ -59,9 +75,7 @@ Solr is configured via ```spring.data.solr```.
 3. Start Solr
 
 ```bash
-   cd solr
-   docker build --tag=scholars/solr .
-   docker run -d -p 8983:8983 scholars/solr
+   cd solr && docker build --tag=scholars/solr . && docker run -d -p 8983:8983 scholars/solr && cd ..
 ```
 
 5. Build and Run the application

diff --git a/pom.xml b/pom.xml
@@ -7,7 +7,7 @@
   <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
-    <version>2.6.14</version>
+    <version>2.6.15</version>
     <relativePath/> <!-- lookup parent from repository -->
   </parent>
 
@@ -29,8 +29,9 @@
 
   <properties>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-    <log4j2.version>2.17.1</log4j2.version>
-    <jena.version>3.17.0</jena.version>
+    <liquibase.version>4.23.2</liquibase.version>
+    <log4j2.version>2.17.2</log4j2.version>
+    <jena.version>4.9.0</jena.version>
     <solr.version>9.1.1</solr.version>
   </properties>
 
@@ -160,7 +161,7 @@
     <dependency>
       <groupId>com.jayway.jsonpath</groupId>
       <artifactId>json-path</artifactId>
-      <version>2.7.0</version><!-- $NO-MVN-MAN-VER$ -->
+      <version>2.8.0</version>
     </dependency>
 
     <dependency>
@@ -172,7 +173,7 @@
     <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-compress</artifactId>
-      <version>1.22</version>
+      <version>1.24.0</version>
     </dependency>
 
     <dependency>
@@ -218,28 +219,20 @@
 
     <dependency>
       <groupId>org.apache.jena</groupId>
-      <artifactId>jena-sdb</artifactId>
+      <artifactId>jena-serviceenhancer</artifactId>
       <version>${jena.version}</version>
-      <exclusions>
-        <exclusion>
-          <groupId>com.github.jsonld-java</groupId>
-          <artifactId>jsonld-java</artifactId>
-        </exclusion>
-        <exclusion>
-          <groupId>org.apache.commons</groupId>
-          <artifactId>commons-compress</artifactId>
-        </exclusion>
-        <exclusion>
-          <groupId>org.slf4j</groupId>
-          <artifactId>slf4j-log4j12</artifactId>
-        </exclusion>
-      </exclusions>
+    </dependency>
+
+    <dependency>
+      <groupId>com.github.ben-manes.caffeine</groupId>
+      <artifactId>caffeine</artifactId>
+      <version>3.1.8</version>
     </dependency>
 
     <dependency>
       <groupId>org.passay</groupId>
       <artifactId>passay</artifactId>
-      <version>1.6.2</version>
+      <version>1.6.4</version>
     </dependency>
 
     <dependency>
@@ -272,14 +265,20 @@
       <version>8.2.1</version>
     </dependency>
 
+    <dependency>
+      <groupId>org.liquibase</groupId>
+      <artifactId>liquibase-core</artifactId>
+      <version>${liquibase.version}</version>
+    </dependency>
+
     <dependency>
       <groupId>com.h2database</groupId>
       <artifactId>h2</artifactId>
       <scope>runtime</scope>
     </dependency>
     <dependency>
-      <groupId>mysql</groupId>
-      <artifactId>mysql-connector-java</artifactId>
+      <groupId>com.mysql</groupId>
+      <artifactId>mysql-connector-j</artifactId>
     </dependency>
     <dependency>
       <groupId>org.postgresql</groupId>
@@ -310,19 +309,26 @@
       <scope>test</scope>
     </dependency>
 
-    <dependency>
-      <groupId>com.github.ben-manes.caffeine</groupId>
-      <artifactId>caffeine</artifactId>
-      <version>3.1.1</version>
-      <scope>test</scope>
-    </dependency>
-
   </dependencies>
 
   <build>
 
     <plugins>
 
+      <!-- USE: mvn liquibase:generateChangeLog -->
+      <plugin>
+        <groupId>org.liquibase</groupId>
+        <artifactId>liquibase-maven-plugin</artifactId>
+        <version>${liquibase.version}</version>
+        <configuration>
+          <driver>org.h2.Driver</driver>
+          <url>jdbc:h2:./h2;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</url>
+          <username>scholars</username>
+          <password>scholars</password>
+          <outputChangeLogFile>src/main/resources/db/changelog/db.changelog-master.yaml</outputChangeLogFile>
+        </configuration>
+      </plugin>
+
       <plugin>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-maven-plugin</artifactId>

diff --git a/solr/Dockerfile b/solr/Dockerfile
@@ -1,4 +1,4 @@
-FROM solr:9.1
+FROM solr:9.3
 
 USER root
 

diff --git a/solr/configsets/scholars-discovery/conf/managed-schema.xml b/solr/configsets/scholars-discovery/conf/managed-schema.xml
@@ -202,10 +202,6 @@
          field first in an ascending sort and last in a descending sort.
     -->
 
-    <!-- The StrField type is not analyzed, but indexed/stored verbatim. -->
-    <fieldType name="string" class="solr.StrField" sortMissingLast="true" docValues="true" />
-    <fieldType name="strings" class="solr.StrField" sortMissingLast="true" multiValued="true" docValues="true" />
-
     <!-- Scholars whole string type  -->
     <fieldType name="whole_string" class="solr.TextField" omitNorms="true" termVectors="false" termPositions="false" termOffsets="false" sortMissingLast="true">
       <analyzer>
@@ -358,6 +354,10 @@
       </analyzer>
     </fieldType>
 
+    <!-- The StrField type is not analyzed, but indexed/stored verbatim. -->
+    <fieldType name="string" class="solr.StrField" sortMissingLast="true" docValues="true" />
+    <fieldType name="strings" class="solr.StrField" sortMissingLast="true" multiValued="true" docValues="true" />
+
     <!-- boolean type: "true" or "false" -->
     <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true"/>
     <fieldType name="booleans" class="solr.BoolField" sortMissingLast="true" multiValued="true"/>