0.12.5-7-osf

This webpage was decommissioned. Please visit https://github.com/SentinelDataHub/dhus-distribution/releases to download the latest OSF Release!

Release Notes v. 0.12.5-7-osf

This software release introduces a fix and a new feature with respect to the previous open source distribution (version 0.12.5-6-osf).

---

Important notes

---

A back-up of DB/SolR is strongly recommended before migrating to DHuS 0.12.5-7-osf.

---

What’s new?

DHus Installer
The DHuS Installer will permit to configure the DHuS software automatically.
Please be careful: select properly the installation directory to avoid losing your previous database.

---

Bug Fixes and Implementations
The Init.sql file has been added to enable Scalability configuration.
DHuS_v.0.12.5-7-osf_Release_Notes.pdf

0.12.5-6-osf

Release Notes v.012-5-6-osf.pdf

Release Notes v. 0.12.5-6-osf

This software release introduces a set of new features and performance improvements with respect to the previous open source distribution (version 0.9.1-osf).

---

Important notes

---

Please note that DHuS 0.12.5-6-osf introduces outstanding changes in the database and this implies:   

• installation process more time consuming than usual: the bigger the amount of data stored in DB/SolR, the more time consuming the first start will be.

• the roll-back to previous version (0.9.1-osf) is not available: a back-up of DB/SolR is strongly recommended before migrating to DHuS 0.12.5-6-osf.

---

What’s new?

---

Mission related features
Sentinel-2
Compatibility with the following new product format:

• Compact format
• Level-2A
• Sentinel 2 addon update:
  The following additional indexes are extracted at ingestion time for Sentinel 2 L2A products:
  o VEGETATION_PERCENTAGE
  o BARE_SOILS_PERCENTAGE
  o WATER_PERCENTAGE
  o LOW_PROBA_CLOUDS_PERCENTAGE
  o MEDIUM_PROBA_CLOUDS_PERCENTAGE
  o HIGH_PROBA_CLOUDS_PERCENTAGE
  o NOW_ICE_PERCENTAGE

Sentinel-3
Compatibility with Level-1 OLCI and SLSTR products

---

Security features

• Fixed security issues on AJS client
• Fixed Privacy Violation
• Removed usercode stub
• SQL injection via OData prevented
• Database encryption configurability at the first start.

---

UI features

New features for users:

• SolR suggester, typing in the search bar
• Extensible search bar and list of results
• Sorting criteria on the advances search panel
• Latitude and longitude coordinates for mouse hovering visualization (Google Earth like)
• Product sub-component download in product inspector

New features for administrators:

• Beta version of the Feature shapefile support (set the "enable_shapefile" to true in appconfig.json file)
  This feature allow to use shapefile (file with extension .shp) to define geographical selection in the DHuS AJS GUI. The shapefile can be imported:
  • Dragging and dropping the file directly in the map (only modern browser support this functionality)
  • By means of manual upload, added in advanced search panel
• Allow to update passwords (set the "show_admin_change_password" to true, in appconfig.json file)
  The administrator can now update the user password through a classic update on the User entity.
• Bulk product deletion from the list of products
• Configurable Terms and conditions
• Possibility to configure an SMTP client from the UI without specifying any authentication
• Search by User e-mail from GUI
• OWC integration
  A first version of the Open Web Component is available under http://[dhus address]/new. Please note that to be able to access OWC, DHuS shall be configured such that it is accessible at the following address http://[IP or name]/dhus

---

API features

• OData “Connections” entity: New filters implemented to Identify pending requests via OData
• New filters available for the OData “User” entity to select user created in a certain time span
• Product deletion via Odata

---

Functionality improvement
Synchronizer updates:

• Target collection field has been turned in a drop down menu available in the UI synchronizer panel
• Possibility to stop any synchronizer even if the BE is unresponsive
• The downloads triggered by a synchronization with copy are performed in parallel

Ingestion and eviction updates:

• The products failing the ingestion can be deleted or transferred to a folder configurable in the dhus.xml
• The evicted products can be delete or moved in a folder configurable in the dhus.xml
• Products Deletion performance improved

---

Bug Fixes and Implementations
Here follows the list of bug fixes and implementations released:

---

Mission related features

• Sentinel-1 Acquisition Period Value: S1 product with sensing date between 12:00-12:59 are now indexed correctly
• Sentinel-3: SLSTR Level-0 reconstructed footprint now computed correctly (please note that it remains a visualization problem due to Spatial4j/JTS library)

---

Major functionality updates

• DHuS FTP Service: the DHuS FTP service has been restored
• Fix corrupted data transfer
  This implementation fixes the bugged data transfer process causing downloads corruption in case of connection drop.
• DHuS Scanner Stuck
  This implementation solves the block of the scanners which is experienced on some Operational Hubs
• Remove action records: Statistics are not more stored in the Hub DB tables

---

UI features fixes
Fixes for users:

• GUI search list supporting quicklooks with size different from 512x512
• Feedback during registration or password reset: Now the message says something which make clear that the e-mail AND/OR the username do not exists.
• Feature multi cart icon: Changed icons to add/remove a product to chart and also the icon to add multiple products to chart in products list

Fixes for administrators:

• The FileScanner panel now shows messages on the status of the file scanner, the date of the next scanner run and, when the ingestion is ongoing, a loading icon is displayed.

---

API features fixes

Solr service

• Added solr client connection timeout to avoid deadlocks when too much tomcat threads are used
• Added socket and connection timeouts on SolrClient to avoid threads lock
• Usage of a dedicated Tomcat connector for DHuS for isolating Solr requests
• Introduction of a limit to the number of rows with the /search API

OData v1 service:

This contribution implements classes renaming and moving. Moreover it brings the following improvements:

• Better class naming overall
• Moved Metalink producing code in a utility class
• Add code to work around missing features of Olingo2 (contained targets).
• Do not display the UserSynchronisers EntitySet to all users in the service document
• Collection access restored
• Fix for wring link in OData Users/Cart entity
• Fix for OData substring() filter

---

Functionality improvement
Queries response performance

• OData and Solr query reponse performance improvements

Odata Synchronizer

• The products synchronizer stability is ensured in case of updates
• User Synchronizer stops when trying to synchronize users having forbidden character in username

---

Logs

• Product deleted from the Hub are now logged in the DHuS log

---

System errors fixes

• Fix too many open files: all "too many open files" issue sources are identified. All are fixed, the only remaining aspect to be managed is to close the Zip stream to remove the error: “Cleaning up unclosed ZipFile for archive /x/x/x “
• Update OpenJpeg Native Library
  This contribution fixes the memory leak in native library OpenJpeg causing DHuS crashes in operations with unexpected signal receive error
• Fixed errors during deletion : : avoid time out of SolR when deleting products in huge databases
• Fix compatibility dependency drbx-impl-jp2k
• Fix exception when browsing manifest/dataObjectSection nodes

---

Known issues and limitations:

• The synchronizers are performed sequentially and use the ingestion date to create a synchronization catalogue
• List of deleted product not available.
• Eviction based on the ingestion date.
• Creation of user accounts with malformed e-mail: it is possible to create user accounts with malformed e-mails like "alaanajeeb1965alaa@gmail". This problem causes issue in the user synchronizers.
• Multiple log-in component popped-up (OWC GUI): If not logged, the search action calls a new login component (OWC GUI) to always appear (first time with an error message). Log in panel shall be unique.
• First node inspection not possible for S3A_SR_1_SRA_A and S3A_SR_1_SRA_BS: The NetCdf driver does not use the correct implementation to decode the data.
• Sentinel-3A SLSTR JTS footprints not well visualized: JTS footprints are not well visualized for S3A SLSTR Level 1 and SLSTR Level 2 NTC product types. In some cases it seems that, the footprint has a bottom line (on the pole) which must go from -45 to 160 passing over the 0, but the JTS algorithm is doing the opposite.
• Error in the log at DHuS start: at every DHuS start some errors are logged.They seem to be infos rather than messages or wrong software behaviour.

-Example below:

[0.12.5-3][2016-11-24 13:49:18,973][INFO ] Server started. (DHuS.java:197 - main)
[0.12.5-3][2016-11-24 13:49:18,975][INFO ] Installing webapp SolrWebapp{name:solr; welcome:[]; allow:null; deny:null} (TomcatServer.java:265 - main)
[0.12.5-3][2016-11-24 13:49:19,271][ERROR] Nov 24, 2016 1:49:19 PM org.apache.catalina.core.ApplicationContext log (StreamEncoder.java:221 - main)
[0.12.5-3][2016-11-24 13:49:19,271][ERROR] INFO: No Spring WebApplicationInitializer types detected on classpath (StreamEncoder.java:221 - main)
[0.12.5-3][2016-11-24 13:49:19,290][ERROR] Nov 24, 2016 1:49:19 PM org.apache.catalina.core.ApplicationContext log (StreamEncoder.java:221 - main)
[0.12.5-3][2016-11-24 13:49:19,290][ERROR] INFO: Initializing Spring root WebApplicationContext (StreamEncoder.java:221 - main)
[0.12.5-3][2016-11-24 13:49:19,290][INFO ] Roo...

0.9.1-osf

This software release fixes the bug described in issue #12 "Account hijacking".

0.9.0-2-hotfix-4

Release Notes v. 0.9.0-2-hotfix-4
This software release implements the following fixes:

• OData product synchronizers
  • metadata synch which got stuck
  • synch with product copy thumbnail
  • synchronization between collections
• OData user synchronizers
• User lock/unlock functionality
• Change password

Known issues and limitations
Errors when upgrading DHuS version Process
At first start of new version, after upgrading the DHuS server, the DHuS logs the following errors when the gwt client (related to the old version of the UI) refreshes its UI with browser cache containing old version of UI.
SMTP Server configuration
It's not possible to configure the SMTP server without any authentication.

Products visualization in the list of results
The complete name of the Sentinel-2 product is not visible in the list of result of a full text search via GUI.
FTP Server
The DHuS module allowing users to connect to the DHuS archive via a Read-only ftp connection and view products stored in the archive (organized in folders each one corresponding to a collection) is not working in this version.
Root Password
The DHuS root password changing mechanism via GUI is not working.
The suggested procedure to change root password is the following:

stop dhus     
echo -n <new password> | md5sum     
copy MD5 string      
open HSQLDB with the client     
update users set password='<paste MD5 here>' where    
login='root';      
commit;         
exit        
start dhus   
login with new password    

Please find below dhus-software-0.9.0-2-hotfix-4-distribution.shar

0.9.0-2

Release Notes 0.9.0-2

This software release introduces a set of new features and performance improvements with respect to the previous open source distribution (version 0.4.3-5.2).
New Features

• Sentinel product ingestion compatibility:
  • Sentinel-1 level 2 products ingestion
  • Sentinel-2 products ingestion
  • Sentinel-3 products ingestion
• HTTP partial get compatility
• New UI based on Angular JS
• Country field in registration panel
• Exposed product eviction date, if present, for each product
• Odata synchronization:
  • Basic odata synchronization
  • Odata synchronization including filters
  • Odata synchronization with copy functions
• Expose tomcat configuration parameters
• Removed private products configuration
• Removed statistics functionality
• Quota limitation on partial download
• Introduced the products request top limit (=100) for any odata search query
• Introduction of new OData entites exposing information regarding:
  • Users
  • Network
  • Connections

Improved Performances

• Concurrent file scanners
• Ingestion performance improvements
• Odata performance improvements

Known issues and limitations
Errors when upgrading DHuS version Process
At first start of new version, after upgrading the DHuS server, the DHuS logs the following errors when the gwt client (related to the old version of the UI) refreshes its UI with browser cache containing old version of UI.
SMTP Server configuration
It's not possible to configure the SMTP server without any authentication.
Products synchronization
The copy function during a synchronization is not properly working when the back end and the front end of the infrastructure of the DHuS instance do not share the same filesystem.
User Management
The lock of a registered user account is not possible.
Products visualization in the list of results
The complete name of the Sentinel-2 product is not visible in the list of result of a full text search via GUI.
FTP Server
The DHuS module allowing users to connect to the DHuS archive via a Read-only ftp connection and view products stored in the archive (organized in folders each one corresponding to a collection) is not working in this version.
Root Password
The DHuS root password changing mechanism via GUI is not working.
The suggested procedure to change root password is the following:

stop dhus     
echo -n <new password> | md5sum     
copy MD5 string      
open HSQLDB with the client     
update users set password='<paste MD5 here>' where    
login='root';      
commit;         
exit        
start dhus   
login with new password    

Please find below dhus-software-0.9.0-2-distribution.shar (SHAR file).