-
Notifications
You must be signed in to change notification settings - Fork 174
DIRAC v6r15
The main new feature in v6r15 release
- New class for handling errors in S_ERROR() return structure
- Changes in the File Catalog client to have all the methods defined locally
- Changes FileCatalog container class for simplifications
- Updates in the File Catalog: recursive operations, dynamic module loading and others
- Changes in the DIRAC installation tools to allow installation from CVMFS if available
- RSS updates adding the CE availability policies
- Enhanced Parametric jobs with multiple parameter sequences
- Initial support for using multi-processor computing slots
- Making use of the Machine-Job-features information if available on worker nodes or in the virtual machines
- Change of Port for ReqProxy service (as of DIRAC v6r15p10)
The PR https://github.com/DIRACGrid/Externals/pull/8 provided several changes to the externals shipped with DIRAC. The new externals tag for servers (v6r4) are compiled for few (more modern) platforms (e.g. for CentOS 7).
New procedures need to be added (BEFORE THE CODE):
ps_set_dir_gid_recursive
ps_set_dir_uid_recursive
ps_set_dir_mode_recursive
And two procedures need to be recreated for bug fix (https://github.com/DIRACGrid/DIRAC/pull/2806) :
ps_delete_replica_from_file_and_se_ids
ps_delete_files
Simply copy paste the code of the mentioned procedures, with eventually the drop statement before, from DataManagementSystem/DB/FileCatalogWithFkAndPsDB.sql
The DirectoryUsage tables will have to be rebuilt at a convenient time by calling the ps_rebuild_directory_usage procedure
As of DIRAC v6r15p10, the port of ReqProxy changed from 9198 to 9161. Please check if this change affects your installation. Related discussion in https://github.com/DIRACGrid/DIRAC/pull/3033
Added a new table HostLogging to the already existing InstalledComponentsDB database. Once the software has been updated, this new table can be automatically created by restarting the ComponentMonitoring service.
This new table will store logging information on the hosts periodically and will now replace the CLI's getHostInfo call ( since checking the database is faster than obtaining the fields for every call ). In order to ensure that the data starts logging into the database it is necessary to restart the SystemAdministrator service once the software has been updated for every host that should have its information logged.
Required settings:
You have to modify the Authorization section of the ComponentMonitor in the CS (starting from v6r15p12):
1. updateLog = group:host
2. getLog = authenticated
TransformationDB TaskInputs table change:
ALTER TABLE TaskInputs CHANGE COLUMN InputVector InputVector MEDIUMTEXT;
The general "from DIRAC.Core.Utilities import *" in the top-level init.py is removed. The DIRAC code is updated to import utilities from where they are defined. However in the extensions such imports can be still done. This should be fixed by the extension developers
The S_ERROR function returns a more complicated structure than it used to before and have 2 alternative ways of calling:
1. S_ERROR( "Error message" )
2. S_ERROR( Errno, "Error message" )
Therefore it retains backward compatibility with all the codes that used this function before. The new form 2. has the first integer argument which represents an Error code. Numerical Error codes are defined in the /DIRAC/Core/Utilities/DErrno.py module. DIRAC extensions can define new specific Error codes in the corresponding DErrno.py module which will be automatically incorporated. The return dictionary contains two additional fields "Errno" and "CallStack". "Errno" contains the numerical error. The "CallStack" contains the calling stack of the line in which the S_ERROR is called. The calling stack is printed out in the logger.debug() function if the new S_ERROR return dictionary is passed to it. In order to facilitate comparison of return structures to a numerical Error a cmpError( inErr, compErr ) function is provided, where compErr is a numerical Erro code and inErr is the result of the new S_ERROR output.
Starting from DIRAC v6r15p8, the CS agents VOMS2CSAgent, GOC2CSAgent and Bdii2CSAgent have a new option flag DryRun. When the option is True, the agents won't modify the CS, rather just print out the proposed changes. The option is False by default, but True for new installations.
The new VOMS2CSAgent agent performs synchronization of the user data in the VOMS service with the DIRAC Registry. It replaces the UserAndGroupAgent. It is working with multiple VOs in multi-VO installations. It also uses the VO data from the CS located in the /Registry/VO section. The /Registry/VOMS section is obsoleted, please, update your Configuration.
On a similar note: the Bdii2CSAgent now completely replaces the CE2CSAgent, which has been removed.
The new GOCDB2CSAgent is used to synchronize information between GOCDB and DIRAC configuration System (CS). For now, it will update the PERFSonar endpoints
SiteDirector Flag to download pilot output to PilotAgentsDB moved to False (was: True)
Since version v6r14p11, DIRAC pilots included by default 2 more commands: CheckCECapabilities and CheckWNCapabilities. These commands have the role to enrich the local dirac pilot configuration file with information including possible tags (e.g. a MultiProcess Tag, if given), the number of available processors, and the total amount of memory of the worker node.
In order to specify the notification Groups, the old field has moved its CS location from
Operations-->Defaults-->ResourceStatus-->PolicyActions-->notificationGroups
to
Operations-->Defaults-->ResourceStatus-->Config-->notificationGroups
There is also a new agent to install:
[user@diracClient ~]$ dirac-admin-sysadmin-cli --host=yourhost.some.where
Pinging yourhost.some.where...
[yourhost.some.where]> install agent ResourceStatus EmailAgent
...
CHANGE: INTEGER -> BIGINT for "id" in "in" accountingDB tables
Use AccountingDB;
show table status WHERE Name like "%\_in\_%";
ALTER TABLE `<INSERT TABLE NAME>` MODIFY id BIGINT AUTO_INCREMENT;
You have to create one "ALTER TABLE" command for each table name given by "show table status..."