diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000..7eb02bb5b --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +* text=auto +*.md text +*.rst text +*.rest text diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml new file mode 100644 index 000000000..4e377c55d --- /dev/null +++ b/.github/workflows/documentation.yml @@ -0,0 +1,25 @@ +name: sphinx + +on: [push, workflow_call] + +jobs: + spellcheck: + runs-on: "windows-latest" + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: install requirements + run: pip install -e .[dev] + - name: run pytest + run: python -m pytest + - name: run spellcheck + run: sphinx-build -E -a -W --keep-going -b spelling doc _build + build: + uses: ISISComputingGroup/reusable-workflows/.github/workflows/sphinx.yml@main + secrets: inherit + with: + deploy-branch: "master" diff --git a/.gitignore b/.gitignore new file mode 100644 index 000000000..d013fa074 --- /dev/null +++ b/.gitignore @@ -0,0 +1,164 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +.pybuilder/ +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +# For a library or package, you might want to ignore these files since the code is +# intended to run in multiple environments; otherwise, check them in: +# .python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# poetry +# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. +# This is especially recommended for binary packages to ensure reproducibility, and is more +# commonly ignored for libraries. +# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control +#poetry.lock + +# pdm +# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. +#pdm.lock +# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it +# in version control. +# https://pdm.fming.dev/latest/usage/project/#working-with-version-control +.pdm.toml +.pdm-python +.pdm-build/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Celery stuff +celerybeat-schedule +celerybeat.pid + +# SageMath parsed files +*.sage.py + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +# pytype static type analyzer +.pytype/ + +# Cython debug symbols +cython_debug/ + +.idea/ +.vscode/ + +.idea/* +.venv/ +ibex_developers_manual.egg-info/ +build/ +_build/ diff --git a/README.md b/README.md index 673634edd..01740a7d0 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,3 @@ # The IBEX Developer's Manual -[Click here to go to the GitHub wiki with the developer's manual.](https://github.com/ISISComputingGroup/ibex_developers_manual/wiki) +[Click here to go to the GitHub wiki with the developer's manual.](https://isiscomputinggroup.github.io/ibex_developers_manual/) diff --git a/doc/Client.md b/doc/Client.md new file mode 100644 index 000000000..4b19fb14e --- /dev/null +++ b/doc/Client.md @@ -0,0 +1,17 @@ +# IBEX GUI (Eclipse) + +## Getting Started + +```{toctree} +:glob: +:titlesonly: + +client/GUI-Getting-Started +client/GUI-Coding +client/GUI-Testing +client/GUI-Eclipse +client/GUI-CSS +client/GUI-Other +client/GUI-Troubleshooting +client/GUI-Design +``` diff --git a/doc/Deployment.md b/doc/Deployment.md new file mode 100644 index 000000000..8154d9576 --- /dev/null +++ b/doc/Deployment.md @@ -0,0 +1,12 @@ +# Releases & Deployment + +```{toctree} +:glob: +:titlesonly: + +deployment/Creating-a-release +deployment/Manual-System-Tests +deployment/Deploy +deployment/Patch +deployment/Future +``` diff --git a/doc/Editing-the-Wiki.md b/doc/Editing-the-Wiki.md new file mode 100644 index 000000000..03e2dd0d3 --- /dev/null +++ b/doc/Editing-the-Wiki.md @@ -0,0 +1,148 @@ +# Editing this Documentation + +This documentation is built using [sphinx](https://www.sphinx-doc.org/en/master/) and the [myst](https://myst-parser.readthedocs.io/en/latest/) markdown +plugin. Both of these tools have excellent online documentation. + +Sphinx is a widely-adopted structured documentation tool, which scales well even for very large projects - for example it +is used to build [Python's documentation](https://docs.python.org/3/), the [Linux kernel documentation](https://docs.kernel.org/), and +the [EPICS documentation](https://docs.epics-controls.org/en/latest/). + +## Markdown + +Markdown is the preferred format, although any format supported by sphinx can be used if needed for example ReST or MediaWiki may be useful if moving documentation from other sources. + +For a 3-minute introduction to Markdown see ['Mastering Markdown'](https://guides.github.com/features/mastering-markdown/). + +## Page Titles + +A top-level heading (e.g. `#` markdown header) is used as the page title. **A page should only have one title** (at the top) - otherwise multiple links to that +page will appear in the navigation structure. Subsections within a page should use sub-headers like `##` or `###`. + +The page title is independent from the filename of the markdown file - however, for clarity, choose to use similar titles and +filenames wherever possible. + +## Adding & editing pages + +For simple edits to existing pages, editing can be done via the github interface. There is an "edit on github" button at +the top of every page, which will take you to the relevant page in github to edit it. + +For more complex changes, for example adding new pages, it is recommended to make the changes and test them locally, to make sure +that the navigation structure renders properly. + +When adding new pages, carefully consider at what level in the hierarchy the new page should be inserted. In particular, be cautious +about adding pages at the very top level of the documentation - these can very quickly clutter the navigation. + +Some sphinx `toctrees` (Table-of-contents trees) are listed explicitly, to promote a natural reading order. Where this is the case, +new pages will need to be added to the `toctree` in the document one level up from where the new page has been added - preserving a +natural reading order. Sphinx will warn you (and fail the build) if you forget to do this. + +## Building the wiki locally + +Check out the wiki into `c:\instrument\dev\ibex_developers_manual`: + +```shell +cd c:\instrument\dev +git clone https://github.com/ISISComputingGroup/ibex_developers_manual.git +``` + +Make a python virtual environment containing the wiki's dependencies: + +``` +cd c:\instrument\dev\ibex_developers_manual +c:\instrument\apps\python3\python.exe -m venv .venv +.venv\Scripts\activate +python -m pip install -e . +``` + +Build the wiki (rebuilding automatically on changes): + +``` +sphinx-autobuild doc _build +``` + +The local wiki will then be available at [http://localhost:8000](http://localhost:8000) in your browser. + +If sphinx gets out of sync with changes, you can clear the cached build output and start again by running: + +``` +rmdir /s /q _build && sphinx-autobuild doc _build +``` + + +## Adding DrawIO Diagram + +Create new diagram + +1. Visit [DrawIO](https://www.draw.io/) choose `device` +1. Create New Diagram, select type etc. +1. Edit diagram until you are happy +1. choose `File` -> `Export` -> `Png...` +1. Then make sure `Include a copy of my diagram` is ticked +1. Add it to the folder in git, next to the page which will use it + +In wiki add to markdown using: + + ![alternative text](.png) + +To edit this just open that png in `draw.io`. + +## Images + +To add images you need to check out the Wiki and add them manually. The images should go into a folder next to the page which will use them. + +```shell +git add some/folder/test.png +git commit -m "Added an image to Using the Wiki page" +git push +``` + +You can then add the image in markdown using the URL `test.png`: + + ![alternative text](test.png) + +## Spellchecking + +The wiki has a built-in spellchecker, which will automatically run when a commit +is pushed to github. + +To run the spellchecker locally, use: + +``` +sphinx-build -b spelling doc _build +``` + +There is an additional dictionary of allowed words in `doc/spelling_wordlist.txt`. This +is used for words which we know are spelt correctly, but are not in the default dictionary. + +The word list can be kept in order by running `sort_word_list` (which will be available in +the python virtual environment). This is also enforced by CI, and can be checked locally by +running `pytest`. Both of these tools should be run from the top-level of an `ibex_developers_manual` +git checkout - the directory that contains `pyproject.toml`. + +Sphinx is strict about capitalisation; where multiple spellings are valid, the word will +need to be added to the word list with each of the valid spellings. An exception is all-lowercase entries in the +word list - these also allow a variant with an initial capital letter. + +Where a word or product has a preferred stylisation, for example **ActiveMQ** or **LabVIEW**, only that spelling +should be added to the word list. Proper nouns should not have uncapitalised variants added to the word list. + +## Admonitions + +Sphinx supports various admonitions, which can be used to draw the reader's attention to +certain topics. + +For example: + +``` +:::{tip} +This is a tip! +::: +``` + +Will render as: + +:::{tip} +This is a tip! +::: + +The list of supported admonitions is [on the MyST documentation](https://myst-parser.readthedocs.io/en/latest/syntax/admonitions.html). diff --git a/doc/Glossary.md b/doc/Glossary.md new file mode 100644 index 000000000..af16de2b1 --- /dev/null +++ b/doc/Glossary.md @@ -0,0 +1,283 @@ +# Glossary + +## ACF + +**Access Control File**. A file used by the [Gateway](#gateway) which contains rules for access. + +## [ActiveMQ](/system_components/ActiveMQ) + +A messaging system that is used in a number of places throughout IBEX. Specifically, [The Alarm Server](system_components/Alarms) and [The IOC Log Server](system_components/IOC-message-logging). See [ActiveMQ Homepage](http://activemq.apache.org/) for more information on the technology. + +## Alarm + +See [the Alarm Server](/system_components/Alarms) + +## Archive Engine + +A [CSS](#css) component that archives [PV](#pv) values. Implemented in [IBEX](#ibex) in the [Block Archive](#block-archive) and [Instrument Archive](#instrument-archive) + +## [Autosave](/iocs/tools/Autosave) + +A system to record PV values and reinstate them on startup of an IOC. + +## [Axis](/specific_iocs/motor_extensions/Axis) + +In IBEX, the term axis refers to a degree-of-freedom within an experimental system. See [Axes in IBEX](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Glossary/Axes%20in%20IBEX%20V1R0M0.pdf) for more detail. + +## Backdoor + +A method of changing the internal parameters of an emulator to mimic the behaviour of the actual device. + +## [Barndoors](/specific_iocs/motor_extensions/jaws/Barndoors-and-Momentum-Slits-on-MUON-Front-End) + +Slits used on each of the Muon instruments to control the neutron flux to the sample. Each "jaw" pivots on one edge, much like a door on a hinge. + +## Block + +## Block Archive + +Archives block values using the [CSS Archive Engine](#archive-engine) and restarts whenever block definitions and/or the configuration changes. + +## [BlockServer](/system_components/BlockServer) + +A [Channel Access](#channel-access-ca) Server (CAS) that allows [blocks](#block) to be configured and [configurations](#configuration) to be created and loaded. + +## Branch + +## [CALab](https://www.helmholtz-berlin.de/zentrum/locations/it/software/exsteuer/calab/index_en.html) + +**Channel Access Laboratory**. A library which enables [EPICS](#epics) and [LabVIEW](#labview) to communicate with each other. + +## [Calibration Files](/system_components/configurations/Calibration-Files) + +Files which contain information about how pieces of sample environment equipment translate their readings into useful data. e.g. temperature sensor calibration files which enable translation from electrical measurements to temperature values. + +## Channel Access (CA) + +A protocol that defines how [PV](#pv) data is transferred between a server and client in an [EPICS](#epics) control system. + +## [Chopper](specific_iocs/Choppers) + +A device, usually a spinning disc of thick metal with a narrow slot, which allows selection of a particular energy range of neutrons by varying its speed. + +## [CLF](https://www.clf.stfc.ac.uk/Pages/home.aspx) + +**Central Laser Facility**. A department at RAL that also uses the [EPICS](#epics) control system. + +## [Code Chats](processes/meetings/Code-Chats-and-Lightning-Talks) + +Short meetings and presentations held within the group to discuss various aspects of the control system, or other related technologies. + +## Collision Detection + +A method for detecting collisions between moving beamline components by simulating the requested move beforehand. See [Collision Detection Project](/specific_iocs/motor_extensions/Collision-Detection-Project) for more information. + +## Commit + +A commit is a snapshot of your repository at a certain point in time. It is also a git command, which saves all of your staged changes on a project. There are various ways to perform a git commit, more info can be found here: [Git Guides - git commit](https://github.com/git-guides/git-commit) + +## Component + +## [Config-Checker](/tools/Config-Checker) + +A Python script to highlight potential issues with current configurations when a new version of [IBEX](#ibex) is installed. + +## [Config-Upgrader](/tools/Config-Upgrader) + +A Python script to upgrade current configurations to be compatible with new versions of [IBEX](#ibex). + +## Configuration + +## ConServer + +## [CSS](http://controlsystemstudio.org/) + +"**Control System Studio** is an [Eclipse](#eclipse)-based collection of tools to monitor and operate large scale control systems, such as the ones in the accelerator community. It's a product of the collaboration between different laboratories and universities." + +## [DAE](/specific_iocs/DAE-and-the-ICP) + +The Data Acquisition Electronics (DAE) is the physical hardware that reads the neutron events out of the detectors. IBEX communicates with this hardware via the Instrument Control Program (ICP). This program is also responsible for combining the neutron and sample environment data into the NeXus file. + +## [Database Server](/system_components/DatabaseServer) + +## DataWeb + +## DB File + +A file containing the definition of [PVs](#pv) for a specific [IOC](#ioc) using [records](#record). + +## Dial values + + +## [Eclipse](http://www.eclipse.org/) + +An IDE and collection of tools for development of [GUIs](#gui). [CSS](#css) and the [IBEX](#ibex) client are based on it. + +## Emulator + +A software implementation of hardware. Usually used to help write and test an [IOC](#ioc) and [OPI](#opi). See [Emulating-Devices](/iocs/testing/Emulating-Devices) for more information. + +## EPICS + +**Experimental Physics and Industrial Control System**. A client/server control system using [Channel Access](#channel-access-ca) as its communication protocol, forming a distributed real-time database of machine values ([PVs](#pv)). +It is a collection of software tools collaboratively developed which can be integrated to provide a comprehensive and scalable control system. + +## Field + +## [Gateway](/system_components/Gateway) + +A service that controls access between two or more networks. + +## Genie Python + +An implementation of the [OpenGenie](http://www.opengenie.org) scripting language in Python, or at least its commands specific to instrument control. Documentation [here](http://shadow.nd.rl.ac.uk/genie_python/sphinx/genie_python.html). + +## GIT + +Git is a distributed version control software. Version control is a way to save changes over time without overwriting previous versions. More info can be found here: [Git Guides: Git](https://github.com/git-guides) + +## Github + +While Git takes care of the underlying version control, GitHub is the collaboration platform built on top of it. It's where most of IBEX source code is kept, along with some other projects. [Git Guides: Git](https://github.com/git-guides) + +## GUI + +**Graphical User Interface**. AKA "[The GUI](Client)" or [IBEX](#ibex) Client. A program which provides a graphical method of interacting with the [IBEX](#ibex) Server. + +## IBEX + +The new ISIS instrument control system. Primarily based on [EPICS](#epics) as the underlying technology. + +## Inhibitor + +## Instrument Archive + +Archives [PV](#pv) values using the [CSS Archive Engine](#archive-engine) which have the ARCHIVE [Field](#field) set in their [Record](#record). + +## IOC + +**Input Output Controller**. A process which reads and writes [PVs](#pv). Often interfaces with hardware (e.g. sample environment equipment) to enable it to be controlled remotely. + +## Java + +## Jenkins + +## Journal Viewer + +Journal viewer is an overloaded term. There are two, one as [part of the ibex GUI](/system_components/Journal-Viewer) and the other is a standalone application supported by the instrument scientists, we only provide data for this. + +## Journal Parser + +The journal parser is a program that runs as part of the end run processes. It looks parses the journal produced by the `isisicp` and adds the details to the database in the journal schema. + +## [LabVIEW](/system_components/LabVIEW) + +**Laboratory Virtual Instrument Engineering Workbench**. A graphical programming language in which the device drivers for [SECI](#seci) are written. + +## [LeWIS](https://github.com/DMSC-Instrument-Data/lewis) + +**Let's Write Intricate Simulators**. A Python framework for producing and running [emulators](#emulator). See [Emulating-Devices](/iocs/testing/Emulating-Devices) for more information. + +## Macro + +A named abstraction for a variable setting, e.g. the macro is PORT, referenced as `$(PORT)`, but the data it is abstracting could be `COM11` or `192.168.0.0`. Regex is used to define the format of the macro in the GUI to ensure that you don't try to use the wrong data type or formatting where the variable is referenced. + +## Mini-Inst + +A term used to describe an instrument which uses components from both the [SECI](#seci) and [IBEX](#ibex) control systems. For example, an [IOC](#ioc) wrapped with a [VI](#vi) on a [SECI](#seci) instrument (using [CALab](#calab)). + +## [Momentum Slits](specific_iocs/motor_extensions/jaws/Barndoors-and-Momentum-Slits-on-MUON-Front-End) + +Slits used on the Muon front-end to control the muon momentum to the three instruments. + +## MySQL + +## [Nicos](/system_components/Nicos) + +A network-based control system. Some elements of it are used in the [script server](#script-server). See [homepage](http://cdn.frm2.tum.de/fileadmin/stuff/services/ITServices/nicos-2.0/dirhtml/) for more information. + +## OPI + +**Operator Interface**. A file used in [CSS](#css) to enable a user to interact with an IOC graphically. + +## Perspective + +## Plugin + +## ProcServ + +## PV + +**Process Variable**. A named piece of data and the primary object of the [Channel Access Protocol](#channel-access-ca), often associated with a piece of equipment (e.g. status, readback, setpoint, parameter). + +## PVLIST + +A file used by the [Gateway](#gateway) which specifies which rules to follow that are defined in the [ACF](#acf). + +## Record + +Used to define a [PV](#pv) in a [DB File](#db-file). + +## Release + +The version of [IBEX](#ibex) to be deployed. + +## Repository ("Repo") + +## Script Server + +## SECI + +**Sample Environment Control Interface. An old instrument control system, used before IBEX. No longer in use on any instruments. Written in C# and incorporating [LabVIEW](#labview) drivers for the sample environment equipment. + +## Sequencer + +An [EPICS](#epics) module to provide support for [SNL](#snl) files. + +## SNL + +**State Notation Language**. A "C-like" language to enable [State Machines](#state-machine) to be written which can then be integrated into [IOCs](#ioc). + +## Soft Motor + +An software implementation of a motor axis to enable conversion between two coordinate systems. + +## State Machine + +A software design in which a program executes _only_ its well-defined states with strict rules governing the changes between them. Written in [SNL](#snl) in this case. + +## Substitution File + +A template for creating [DB files](#db-file) using [macros](#macro). + +## Synoptic + +A graphical representation of an ISIS beamline, showing permanent components as well as Sample Environment equipment. It is shown in a [perspective](#perspective) view in the [IBEX](#ibex) client. + +## Ticket + +## Umbrella Ticket + +## User Values + +## Vagrant + +A tool for building and managing [virtual machine](#virtual-machine) environments. See [homepage](https://www.vagrantup.com) for more information. + +## Virtual Box + +A tool for running [virtual machines](#virtual-machine). + +## VI + +**Virtual Instrument**. A [LabVIEW](#labview) program, usually a device driver for interacting with sample environment equipment. + +## Virtual Machine + +An entire computer system running, without its own dedicated hardware, as software on a host computer. Each of the ISIS control machines (NDX...) is a virtual machine, as well as many other servers used as part of the development process. + +## [Web Dashboard](/webdashboard/Web-Dashboard) + +## ZeroMQ + +Another messaging broker that is used in [NICOS](#nicos) diff --git a/doc/IOCs.md b/doc/IOCs.md new file mode 100644 index 000000000..1dc23ea9e --- /dev/null +++ b/doc/IOCs.md @@ -0,0 +1,111 @@ +# EPICS IOCs & Support Modules + +```{toctree} +:glob: +:titlesonly: +:hidden: + +iocs/* +``` + +## Before Creating an IOC + +Remember, EPICS is a collaboration! Why create an IOC when you can just use one that's already out there? If the device is not ISIS specific, before thinking about creating an IOC check whether it is listed [here](https://epics.anl.gov/modules/manufacturer.php) and, if not, email the mailing list [here](https://epics.anl.gov/tech-talk/index.php). + + +## Creating an IOC + +Building a new IOC? Before you get started, take a look at the relevant [instrument page](https://github.com/ISISComputingGroup/IBEX/wiki) to see if there's any useful information for the device you're about to write an IOC for. Otherwise, let's get going. If you have problems see [See IOC and device Troubleshooting](iocs/Troubleshooting). + +There are two sorts of IOCs, EPICs and pycaspy. For EPICs type see the automatic creation below for pycaspy see the [pycaspy ioc](/iocs/creation/ioc-pcaspy). + +Need more than 1 IOC? See [IOC Duplication](iocs/creation/Duplicating-IOCs) + +### Automatically +If you're building a new StreamDevice IOC you can use the [IOC Generator script](https://github.com/ISISComputingGroup/IBEX-device-generator). + +### Manually +If you're not creating a StreamDevice IOC or you would like to know how an IOC could be created manually see: +* [Creating an ISIS StreamDevice IOC](iocs/creation/Creating-an-ISIS-StreamDevice-IOC) +* [Creating an LvDCOM IOC](iocs/creation/Creating-IOC-wrapper-VI) + +### Fleshing out the IOC +1. You can now flesh out your IOC to work. I would encourage you to do this in a TDD style (if not write the tests afterwards) and make sure you follow the [conventions](iocs/Conventions). To write in a TDD style use: + * [IOC Testing framework](iocs/testing/IOC-Testing-Framework) + * [Emulating devices](iocs/testing/Emulating-Devices) + + If you used the script to create your IOC an empty emulator and IOC tests will have been created for you. + +1. [Record simulation](iocs/testing/Record-Simulation) + +1. [IOC Finishing Touches](iocs/creation/IOC-Finishing-Touches) + * Additional for motors: + 1. [Adding motor IOC to global motor moving and stop all](/specific_iocs/motors/Adding-motor-IOC-to-global-motor-moving-and-stop-all) + 1. [Add support for motor extras, e.g. axes](/specific_iocs/motors/Add-support-for-motor-extras) + +1. [Running (and testing) IOC](iocs/testing/Running-IOCs) + +1. Add any manuals used to the [manuals directory](iocs/conventions/Manuals) + +1. Add a page about the device in the correct categories in [Specific Device IOC](Specific-IOCs) + +1. [Create an OPI](client/opis/OPI-Creation) and send it to instrument scientists for approval. + +## Conventions + +* [IOC naming](iocs/conventions/IOC-Naming) + +* [IOC descriptions](iocs/conventions/IOC-Descriptions) + +* [ISIS PV guide](iocs/conventions/ISIS-PV-Guide) + +* [PV naming](iocs/conventions/PV-Naming) + +* [PV units & standards](iocs/conventions/PV-Units-&-Standards) + +* [Macro naming](iocs/conventions/Macro-Naming) + +* [IOC Startup](iocs/conventions/IOC-Startup) + +## Other + +* [aSub records](iocs/tools/aSub-records) +* [Motor IOCs](specific_iocs/Motors) +* [Libraries to include](iocs/compiling/IOC-Libraries-to-include-with-order) +* [Using LVDCOM](iocs/creation/Creating-IOC-wrapper-VI) +* [Convert Record](iocs/tools/Convert-Record) +* [Stream Device Tips and Tricks](iocs/tips_tricks/Stream-Device-Tips-and-Tricks) +* [IOC Utilities](iocs/tools/Utilities-Library) includes general templates +* [Removing or Renaming an IOC module](iocs/compiling/Removing-or-Renaming-IOC-module) +* [IOC Testing framework](iocs/testing/IOC-Testing-Framework) +* [Complexity of LabVIEW Drivers](system_components/labview/Complexity-of-LabVIEW-Drivers) +* [Typical start up log for an ioc](iocs/troubleshooting/IOC-Start-Example) +* [Some Design thoughts for a serial/Ethernet IOC](iocs/tips_tricks/Some-Design-thoughts-for-a-serial-or-Ethernet-IOC) +* [Reducing build dependencies for older iocs](iocs/compiling/Reducing-Build-Dependencies) +* [Template Substitutions](iocs/tools/Template-Substitutions) +* [Creating a State Machine](iocs/tools/Creating-a-State-Machine-(Sequencer)) +* [Limited range on PV](iocs/tips_tricks/PV-with-a-limited-range) + +## Remake and run IOC + +When modifying and testing an IOC it helps if you chain the commands to make and run the IOC like below: +``` +cd C:\Instrument\Apps\EPICS\support\\master && make clean uninstall && make && cd C:\Instrument\Apps\EPICS\ioc\master\\ && make clean uninstall && make && cd C:\Instrument\Apps\EPICS\ioc\master\\iocBoot\ioc\ && runioc.bat +``` + +## Debugging IOCs + +Debug builds of IOCs are built in jenkins and can be deployed to instruments, see [here](deployment/patch/Deploying-a-DEBUG-build-IOC). + +It can be useful to add a `msgBox` to pause so you can attach a debugger, see [here](#pausing_an_ioc_at_startup) for instructions. If your error is during or after the db load is to call `runioc.bat a` this will start he ioc but run `a` as the start script. Now attach the debug and in the console type `< st.cmd`. + To attach the debugger start the IOC, open visual studio with the code in you wish to debug, set breakpoints as you wish and in the top toolbar select "Debug >> Attach to process" find your process in the list and attach. You can control the debugger as you would other debuggers using the buttons on a toolbar just below the toolbar you clicked on with "Debug" in it. + +## Specific IOC Information + +For information on specific IOCs see [Specific Device IOC](Specific-IOCs) + +## Further reading + +* [EPICS record reference manual](http://www.aps.anl.gov/epics/EpicsDocumentation/AppDevManuals/RecordRef/Recordref-1.html) +* [IOC doxygen Documentation](http://epics.isis.rl.ac.uk/doxygen/main/) +* [IOC message logging](system_components/IOC-message-logging) \ No newline at end of file diff --git a/doc/Overview.md b/doc/Overview.md new file mode 100644 index 000000000..467ff9d64 --- /dev/null +++ b/doc/Overview.md @@ -0,0 +1,42 @@ +# Project Overview + +```{toctree} +:glob: +:titlesonly: +:hidden: + +overview/High-Level-Architectural-Design +overview/New-Starters +overview/First-Time-Build +overview/Links-and-Resources +``` + +## Vision & Architecture + +- [ICP/IBEX Project Overview](https://github.com/ISISComputingGroup/IBEX/wiki): A short overview of the IBEX project, its vision and information on the main instruments it is supporting +- [Architectural Design](overview/High-Level-Architectural-Design): Documents describing the high-level architectural design of the project + +## New Starter's information + +- [New Starter's Guide](overview/New-Starters): For those just joining the team +- [First time build](overview/First-Time-Build): Instructions for setting up IBEX from scratch on a new machine + +## Major IBEX components + +- [IBEX GUI](/Client): Information about the IBEX client; the user interface which scientists see +- [System Components](/System-components): A set of services which make up the IBEX Backend (also called the IBEX Server) + * Including [Python](/system_components/Python), [Configurations](/system_components/Settings-and-Configurations), and many others +- [General IOC information](IOCs): Everything you could possibly want to know about IOCs in general, creation and conventions +- [Specific Device IOC](Specific-IOCs): Detailed information about specific IOCs & scientific hardware +- [Project Tools](Tools): Developer-facing tools which have proven useful for the project +- [System Administration & Hardware](/Systems): All about the physical & virtual computers on which IBEX runs + +## Documentation & Processes + +- [How to edit this documentation](Editing-the-Wiki) +- [How to demonstrate IBEX](processes/meetings_with_scientists/Demo-notes): Some notes on what to show someone when demonstrating IBEX for the first time +- [Notes from Retrospectives](processes/Retrospective-Notes): Notes from Sprint Retrospective meetings are recorded here. +- [Decision Log](processes/Decision-Log): A place to write down decisions made. +- [Documentation](processes/dev_processes/Project-Documentation): Documentation of the project and processes _(including why we have 3 wikis)_ +- [Data Protection](processes/admin/Data-Protection): Data Protection & GDPR as it applies to IBEX. +- [Links & Resources](overview/Links-and-Resources): Useful external links & documentation. diff --git a/doc/Processes.md b/doc/Processes.md new file mode 100644 index 000000000..eb6d699f3 --- /dev/null +++ b/doc/Processes.md @@ -0,0 +1,8 @@ +# Processes + +```{toctree} +:glob: +:titlesonly: + +processes/* +``` diff --git a/doc/Providing-Support.md b/doc/Providing-Support.md new file mode 100644 index 000000000..d55de491c --- /dev/null +++ b/doc/Providing-Support.md @@ -0,0 +1,211 @@ +# Providing Support + +There is a long list of things to worry about when you have the support phone or are dealing with a support issue, and this page should provide a starting point to look for guidance on what to do if no one is around. + +These pages should be a living document - as people learn things add them to the appropriate place if things change, and you notice, update them. If you are experienced, make sure the rest of us are getting things right and verify what is linked to from here. Just remember, this is a public wiki, so be wary of the detail added. + +Don't forget to "Send As" ISIS Experiment Controls in outlook and also cc: ISIS Experiment Controls in the message, that way it will be possible for others to see what has already been discussed. Don't forget to include your name when signing off emails from ISIS Experiment Controls, it's much nicer to deal with a person who is part of a group than an anonymous entity. + +Everyone should start a thread in the "support issues" teams channel for any support work whether that is the emails being dealt with, the calls received, or anything that comes out of people walking into the office. + +If remote, make sure you are either on the VPN or accessing a system on-site in some way. + +The on call number is `x1763` (as called from a RAL phone, and should be available between 7:00-23:00 7 days a week during cycle. +The email address can be found in the address book as `ISIS Experiment Controls`. + +## Contents +- [The types of support](#the-types-of-support) +- [Duties for "first line"](#duties-for-first-line) +- [What does being on call mean?](#what-does-being-on-call-mean) +- [Trouble Shooting](#trouble-shooting) +- [Answering the support phone](#answering-the-support-phone) +- [Basic types of call](#basic-types-of-call) +- [Other Tasks](#other-tasks) + +## The types of support +All permanent members of the ISIS Computing Group are expected to be on the support rotas. + +Note that being the person on rota means you are responsible for seeing that support items are dealt with. This doesn't mean you have to do them, but you have to make sure they are done. + +There is no requirement to change working patterns when on call. + +### Some basic support principles + +If a scientist/user contacts you directly about an urgent issue, you should politely remind them that there is an on call number, and a generic email, and they will be much more likely to get swifter assistance by calling/emailing it rather than individuals. + +If you are working on a problem that you believe has the potential to stop all instruments taking data (e.g. the central beam status is reporting incorrectly or there is a critical bug in a core IBEX component) you should immediately email all instrument scientists using the `ISIS Instrument Scientist` mailing list. You should make them aware of the problem and any workarounds that you have found. + +If you encounter an urgent problem that you are really unsure of you can find other team member's numbers in the support phone's contacts list or at `\\isis\shares\ISIS_Experiment_Controls\On Call\Phone Numbers.xlsx`. + +### Who is on call when? +The "first line" and on call rotas will be made available via the shared calendar as is currently the case. +There will always be someone who is "first line", but those duties vary in and out of cycle. +On call only exists in cycle. + +## Duties for first line +This role is active between 8:30 and 17:00, every weekday, generally with a different person on duty each week. +You will keep an eye on the support emails, and any voicemails, and make sure they are dealt with by someone. Chase up if something doesn't happen. +If you don't have the experience to resolve something, then you should ask the team for help. + +Your phone will ring first for support issues, be prepared to answer it. + +If you don't have the experience to resolve the issues, then in cycle you should ask whoever is on call for help first, followed by others in the team. Out of cycle ask the team. + +Extra note for the first week of cycle, these are usually busy, be prepared to delegate tasks. It may be necessary to ask specific individuals to undertake tasks, and you may need to keep track of who is dealing with which problem and where they are. It would also be wise to share the issues as appropriate, but do remember that working patterns and individual needs may play a part in who can help with what. + + +## What does being on call mean? +This role is active between 07:00 and 23:00, everyday in cycle, generally with a different person on duty each week. + +An on call week starts on a Friday and finishes a week later. (So that the first on call week of a cycle starts the Friday before cycle starts, and the final on call week ends with cycle.) + +During working hours the person scheduled for first line will receive phone calls first, therefore the duties here are more focussed on the out of hours time (07:00 to 08:30 and 17:00 to 23:00). + +Your mobile phone will ring, be prepared to answer it, and deal with the issue detailed. + +If the person on first line support is inexperienced, be ready to support them and take calls from them. + +## Trouble Shooting + +There are a number of tips for [trouble shooting](#all_troubleshooting_links) already on the wiki. + +## Answering the support phone +
+ Steps to answer the phone. + + 1. Greet the caller with something that tells them they are talking to the right team, e.g. just respond with "ISIS Experiment Controls Support" + 1. Make a note* of the time + 1. Make a note* of the name of the instrument and the name or at least the role of the caller, if possible - sometimes they are quick and you don't get to catch it, or they don't actually say who it is. These calls can also come from visiting users in cabins, or from the Main Control Room (MCR), knowing who called you about the problem can help if others need to follow it up. Check how best to call them back - scientist may call from cabin, but say to email or phone back on their mobile in case they are no longer in the cabin. + 1. Make a note* of the basic problem. Useful details like: + 1. When did the problem start happening (this helps us to know where to look for in log files) + 1. Is it happening now, or only when you try and do something + 1. What events led up to the problem happening e.g. which commands, opened something on GUI, connected new equipment, ... + 1. how urgent is it - do they have a workaround for now etc. When is a convenient time for us to start to investigate issue. + 1. If you can solve the problem do so, if you can't start finding the appropriate answers in this guide or by reaching out to others (if onsite, ask in office - if offsite, post in teams). It is OK to tell the user you will call them back + + * Notes can be mental notes - but don't be afraid to write everything down either, you have to write it all down for the out of hours calls anyway. +
+ +## Basic types of call +
+ I can't log in to *** computer + + 1. What is the name of the computer? + - If it is the NDX or NDH, we care, look at the next steps, there are a small subset of other systems we support that others might be logging into which will be listed in the older SharePoint along with the access information as appropriate. + - Anything else, NDC, NDL, NDW, cabin PCs, or an ISIS or CLRC domain account as opposed to a local instrument account: + - in office hours refer them to the Facilities IT service desk + - out of hours, if in cycle then in zoom phone call "ISIS IT Infrastructure on-call" (phone number 94499) + - If attempting to connect to EMMA, remember the -A + 1. Unable to connect to NDX via RDP + - Try yourself to RDP, if you can ask reporter to try again, if they can't it is a connectivity issue for the system they are using to site, in hours refer them to the service desk, out of hours this is best efforts. Check the ISIS Computing O365 SharePoint for more information. + - Check ping to the NDX, and to the NDH + - If there are any issues with these pings, you can check DRAC. [Basic DRAC information here](https://stfc365.sharepoint.com/sites/ISISExperimentControls/SitePages/Getting%20to%20Instrument%20DRACs.aspx) and instrument [DRAC IDs here](https://stfc365.sharepoint.com/sites/ISISExperimentControls/Lists/Instrument%20Control%20Computers/AllItems.aspx). + 1. Wrong password entered too many times on anything other than NDX + - We can't resolve this +
+ +
+ I can't see the dataweb for {instrument} + + 1. Check whether or not you can see that dataweb, and how extensive the failure is (one instrument, many, all) (TODO: Find out the solution for this, is it always restart the dataweb server/JSON_BOURNE? + 2. Try the troubleshooting section on the [dataweb](#webdashboard_troubleshooting) + 3. If this doesn't work try restarting ndaextweb3 + +
+ +
+ Device issues + + 1. I can't talk to device/my blocks are showing as disconnected/IOC isn't working + - Check that the IOC is running + - Check that the device is turned on + - Check if your problem is already listed on the [device/ioc wiki pages](Specific-IOCs). + - If the device is a DAQmx one, look at it in MAX, and perform a self-test + - Device not responding + * Stop the IOC (or VI) and try to connect via a more direct route, e.g. Putty + * Check the cabling, and that ports etc. are correct + * For serial devices, check using the MOXA web interface that bytes are being sent and received on the correct port. Moxa IPs are listed [here](http://beamlog.nd.rl.ac.uk/inst_summary.xml) and the login details are on the usual passwords page. + - If the device is unable to interact in any way, refer this to the appropriate hardware team (via the MCR out of hours) + - If the device responds via a more direct route, but not via the IOC/VI + * Make sure the settings in the IOC/VI are correct + * If the IOC/VI have been updated since the device last worked correctly, roll back to a version that is known to work, and raise a ticket to investigate the issue and find a more sustainable solution + 1. I can't use this button to get to more details/why doesn't this bit of the OPI work + - Check they are in manager mode + 1. I need to add this device to my system + - Check [the user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki) for IBEX, for SECI, if you don't know already ask someone else + 1. My motor won't move + - Are both limits made? + - Yes: Something has happened in the physical realm, refer it to EUSG via the MCR + - No: Go to next consideration + - Is any of the other information updating for that motor controller? + - Yes: Go to next consideration + - No: Under IBEX go to the engineering device screen, under SECI open the advanced motor functions and go to the console tab, do not send any characters but send a command, if the response is anything but `:` then the Galil is in a fault mode of some kind which will involve restarts etc. + * If the Galil is unresponsive refer it to EUSG via the MCR + - Looking at the specific motor: + - Are you trying to move in the same direction as an active limit switch? + - Yes: Try moving in the other direction, if you can move that way to a requested position all is fine + - Are you able to move in either direction? + - No: Check for hardware faults (potentially as a referral to EUSG via the MCR) + - If it is able to move, is the encoder tracking in the same direction as the requested motion? + - No: The motor setup is probably incorrect, recommission the motor + - Yes: Feel bewildered as this should be a moving motor + 1. My device isn't behaving as I expect + - Verify that the device expected on that port is the device connected on that port + - Treat it as a device that is unable to interact + - Check the error logs (either through the log interface in the GUI, the console logs, or other appropriate method) + - Verify that the behaviour you're seeing is not a known quirk of the device. These quirks are often detailed on the [device/ioc wiki pages](Specific-IOCs) + +
+ +
+ This network device isn't connecting + + 1. If the device has been working on the ISIS network in that specific hall since the start of May 2025 then this is most likely an issue with the device or the port. + 1. If this is a new to that hall, or to ISIS generally device, or it hasn't been used since sometime before May 2025, then this may be a network access issue, and it will need some help from the Infrastructure team, the MCR will have a number for them. + +
+ +
+ A service isn't running + + 1. There are a few things that have services which run, especially the databases, and it is possible after a crash/other restart that these don't start up again, starting task manager as an administrator should allow you to start the service in question + 1. If it is not one of our services (e.g. swipe systems, ERA), we cannot resolve the issue, escalate as appropriate (TODO: Make sure the different escalation methods are documented) + 1. If the MCR news service isn't working, then so long as there is space, restarting our [webserver](/systems/Webserver) may help. + +
+ +
+ Script issues + + 1. My script won't load + - If `g.load_script` is being used and you see errors of the form `E: 1: error description (error-name)`, these errors are coming from the linter. Detailed linter troubleshooting is available [here](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Error-Checking-Troubleshooting). + 1. My script isn't behaving in the way I expect it to + - This is a best efforts, and not everyone can provide the same level of support + - Look at it with respect to basic coding standards and obvious race condition points + - (TODO: Complete this section) + +
+ +## Other tasks +
+ Checking Nagios + + 1. This is usually considered during the daily stand up + 1. Out of cycle we only worry about the most critical items + 1. In cycle there are more things to be aware of and waiting until the next stand up meeting can be too long + 1. Some issues can only be solved by specific individuals, it is still worth making sure they are aware that there is a problem as they might not have seen the issue yet + 1. Any space issues in a computer + - Check that there isn't just one large raw/nexus file - if there is then the warning should disappear soon or it is a different problem than just space, otherwise go on to the next step + - Contact the Instrument Scientists to let them know that something needs doing to their system, and ask when you might be able to take over their computer to ensure they have enough space + - When there is an opportunity delete some of the oldest log files, if the database (IBEX systems only) is large then trim it as per an update + 1. The CPU/memory usage is high + - Check which processes are high, and the graph to see how long they have been high + - If this looks like a steady climb, if you can determine the source create a ticket for investigation + - Contact the Instrument Scientists to let them know that there is an issue, and ask them to restart appropriate items when they have a chance. Note that restarting the IBEX server is least likely to be required. +
+ +
+ Finding Instrument Scientist Phone Numbers + +To find contact numbers for instrument scientists and cabin phone numbers for instruments, the following website is available: [Instrument Map](https://www.isis.stfc.ac.uk/Pages/Instruments.aspx). On this page you can either click on the instrument (in the top image), or click the `Contacts And Support > Instrument Scientists` tab and find the instrument you are looking for. +
diff --git a/doc/Script-Generator.md b/doc/Script-Generator.md new file mode 100644 index 000000000..88279c0b0 --- /dev/null +++ b/doc/Script-Generator.md @@ -0,0 +1,178 @@ +# Script Generator + +The script generator developed as part of IBEX is held in the same repository as the IBEX client/GUI. The script generator requires both a perspective in the IBEX GUI and a standalone application. + +To achieve this, the script generator can be built in both eclipse and using maven, with minimal packages required from the main GUI codebase. The build system for the standalone application mirrors that of the main IBEX gui, with an independent `tycho.parent`, see [below](#the-directory-structure) + +To prevent duplication, the GUI perspective depends wholly on the standalone application. + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +script_generator/* +``` + +## The directory structure +| Purpose | Location | +| -------------- |----------| +| Perspective for the IBEX GUI ONLY | `base/uk.ac.isis.ibex.scriptgenerator` | +| Standalone app build directories | `base/uk.ac.isis.scriptgenerator/*` | + +## Eclipse build +To build and run the app through eclipse, the script generator product is found in `base\uk.ac.stfc.isis.scriptgenerator.client.product`. The instructions for setting up eclipse and loading the target platform (necessary for the script generator) are found in the instructions for the [main IBEX GUI setup](/client/getting_started/Building-the-GUI) + +{#scriptgenerator_enable_perspective} +### Adding the perspective into the client +Currently the script generator perspective is not shown or selectable in the main IBEX GUI. It can be re-added by selecting `Preferences > Select Visible Perspectives` from the IBEX Main Menu. + +## Maven build +To build the standalone app through maven run `build\build_script_generator.bat` + +There is a Jenkins pipeline which will build the script generator with every new commit to the IBEX GUI master branch. + + +## Data structures in the GUI + +Each 'action', or step in a script is represented in the GUI as a row. Each cell contains one parameter for an action, which the users change to define their experiment. A complete row of defined parameters should be enough information to run the action once (see [Script Generator High-Level Design](#scriptgenerator_script_definition)) + +![](scriptgen.png) + +The parameter values are stored as strings in the underlying action. These strings are passed to the python process for validation, and to insert them into the output script. When the value of an underlying cell is updated, the GUI is notified by firing a property change (`actions`). + +To change the type of action represented in the table, the singleton drops the current table and replaces it with a new one constructed using the new action type. + +## Importing user-supplied script definitions +The `ScriptDefinitions` are supplied by instrument scientists in [this form](#scriptgenerator_script_definition). All the python modules containing an `ScriptDefinition` in a subdirectory are imported and stored in a `ScriptDefinitionWrapper` class. Using an instrument scientist supplied `ScriptDefinition`, a `ScriptDefinitionWrapper` class is created containing the `ScriptDefinition` with helper methods like `getParameters` and `parametersValid` to expose the parameters used in the action and their validation. These `ScriptDefinitionWrapper` classes are collected by the `ScriptDefinitionsWrapper`, which is then passed through the py4j interface to the GUI code. + +On startup of the script generator, the latest versions of the Script Definitions are pulled from [this repository](https://github.com/ISISComputingGroup/ScriptDefinitions). The default location for this repository is in `path_to_script_generator\script_definitions` but can be changed in the `plugin_customisation.ini`. If this directory does not contain the ScriptDefinitions repository on script generator startup, a clone is attempted. If a clone of the ScriptDefintions repository failed from github, a clone is made from a `git bundle` form of the repository, which is shipped with the script generator and is created when the script generator is built. + +Once in the GUI, the script generator table is created by generating `ActionParameters` from the parameter names exposed through the ScriptDefinitions class. + + +![Script generator code structure](ScriptGenerator.png) + +## Checking the validity and generating python scripts + +Generator classes are used to check validity and generate scripts. They use the script definition's supplied `run` and `parameters_valid` methods to check validity and generate scripts. We do not want to block the execution of the UI thread, and as such, all parameter validity checking and script generation is done by running a Java CompletableFuture and then firing a property change and passing this up the chain to the UI. This allows updates to happen in the background without blocking the UI thread. + +## Saving the Parameter values and loading back up +Parameters values from the Script Generator can be saved in a file. The file format chosen is JSON due to its flexibility and well-tested parser available for upgrading it using upgrade script in future. + +Parameters are saved as `.sgp` (Script Generator Parameters) files. + +The format of the chosen JSON file is as follows: + +``` +{ + "version_JSON_format": "", + "script_generator_version": "", + "date_and_time": "", + "script_definition_file_path": "", + "script_definition_file_git_hash": "", + "genie_python_version": "", + "script_definition_content": "", + "actions": [ + { + "action_name": "", + "parameter": [ + { + "param1": "value", + "param2": "value" + } + ] + }, + { + "action_name": "", + "parameter": [ + { + "param1": "value", + "param2": "value" + } + ] + } + ] +} + +``` +Note: + +- For each action in `Actions` in the JSON file, there is a table in the script generator. +- The `version` in the file is the version of the JSON format itself. This will be an integer and not tied to any other version number. +- configuration_path: this is the path of the configuration file within the git repository. Allowing us to group then by instrument if this becomes useful. +- configuration_file_git_hash: this is the git hash at the time the config file is used +- `configuration_content` contains the actual content of the config file that was used when saving parameters. This is to make sure that the configuration that was used to generate JSON file is the same configuration that is being used to load the data from the JSON file. + +We want the JSON in a separate file so that: +* It's language independent +* The user is less likely to modify it +* It would probably easier to upgrade at a later date if the JSON format changes + +The generated script file will contain the compressed JSON. The compressed JSON can be used as a back up if the JSON file is lost to retrieve original values. For the MVP reading the JSON from the script will not be done and we will only read the JSON file. + +JSON formats etc. were decided in a meeting on 20/02/2020 attended by Bish, Dom, John, Tom, Alistair and James. + +On 12/05/2021 it was decided by James, Kathryn and Dom to hide this complexity from the user by removing references to parameter files and tying the two files directly to generating scripts. This was captured in [#6492](https://github.com/ISISComputingGroup/IBEX/issues/6492). + +## Pre-release Install Instructions + +As dev: + +1. Copy build to releases area (this is two part because of the deeply nested python) first copy to icp area then move to releases release number +1. Zip this folder because it is easier to get this on a machine +1. Copy current script definitions git repo to here +1. Then issue following instructions to user: + +``` +These are the instruction to install the beta release of the script_generator. It needs to be installed on a windows machine but does not need ibex running. To install this please do the following: + +Either: + + i: Quicker but requires more disk space + 1. copy the zip file from to somewhere on your local machine. + 2. Unzip it + 3. Click on install_script_generator.bat + 4. Delete zip file and unzipped files + + ii: Slower but requires less disk space + 1. Open the folder + 2. Click on install_script_generator.bat + +This will take a while to copy the script generator to your machine (it took at least 2 hours). The final directory is ~ 3 Gb and has ~50k files. + +1. Then copy the script definitions to your local machine from to c:\ScriptDefinitions + +Finally: + +1. Create a script dir to save scripts to in `c:\scripts` + +You should then be good to go just double click C:\Instrument\Apps\script_generator\ibex-script-generator.exe. If it starts but with an empty window it may be that it failed to load something earlier in that case delete the folder c:\instrument\app\script_generator\workspace and try again. + +``` + +## Script definitions + +Script definitions are configuration files which specify the following in the script generator: +- How many columns to display and what their names are +- Any constraints on the data entered under a particular column +- How to "run" a row of the script generator (i.e. the python code that will eventually be executed) + +The script definitions are stored in `c:\instrument\settings\ibex_script_generator\scriptdefinitions`. This location should exist on an instrument control machine. If this location does not exist, the script generator instead looks for a folder named `scriptdefinitions` next to the executable. + +## Dynamic Scripting + +[See here](script_generator/IBEX-Script-Generator-Dynamic-Scripting) + +## Interested instruments/groups + +- Muons +- Reflectometry +- SANS +- MARI +- WISH +- IMAT +- HRPD +- ENGIN-X +- MERLIN diff --git a/images/ScriptGenerator.png b/doc/ScriptGenerator.png similarity index 100% rename from images/ScriptGenerator.png rename to doc/ScriptGenerator.png diff --git a/doc/Scripting.md b/doc/Scripting.md new file mode 100644 index 000000000..122371397 --- /dev/null +++ b/doc/Scripting.md @@ -0,0 +1,13 @@ +# Scripting (Python) + +```{note} +These pages document user-facing scripting workflows. Looking for developer information about python? See [Python](system_components/Python). +``` + +```{toctree} +:glob: +:titlesonly: + +scripting/* +``` + diff --git a/doc/Specific-IOCs.md b/doc/Specific-IOCs.md new file mode 100644 index 000000000..d1474ca7c --- /dev/null +++ b/doc/Specific-IOCs.md @@ -0,0 +1,11 @@ +# Specific Devices & IOCs + +These are the list of categories for an IOC. There are details relating to specific device IOCs listed from within the categories where applicable. Most IOC control a device which has [manuals](iocs/conventions/Manuals). + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +specific_iocs/* +``` diff --git a/doc/System-components.md b/doc/System-components.md new file mode 100644 index 000000000..fc01aef5e --- /dev/null +++ b/doc/System-components.md @@ -0,0 +1,41 @@ +# Backend System Components + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +system_components/* +``` + +This is a table of the major components of IBEX server. + +Name | Console Name | Type | What it does +--- | ------------ | ----- | ------------ +[Active MQ](system_components/ActiveMQ) | `JMS` | Java | ActiveMQ used to transmit log and alarm data +[Alarm server](system_components/Alarms) | `ALARM` | Java | Serves alarms which appear in the alarms perspective. It checks to PVs to see if they are in alarm mode and relays that information to the client. Actual range checking is done a EPICS server. +[Archive Access](system_components/Logging-from-the-archive) | `ARACCESS` | python | Creates log files based on the MySQL database for some devices with special needs. +Archive Engine | `ARBLOCK` | Java | Archives (in mysql db) blocks set in a configuration (restarted when blocks change) see [CSS-Archive-Engine](system_components/CSS-Archive-Engine) +Block Gateway | `GWBLOCK` | EPICS Gateway | Aliases the dynamically created block PVs e.g. CS:SB:FURNACE_TEMP to the underlying PV e.g. EUROTHRM_01:A01:TEMP see [Block server](system_components/BlockServer) +[Block server](system_components/BlockServer) | `BLOCKSRV` | Python | Manages configurations and blocks associated with them see [Settings-and-Configurations](system_components/Settings-and-Configurations). It configures the Archive Engine and starts the IOCs read from the configuration files. +CA Repeater | `CAREP` | Executable | A epics CA repeater that is started before all other processes. This repeats UDP broadcasts to CA clients on the same machine +[Database server](system_components/DatabaseServer) | `DBSVR` | Python | Intermediary between MySQL and the GUI, only used for PVs that hold instrument information, such as experiment data, which IOCs are used and information about PVs of an instrument. Legacy software, not that necessary now. +[Experiment DB](https://github.com/ISISComputingGroup/ExperimentDatabasePopulator) | N/A - runs centrally | Python | Similar to DB server but for PVs for the experimental database which contains users and run numbers. +[External/Access Gateway](system_components/Gateway) | `GWEXT` | EPICS Gateway | Gateway to allow access to PVs from outside localhost +[icpconfig](iocs/tools/icpconfig) | N/A | Library of functions | functions called on IOC start up to load macros into the IOC. +[Inst etc](system_components/Inst-etc-IOC) | `INSTETC_01` | Epics IOC | PVs which are for instrument level, e.g. motors moving and security pvs +Instrument Archive | `ARINST` | Java | Archives (in mysql db) pvs with archive property see [CSS-Archive-Engine](system_components/CSS-Archive-Engine). It is a separate archive engine from Archive engine. PVs with archive property are PVs set by developers to be logged always, regardless of what scientists do, so that we can use them for diagnostics. +[IOC Message Logger](system_components/IOC-message-logging) | `IOCLOG` | Java | Instrument level software that collects messages sent by any IOC, logs it in the database and put it on the JMS message queue. +[ISIS DAE](specific_iocs/DAE-and-the-ICP) | `ISISDAE_01` | Epics IOC | Controls the ISIS ICP program which collects data. It is an IOC-like needed because ICP can not talk over Channel Access. +[LabVIEW](system_components/LabVIEW) | | LabVIEW | Controls certain legacy devices which have not been migrated to EPICS yet. +[MySQL](system_components/MySQL-Database) | runs as a service | Service | My SqlDatabase is used for persisting data +[Nicos](system_components/Nicos) | `NICOSDAEMON` | Python | Process which runs and queues users' python scripts on an instrument +Proc Serve Control | `PSCTRL` | Epics IOC | Control proc serves (start, stop and status). A wrapper that makes starting and stopping IOCs much easier. +[Python (Uktena)](system_components/Python) | | python | Python distribution. +[Remote IOC Server](system_components/Remote-IOCs) | | pcaspy | Manages IOCs hosted on PCs other than the local instrument. +[Run Control](system_components/Run-control) | `RUNCTRL_01` | Epics IOC | Add run control to blocks. Run Control is a feature of IBEX that allows users to configure the instrument so when a PV is out of a certain range, neutron data is not being gathered. Needed because neutron data has an extremely large volume and sometimes that data would not be needed so it is better to not pollute the Nexus files with it. +[Script server (Interface to Nicos)](system_components/Nicos) | `SCRIPTSERVER` | Python | Script server is a proxy in front of nicos to allow communication with it. +[The Journal Viewer](system_components/Journal-Viewer) | N/A | Database, webserver and part of the client | Provides information about what experiments have been done on the instrument in the past + +All components are started via [Startup and Shutdown](system_components/Startup-and-Shutdown) diff --git a/doc/Systems.md b/doc/Systems.md new file mode 100644 index 000000000..c0f56d188 --- /dev/null +++ b/doc/Systems.md @@ -0,0 +1,9 @@ +# Systems Administration & Hardware + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +systems/* +``` diff --git a/doc/Tools.md b/doc/Tools.md new file mode 100644 index 000000000..059efa33c --- /dev/null +++ b/doc/Tools.md @@ -0,0 +1,54 @@ +# Tools + +```{toctree} +:glob: +:titlesonly: + +tools/* +``` + +## Caget, caput, zhex and uzhex + +To get the value of a compressed, hexed PV (such as those on the blockserver) type into an EPICS terminal: +`caget -S -t PV | uzhex` where PV is the PV you are looking for + +To send compressed, hexed values to a PV (such as those on the blockserver) type into an EPICS terminal: +`for /f "tokens=*" %x in ('zhex VAL') do caput -S PV %x` where PV is as above and VAL is the data you wish to send + +## Console + +The console can be used to inspect output from an IOC. To use it, launch the IBEX server and then, from an EPICS terminal, run + +``` +console -M localhost [IOCNAME] +``` +and then if you see +``` +[Enter `^Ec?' for help] +[-- MOTD -- IOC console for IOCNAME] +``` +then press Ctrl+X to start the IOC. +The IOCs available can be listed using +``` +console -M localhost -x +``` + +To look at an IOC remotely + +``` +console -M [MACHINE_NAME] -l [USERNAME] [IOCNAME] +``` + +where username is the local user on the instrument PC (e.g. `spudulike`). + +Key combos in the console: + +- `+X`: stop/restart an IOC +- `+E` then `C`, then `.`: exit the console press +- `+T`: toggle auto restart process on/off + +If attempting to run your console gives an error about port 782 being in use, the issue may be due to incorrect permissions in your Instrument\Var folder. Try moving/deleting/renaming your current var (you may need to stop the mysql service), then running `config_env.bat` to generate a new one and `upgrade_mysql.bat` in IBEX_Utils to rebuild your database. + +## Modbus + +[QModMaster](https://sourceforge.net/projects/qmodmaster/) is a useful piece of software to probe Modbus devices much like Putty is used with serial/Ethernet connected devices. diff --git a/doc/Web.md b/doc/Web.md new file mode 100644 index 000000000..55fc59616 --- /dev/null +++ b/doc/Web.md @@ -0,0 +1,15 @@ +# Web Dashboard & Chat + +```{toctree} +:glob: +:titlesonly: + +webdashboard/Web-Dashboard +webdashboard/PVWS +webdashboard/ISIS-Info-Slack +webdashboard/MCR-News-in-Teams +``` + +```{seealso} +- [WebServer](systems/Webserver), for the physical computer on which the web dashboard is hosted. +``` diff --git a/doc/_static/css/custom.css b/doc/_static/css/custom.css new file mode 100644 index 000000000..1b2b3cd9f --- /dev/null +++ b/doc/_static/css/custom.css @@ -0,0 +1,14 @@ +.wy-table-responsive table td, .wy-table-responsive table th { + white-space: inherit; +} + +a:hover { + color: #01afec; +} + +.wy-menu-vertical p.caption { + color: #01afec; +} +.wy-nav-content { + max-width: 1200px !important; +} diff --git a/doc/_templates/custom-module-template.rst b/doc/_templates/custom-module-template.rst new file mode 100644 index 000000000..f483c4114 --- /dev/null +++ b/doc/_templates/custom-module-template.rst @@ -0,0 +1,40 @@ + + +{{ ('``' + fullname + '``') | underline }} + +{%- set filtered_members = [] %} +{%- for item in members %} + {%- if item in functions + classes + exceptions + attributes %} + {% set _ = filtered_members.append(item) %} + {%- endif %} +{%- endfor %} + +.. automodule:: {{ fullname }} + :members: + :show-inheritance: + + {% block modules %} + {% if modules %} + .. rubric:: Submodules + + .. autosummary:: + :toctree: + :template: custom-module-template.rst + :recursive: + {% for item in modules %} + {{ item }} + {%- endfor %} + {% endif %} + {% endblock %} + + {% block members %} + {% if filtered_members %} + .. rubric:: Members + + .. autosummary:: + :nosignatures: + {% for item in filtered_members %} + {{ item }} + {%- endfor %} + {% endif %} + {% endblock %} \ No newline at end of file diff --git a/doc/client/GUI-CSS.md b/doc/client/GUI-CSS.md new file mode 100644 index 000000000..5ee748a59 --- /dev/null +++ b/doc/client/GUI-CSS.md @@ -0,0 +1,39 @@ +# CS-Studio Views in the GUI + +## Source Code + +We have our own fork of the CSS source code [here](https://github.com/ISISComputingGroup/cs-studio). This should be kept up to date with the original code [here](https://github.com/ControlSystemStudio) and only be used to branch from when modifying CSS for our own purposes. Modifying CSS should be done in discussion with the Core CS-Studio Developers and once any modifications have passed our review every effort should be made to merge them with the original. + +CSS is built via jenkins. Clone [this repository](https://github.com/ISISComputingGroup/isis_css_top) recursively and build using `build.bat`. + +## Archive Engine + +The Archive Engine is part of CSS and is used to archive PV values. Note that most facilities use the [Archive Appliance](https://slacmshankar.github.io/epicsarchiver_docs/index.html) as the Archive Engine (specifically MySQL) was found to be slow at millions of PVs. As we don't have that many PVs we have continued to use the Archive Engine. For more details see [here](/system_components/CSS-Archive-Engine). + +## Alarm Server + +The Alarm Server is part of CSS and alerts the GUI to any PV alarm changes. There is also a CSS Alarm View that we use as part of our GUI. + +* [Building the alarm server for MySQL](/system_components/alarms/Building-the-alarm-server-for-mysql) + +* [Using the alarm server with MySQL](/system_components/alarms/Using-the-alarm-server-with-mysql) + +## Operator Interfaces + +OPIs are the main way that users interact with an IOC. They are GUIs created in CSS and imported into our GUI. + +```{toctree} +:glob: +:titlesonly: + +opis/* +``` + +## Other + +```{toctree} +:glob: +:titlesonly: + +css_other/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Coding.md b/doc/client/GUI-Coding.md new file mode 100644 index 000000000..d8b835137 --- /dev/null +++ b/doc/client/GUI-Coding.md @@ -0,0 +1,8 @@ +# Coding + +```{toctree} +:glob: +:titlesonly: + +coding/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Design.md b/doc/client/GUI-Design.md new file mode 100644 index 000000000..57c1a2b82 --- /dev/null +++ b/doc/client/GUI-Design.md @@ -0,0 +1,8 @@ +# Design Documents + +```{toctree} +:glob: +:titlesonly: + +design_docs/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Eclipse.md b/doc/client/GUI-Eclipse.md new file mode 100644 index 000000000..f4cabf9ea --- /dev/null +++ b/doc/client/GUI-Eclipse.md @@ -0,0 +1,8 @@ +# Eclipse + +```{toctree} +:glob: +:titlesonly: + +eclipse/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Getting-Started.md b/doc/client/GUI-Getting-Started.md new file mode 100644 index 000000000..229a8a4a4 --- /dev/null +++ b/doc/client/GUI-Getting-Started.md @@ -0,0 +1,8 @@ +# Getting Started + +```{toctree} +:glob: +:titlesonly: + +getting_started/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Other.md b/doc/client/GUI-Other.md new file mode 100644 index 000000000..cebe82e14 --- /dev/null +++ b/doc/client/GUI-Other.md @@ -0,0 +1,8 @@ +# Miscellaneous + +```{toctree} +:glob: +:titlesonly: + +misc/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Testing.md b/doc/client/GUI-Testing.md new file mode 100644 index 000000000..daaf348f3 --- /dev/null +++ b/doc/client/GUI-Testing.md @@ -0,0 +1,8 @@ +# Testing + +```{toctree} +:glob: +:titlesonly: + +testing/* +``` \ No newline at end of file diff --git a/doc/client/GUI-Troubleshooting-remote-debugging.png b/doc/client/GUI-Troubleshooting-remote-debugging.png new file mode 100644 index 000000000..d02f12c72 Binary files /dev/null and b/doc/client/GUI-Troubleshooting-remote-debugging.png differ diff --git a/doc/client/GUI-Troubleshooting.md b/doc/client/GUI-Troubleshooting.md new file mode 100644 index 000000000..cde5d4f32 --- /dev/null +++ b/doc/client/GUI-Troubleshooting.md @@ -0,0 +1,123 @@ +# GUI Troubleshooting + +This page contains information on how to troubleshoot some common issues with the GUI. These are issues that occur once the GUI has started, not issues in starting the GUI. A good place to start are the log files, they are stored in `...\Instrument\Apps\Client\workspace\logs`. + +The following pages also contain GUI troubleshooting information for specific problems: + +```{toctree} +:glob: +:titlesonly: + +troubleshooting/* +``` + +## IBEX appears fullscreen with top menu items hidden + +Solution: right click inside any OPI and select "exit full screen". + +## Runtime failure due to security manager + +If you use eclipse IDE version 2022-03 to attempt to launch the GUI, you will get an error like the following: + +``` +org.osgi.framework.BundleException: Exception in org.eclipse.osgi.internal.framework.SystemBundleActivator.start() of bundle org.eclipse.osgi. +... +Caused by: org.osgi.framework.BundleException: Failed to create security manager +... +Caused by: java.lang.ClassNotFoundException: allow +``` + +The solution to this error is to use an older IDE - for example 2021-12 works without issue. Longer-term solution to this issue is to wait for the eclipse IDE team to fix this bug. + +## Build fails due to Tycho dependencies (e.g. equinox.ds not resolvable; this is only for the Maven build) + +Try deleting the `.m2` cache directory (in `ibex_gui/build/.m2` for GUI/script generator or `isis_css_top/.m2` for CSS) and then rebuilding. + +## ModuleNotFoundError: No module named 'pywin32_bootstrap' + +This may be due to windows file path limits. If you are running from a deeply nested directory after build and try to access the path to python it may run over this file path limit and not be able to find it. + +## IOC Start/Stop list is not Populated + +If the IOC Start/Stop list is blank when the instrument is running then there is a problem with the PV serving this. The PV serving it comes from the DBSRV ioc and ultimately comes from the MySQL database. Console to the BD server: + +`console -M localhost DBSVR` + +should not be producing errors (pressing return creates blank lines) + +Next check that the [database is up](/system_components/mysql/Database-Troubleshooting). + +## Target Platform Errors + +In some cases when you receive a `Resolving Target Definition` error, this can relate to some cached files that need to be refreshed and repeatedly performing the loop of `Clean project > Set as Target Platform > Reload` between 3-10 times can fix this issue. Set as Target Platform and Reload are located on the target view, located at `uk.ac.stfc.isis.ibex.targetplatform.target`. + +## JAXB dependency issues + +Firstly, as of the March 2020 dependency updates, make sure you have imported the `uk.ac.stfc.isis.ibex.jaxb` project into eclipse (check out master and pull first). + +We have seen some cases of JAXB causing issues with the GUI building. Since Eclipse builds in parallel a race condition can arise between building CSS (which expects some XML classes in the standard library) and the rest of the GUI (which has XML outside of the standard library due to using Java 11). If you have a load of JAXB issues, try the following: + +1. In the project explorer delete all the plugins (Do not delete the files from disk) +1. Restart Eclipse +1. Select "Import Existing Projects" and select the GUI base folder +1. Open the `uk.ac.stfc.isis.ibex.targetplatform` file +1. After the project has been imported click the "reload" button and then "Set Target Platform" in the top right. Note that these two buttons do not do the same thing and it necessary to press them both in order. + +This seems to be the best way to resolve this. If you have persistent issues with JAXB and this method not working, please inform the team as we would like to investigate. + +## Delete dotfiles + +If Eclipse gets very confused with the project build, you might want to completely wipe clean all files/folders used in the project. You can create a new workspace but there are still several additional folders worth deleting if needed. In your user folder, there should be hidden "dot" folders called: + +- `.m2` (contains downloaded dependencies) +- `.p2` +- `.eclipse` + +## Client fails to start at runtime + +If it produces a log with: +``` +Unresolved requirement: Require-Capability: osgi.ee; filter:="(&(osgi.ee=JavaSE)(version=11))" +``` + +You may have built with Java 11 but are running with Java 8 make sure the path is set correctly without Java 8 on it and that you are using the correct version at run time. + +You can verify that you are using the correct java version by running `java -version` in a command prompt. It should return java 11 in a non-EPICS terminal, or java 8 in an epics terminal (note, you cannot launch the GUI from an EPICS terminal). If it returns the incorrect version, you can add the correct version to your PATH. + +## File set from project ... has no valid check configuration. +Try [setting up the checkstyle](eclipse/Checkstyle-setup). + +## GUI claims another instance is running upon start (but there isn't) +Navigate to: `C:/Users//AppData/Local/IBEX` and delete the file "instance.txt" + +## maven-built client can't see referenced files +Add `Eclipse-BundleShape: dir` in the `MANIFEST.MF`; see the `.opis` `MANIFEST.MF` for an example of this. +This makes the maven build build the directory into a directory rather than a jar, which is sometimes necessary to let eclipse then "see" the files natively on the filesystem. If you look in the `plugins/` folder of a built client you should see a folder for `.opis` but a `.jar` for most other ibex modules - except for a few that need to be built as directories, as you may be finding. + + + +## Other issues +### Remote debugging with Eclipse +The following has been done to ensure that it is possible to debug the IBEX GUI running on any particular instrument: +1. The JDWP agent is added to IBEX GUI. In individual installations it will run on an available port dynamically determined at run-time. +2. There is a utility available in `\\isis\shares\ISIS_Experiment_Controls_Public\ibex_utils\ibex_gui_utilities - getJVMInfo.bat` +3. SSH server is installed on all instruments and port forwarding is enabled. + +The following needs to be done to debug the instance: +1. Open Command prompt in the Instrument. +2. Get the port on which JDWP is enabled for the instrument by executing the following on the command prompt: +`\\isis\shares\ISIS_Experiment_Controls_Public\ibex_utils\ibex_gui_utilities\getJVMInfo.bat`. +From the output copy the value against dt_socket. +3. In your local machine open command prompt and enter `ssh –L :127.0.0.1: `. +For example `ssh –L 12345:127.0.0.1:66666 spudulike@ndxscidemo` +4. Set the debug configuration (Run ==> Debug Configurations... ==> Remote Java Application) on the eclipse as shown below. Use 127.0.0.1 as host name and as the port: +![image](GUI-Troubleshooting-remote-debugging.png) + + +### Debugging local code with Eclipse +1. When eclipse build starts it will print on the console the port it is listening for debugging. Similar to : `Listening for transport dt_socket at address: 60796` +2. Set a debug configuration for Remote Java Application with host as localhost and port as obtained above. + +[Memory "leaks"](troubleshooting/Debugging-memory-leaks-in-the-IBEX-GUI) + +[PV Manager and Observers](troubleshooting/PV-Manager-and-Observers-Logging) \ No newline at end of file diff --git a/doc/client/coding/Adding-a-Button-to-the-Perspective-Switcher.md b/doc/client/coding/Adding-a-Button-to-the-Perspective-Switcher.md new file mode 100644 index 000000000..1a4d4378a --- /dev/null +++ b/doc/client/coding/Adding-a-Button-to-the-Perspective-Switcher.md @@ -0,0 +1,352 @@ +# Adding a perspective + +![Perspective Switcher](IBEX_complete_perspective_switcher_highlighted.png) + + +### Creating a plug-in + +* In Eclipse, add a new UI plug-in via File->New->Other->Plug-in Development->Plug-in Project + +* Give it a name, for example: uk.ac.stfc.isis.ibex.ui.myperspective + +* Do not use the default location, instead set it to C:\\Instrument\\Dev\\client\\ISIS\\base\\ plus the plug-in name. + * for example: C:\\Instrument\\Dev\\ibex_gui\\base\\uk.ac.stfc.isis.ibex.ui.myperspective + +* The dialog should look like this: + +![Add Perspective](eclipse_add_perspective_plugin1.png) + +* Click 'Next' + +* On the next page of the dialog check it looks like the following and click 'Finish': + +![Add Perspective](eclipse_add_perspective_plugin2.png) + +Note: Generate an activator is unchecked as generally we do not need one for UI plug-ins, but it does not really matter if there is one. + +### Creating a Perspective + +Now the plug-in has been created: + +* Add a source package via File->New->Other->Java->Package + +* Set the source folder to the path of the source folder in the newly created plug-in + +* Set the name to something sensible prefixed by the plug-in name (or just the plug-in itself) + +* The dialog should look something like this: + +![Add a package](eclipse_adding_a_package.png) + +Note: Package names should be in the format of uk.ac.stfc.isis.ibex.ui.packagename, not org.csstudio.isis.ui.packagename as shown in all images. + +We now need to add a Perspective class to the new package; the easiest way to do this is to copy and paste an existing one, for example: the one in uk.ac.stfc.isis.ibex.ui.scripting, and edit it. + +The first thing you will notice is that there are numerous red errors. + +![Eclipse errors](eclipse_perspective_copy_errors.png) + +These are easily fixed: + +* Hover the mouse over `BasePerspective` on the `public class Perspective` line and select `Fix project setup...` + +* Delete the `Consoles.getDefault().createConsole();` line + +* Open the MANIFEST.MF file in META_INF and select the `Dependencies` tab; on that tab click the `Add` button under `Required Plug-ins`. From the list select uk.ac.stfc.isis.ibex.ui and click `OK`. Add org.eclipse.ui and org.eclipse.core.runtime with the same method. + +* There are likely to be other dependencies missing, fix any other errors. + +* Save all the changes. + +The errors should now have disappeared, but there are a few more things to do: + +* Change the value returned by the name function to be what you want shown on the Perspective Switcher. Put an ampersand before the character that you want to use as the perspective shortcut making sure that the chosen character is not in use already. (More on this later) + +* Add a new icon (we will leave that as it is for now and fix it later!) + +### Creating a View + +With the Perspective now in place we need to add a View: + +* Add a new class via File->New->Other->Java->Class + +* Make sure the Package is the same as one created earlier + +* Enter a sensible name + +* Click the 'Browse' button next to the Superclass and select ViewPart + +The dialog should look something like this: + +![Add a View](eclipse_adding_a_View.png) + +The new class file should open in the editor. For this example I am just going to add a web browser to the view: + +* Right-click on the View in the Package Explorer in Eclipse and select Open With->WindowBuilder Editor. + +* In the window that pops up select the Design tab + +* Select the FillLayout and click on the mock-up of the View to add it + +* Select the Browser control and click on the mock-up to add it + +* Now click on the Source tab + +* If the `createPartControl` method has a Composite argument called arg0 change it to parent + +* Delete the TODO comments + +* Add an ID for the View which is the full-name of the class in lower-case; for example: uk.ac.stfc.isis.ibex.ui.myperspective.myview + +* It should now look something like: + +```java + package uk.ac.stfc.isis.ibex.ui.myperspective; + + import org.eclipse.swt.widgets.Composite; + import org.eclipse.ui.part.ViewPart; + import org.eclipse.swt.layout.FillLayout; + import org.eclipse.swt.SWT; + import org.eclipse.swt.browser.Browser; + + public class MyView extends ViewPart { + public static final String ID = "uk.ac.stfc.isis.ibex.ui.myperspective.myview"; + + public MyView() { + } + + @Override + public void createPartControl(Composite parent) { + parent.setLayout(new FillLayout(SWT.HORIZONTAL)); + + Browser browser = new Browser(parent, SWT.NONE); + } + + @Override + public void setFocus() { + } + } +``` + +### Adding the Perspective and View to the GUI + +Note: The extensions are defined in XML inside plugin.xml, so example XML is included below if you would prefer to edit that directly. + +Note: Sometimes Eclipse cannot find the schema for the extensions, so when trying to select, say, New->contribution contribution does not appear in the list rather it says Generic; in this case, it is necessary to edit the XML directly. + +To add both the new Perspective and View to the main GUI we use extensions. First let's add the Perspective: + +* Open the MANIFEST.MF file in META_INF and select the 'Extensions' tab, it should be empty like this: + +![Extensions](eclipse_no_extensions.png) + +* Click the 'Add' button and select uk.ac.stfc.isis.ibex.ui.perspectives extension point and click 'Finish' + +![Extension points](eclipse_select_extension_point.png) + +* It should now look like this: + +![Extension points](eclipse_extensions_added1.png) + +* Right-click on the uk.ac.stfc.isis.ibex.ui.perspectives extension point and select New->contribution + +* The contribution should appear below the extension point + +* Using the 'Browse' button select the Perspective class created earlier, the screen should now look like like this: + +![Extension points](eclipse_extensions_added2.png) + +* Using the 'Add' button as before we need to add org.eclipse.ui.perspectives extension point + +* Right-click on the new extension point and select New->perspective; the new perspective should appear below the extension point + +* Select the newly added item and fill in the required details, it should look something like this: + +![Extension points](eclipse_extensions_added3.png) + +The XML in plugin.xml for what we have done so far is: +```xml + + + + + + + + +``` + +Now we add the extensions for the View: + +* Using the 'Add' button as before we need to add org.eclipse.ui.perspectiveExtensions extension point + +* Right-click on the new extension point and select New->perspectiveExtension; the new perspectiveExtension should appear below the extension point + +* For the new perspectiveExtension set the targetID to the ID of your perspective, for this example it is uk.ac.stfc.isis.ibex.ui.myperspective.perspective + +* Right-click on the perspectiveExtension and select New->view; the new view should appear + +* Select the view and change the ID to the ID of your View, for example: uk.ac.stfc.isis.ibex.ui.myperspective.myview + +* Change the relative to uk.ac.stfc.isis.ibex.ui.perspectives.PerspectiveSwitcher + +* The remaining setting determine how the View will appear and behave, it is recommended that you set the following values: + + * closeable: false + * minimized: false + * moveable: false + * showTitle: false + * standalone: true + * visible: true + * relationship: right + +* It should look something like this: + +![Extension points](eclipse_extensions_added4.png) + +* Next, using the 'Add' button we need to add org.eclipse.ui.views extension point + +* Right-click on the new item and select New->view; the new view should appear below the extension point + +* Set the class and id to the name and id of your View class respectively; it should look something like this: + +![Extension points](eclipse_extensions_added5.png) + +The XML in the plugin.xml for the View related stuff is: +```xml + + + + + + + + + + +``` + +Finally, the last step is to add the plug-in we created to uk.ac.stfc.isis.ibex.feature.base: + +* Open the feature.xml file in uk.ac.stfc.isis.ibex.feature.base and select the Plug-ins tab + +* Click the 'Add' button and select the new plug-in we created + +* Save everything and run the main GUI - hopefully, the new Perspective will appear + + +### Adding a new icon + +* Grab a nice png icon which is appropriately sized (`24x24` pixels) from somewhere like http://www.flaticon.com/ + +* Create a icons folder in the top-level of the plug-in + +* Drag the icon into it + +* Open the MANIFEST.MF file and select the Build tab + +* Tick the icons box under Binary Build to include the icons folder in the build + +![Add an icon](eclipse_add_icons_to_build.png) + +* Open the Perspective class created earlier + +* Change the image method to return the new icon from the correct plug-in by changing the plug-in name and icon name, like so: +```java + package uk.ac.stfc.isis.ibex.ui.myperspective; + + import uk.ac.stfc.isis.ibex.ui.perspectives.BasePerspective; + import org.eclipse.swt.graphics.Image; + import org.eclipse.ui.IPageLayout; + import org.eclipse.wb.swt.ResourceManager; + + public class Perspective extends BasePerspective { + + public static final String ID = "uk.ac.stfc.isis.ibex.ui.myperspective.perspective"; //$NON-NLS-1$ + + @Override + public void createInitialLayout(IPageLayout layout) { + super.createInitialLayout(layout); + } + + @Override + public String ID() { + return ID; + } + + @Override + public String name() { + return "My Perspective"; + } + + @Override + public Image image() { + return ResourceManager.getPluginImage("uk.ac.stfc.isis.ibex.myperspective", "icons/myperspective.png"); + } + } +``` + +* Finally, start the GUI to check the new icon is shown + + +### Adding a keyboard shortcut + +* In the plugin xml for your new perspective add the following: + +```xml + + + + + + +``` + +* Run the GUI and hit ALT + SHIFT + _chosen char_ and confirm that the perspective is switched to. + +### Setting the Perspective to be invisible + +If you are creating the perspective for testing a new feature that you do not want displayed to the user by default then add the following code to your perspective class: +``` + @Override + public boolean isVisibleDefault() { + return false; + } +``` +by default this value is set to true and so perspectives are displayed. + +To subsequently display the perspective run IBEX and go to the perspective window (CTRL + ALT + P) then enable the checkbox in ISIS Perspectives. On the next restart of the GUI your perspective should be displayed. + +### Troubleshooting + +If the perspective is not being shown in the switcher at the side it may be Eclipse being silly or you may not be running the right product. Be sure to re-run by going selecting the client product, rather than using the drop-down (which will run the same product as you used previously) Finally, try clearing the workspace and resetting the target platform etc. \ No newline at end of file diff --git a/doc/client/coding/Adding-a-Plugin-or-Feature-to-Maven-Build.md b/doc/client/coding/Adding-a-Plugin-or-Feature-to-Maven-Build.md new file mode 100644 index 000000000..b3ea650ab --- /dev/null +++ b/doc/client/coding/Adding-a-Plugin-or-Feature-to-Maven-Build.md @@ -0,0 +1,56 @@ +# Adding a plugin or feature to Maven + +The steps for adding a plug-in (one small part of IBEX, such as the blocks view) or feature (a larger collection of plug-ins, such as CSS) to the maven build are below: + +## Step by step: +* Add the plug-in to `feature.base`: + * Open `feature.xml` in `uk.ac.stfc.isis.ibex.feature.base` + * Go to "Included Plug-ins" (or "Included Features") tab and click "Add..." + * Find your new plug-in in the list and add it + +* Add the plug-in to `ibex.product` + * Open `ibex.product` in `uk.ac.stfc.isis.ibex.e4.client.product` + * Go to "Configuration" tab and click "Add..." next the "Start Levels" section + * Find your new plug-in in the list and add it + +* Add the plug-in to `feature.base`: + * Open `feature.xml` in `uk.ac.stfc.isis.ibex.feature.base` + * Go to "Included Plug-ins" tab and click "Add..." + * Find your new plug-in in the list and add it + +* Convert the plug-in to a Maven project. + * Right-click on the plug-in and select Configure > Convert to Maven Project + * Click "Finish". This should create a `pom.xml` inside the project. + +* Add the new plug-in to the Parent POM + * Select the `pom.xml` file in `uk.ac.stfc.isis.ibex.client.tycho.parent` + * On the overview tab click "Add..." under the Modules section + * Select the new plug-in from the list + * Enable the "Update POM parent section in selected projects" option and click "OK" + * Save it + +* Edit the plug-in pom.xml file + * Select the pom.xml file + * Open the pom.xml tab + * Change/add the packaging to `eclipse-plugin` (or `eclipse-test-plugin` if it's a unit test plugin) + * Remove the build section + * Remove the `groupID` and version entries outside of parent + * Save it + +## Example POM + +An example of what the plug-in POM should look like: + +``` + + 4.0.0 + tychodemo.bundle.tests + eclipse-plugin + + CSS_ISIS + parent + 1.0.0-SNAPSHOT + ../tychodemo.parent + + +``` diff --git a/doc/client/coding/Connecting-a-View-to-a-PV.md b/doc/client/coding/Connecting-a-View-to-a-PV.md new file mode 100644 index 000000000..c1ae8abbf --- /dev/null +++ b/doc/client/coding/Connecting-a-View-to-a-PV.md @@ -0,0 +1,643 @@ +# Connecting a view to a PV + +See [An Introduction to Databinding](Databinding) for a explanation of databinding. + +The basic arrangement for the mechanism for connecting a View to a PV is shown below: + +![Basic principle](basic_principle.png) + +## Adding an Observable + +The Observable is responsible for the connection to the PV and monitoring it for changes. + +For this example we are going to connect and read the title PV on the DAE IOC. There is already a way of getting the title in the IBEX GUI, but for demonstration purposes we are going to ignore that! + +First we need to create a new Plug-in in Eclipse: + +* Select File->New->Plug-in Project + +* Enter org.csstudio.isis.title for the project name + +* Un-check 'Use default location' and browse to where the GUI code is located and append the project name to the path. For example, on my system the location would be C:\\Instrument\\Dev\\client\\ISIS\\base\\org.csstudio.isis.title + +* Click 'Next' + +* On the next screen, check the 'Generate an activator ...' check-box and un-check the others. + +* Set 'Would you like to create a 3.x rich client application?' to 'No' + +* Click 'Finish' + +Now the Plug-in is created we can create an Observable: + +* Open the MANIFEST.MF file in META-INF and under the 'Dependencies' tab add org.csstudio.isis.instrument to the list of required plug-ins + +* Don't forget to save the changes + +* Create a new Class called TitleVariable, this will be the class that holds the connection to the PV + +* First we will need to create an observable factory to get our observable, in this case the PV should be updated when the instrument switches so add the following as a field. +```java +private final ObservableFactory obsFactory = new ObservableFactory(OnInstrumentSwitch.SWITCH); +``` + +* Next we get the specific variable for the PV the observable for the PV: +```java +package org.csstudio.isis.title; + +import uk.ac.stfc.isis.ibex.epics.observing.ForwardingObservable; +import uk.ac.stfc.isis.ibex.epics.switching.ObservableFactory; +import uk.ac.stfc.isis.ibex.epics.switching.OnInstrumentSwitch; +import org.csstudio.isis.instrument.Channels; +import org.csstudio.isis.instrument.channels.CharWaveformChannel; +import uk.ac.stfc.isis.ibex.instrument.InstrumentUtils; + +public class TitleVariable { + + private final ObservableFactory obsFactory = new ObservableFactory(OnInstrumentSwitch.SWITCH); + public final ForwardingObservable titleRBV = + obsFactory.getSwitchableObservable(new CharWaveformChannel(), InstrumentUtils.addPrefix("DAE:TITLE")); + + public TitleVariable() { + } + +} +``` +Explanation on Channels, `InitialiseOnSubscribeObservable` and reader. + +## Adding a Model interface + +Next we create a Model interface: + +* Create a new Interface called TitleModel`` + +* Implementing the interface will require `getTitle` and `setTitle` methods, so the code needs to look like: +``` +java +package org.csstudio.isis.title; + +public interface TitleModel { + String getTitle(); + void setTitle(String value); +} +``` + +## Adding an ObservableModel + +Next we create an ObservableModel which allows us to bind controls on the View to the PV Observable: + +* Create a new Class called ObservableTitleModel + +* ObservableTitleModel needs to inherit from ModelObject from org.csstudio.isis.model, so change the code to look like: +```java +package org.csstudio.isis.title; + +public class ObservableTitleModel extends ModelObject { + +} +``` +* To fix the errors we need to add org.csstudio.isis.model to the required plug-ins list in the MANIFEST.INF. We can either add it manually or hover over the error and select "Add 'org.csstudio.isis.model' to required bundles" + +* The next error can be fixed via the drop-down by selecting "Import 'ModelObject' (org.csstudio.isis.model)" + +* The class also needs to implement TitleModel, so change the code to implement TitleModel: +```java +public class ObservableTitleModel extends ModelObject implements TitleModel { +``` +* There should now be an error because the methods of TitleModel are not implemented. Hover over the error and select 'Add unimplemented methods'. The code should look like this: +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.epics.observing.BaseObserver; +import org.csstudio.isis.model.ModelObject; + +public class ObservableTitleModel extends ModelObject implements TitleModel { + + private String title; + + @Override + public String getTitle() { + // TODO Auto-generated method stub + return null; + } + + @Override + public void setTitle(String value) { + // TODO Auto-generated method stub + + } + +} +``` +* The correctly cased getter and setter are requirements for the object to be bindable from a View (http://en.wikipedia.org/wiki/JavaBeans). We now need to add our own code to the methods: +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.model.ModelObject; + + +public class ObservableTitleModel extends ModelObject{ + private String title; + + public String getTitle() { + return title; + } + + public void setTitle(String value) { + firePropertyChange("title", this.title, this.title = value); + } +} +``` +The `firePropertyChange` method raises an event when the title changes. + +* Next we create a BaseObserver inside the class, like so: +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.epics.observing.BaseObserver; +import org.csstudio.isis.model.ModelObject; + +public class ObservableTitleModel extends ModelObject implements TitleModel { + + private String title; + + private final BaseObserver titleObserver = new BaseObserver(){ + + @Override + public void onValue(String value) { + setTitle(value); + } + + @Override + public void onError(Exception e) { + + } + + @Override + public void onConnectionChanged(boolean isConnected) { + + } + }; + + @Override + public String getTitle() { + return title; + } + + @Override + public void setTitle(String value) { + firePropertyChange("title", this.title, this.title = value); + } +} +``` +* The BaseObserver is responsible for observing changes in the TitleVariable, but for that we need to wire the objects up. Add a private variable for TitleVariable, like so: +```java +... +public class ObservableTitleModel extends ModelObject implements TitleModel { + + private String title; + private final TitleVariable titleVar; + + private final BaseObserver titleObserver = new BaseObserver(){ +... +``` +* Now we add a constructor for that allows the wiring up of the BaseObserver and the TitleVariable: +```java +public ObservableTitleModel(TitleVariable titleVar) { + this.titleVar = titleVar; + titleVar.titleRBV.subscribe(titleObserver); +} +``` +* The final class looks like this: +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.epics.observing.BaseObserver; +import org.csstudio.isis.model.ModelObject; + +public class ObservableTitleModel extends ModelObject implements TitleModel { + + private String title; + private final TitleVariable titleVar; + + private final BaseObserver titleObserver = new BaseObserver() { + + @Override + public void onValue(String value) { + setTitle(value); + } + + @Override + public void onError(Exception e) { + + } + + @Override + public void onConnectionChanged(boolean isConnected) { + + } + }; + + @Override + public String getTitle() { + return title; + } + + @Override + public void setTitle(String value) { + firePropertyChange("title", this.title, this.title = value); + } + + public ObservableTitleModel(TitleVariable titleVar) { + this.titleVar = titleVar; + titleVar.titleRBV.subscribe(titleObserver); + } +} +``` +## The Activator + +The Activator is the entry-point for the Plug-in - it gets created when the Plug-in is first used. +This is where we wire up the ObservableTitleModel and the TitleVariable: + +* First we want to rename Activator because it is not a very descriptive name, let's refactor it to be called Title (okay that is not much better...) + +* We want to make the Activator a singleton, so we need it to contain a static instance of itself, a constructor and a `getInstance` method: +```java +package org.csstudio.isis.title; + +import org.osgi.framework.BundleActivator; +import org.osgi.framework.BundleContext; + +public class Title implements BundleActivator { + + private static BundleContext context; + private static Title instance; + + public Title() { + instance = this; + } + + public static Title getInstance() { + return instance; + } + + static BundleContext getContext() { + return context; + } + + /* + * (non-Javadoc) + * @see org.osgi.framework.BundleActivator#start(org.osgi.framework.BundleContext) + */ + public void start(BundleContext bundleContext) throws Exception { + Title.context = bundleContext; + } + + /* + * (non-Javadoc) + * @see org.osgi.framework.BundleActivator#stop(org.osgi.framework.BundleContext) + */ + public void stop(BundleContext bundleContext) throws Exception { + Title.context = null; + } +} +``` +* Now we need it to wire up our classes from earlier, so we add the wiring up to the constructor: +```java +... +private static BundleContext context; +private static Title instance; + +private TitleVariable titleVar; +private ObservableTitleModel model; + +public Title() { + instance = this; + titleVar = new TitleVariable(Instrument.getInstance().channels()); + model = new ObservableTitleModel(titleVar); +} +... +``` +* Finally we add a method to retrieve the model so we can bind against it. The final code look like this: +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.instrument.Instrument; +import org.osgi.framework.BundleActivator; +import org.osgi.framework.BundleContext; + +public class Title implements BundleActivator { + + private static BundleContext context; + private static Title instance; + + private TitleVariable titleVar; + private ObservableTitleModel model; + + public Title() { + instance = this; + titleVar = new TitleVariable(Instrument.getInstance().channels()); + model = new ObservableTitleModel(titleVar); + } + + public static Title getInstance() { + return instance; + } + + public ObservableTitleModel model() { + return model; + } + + static BundleContext getContext() { + return context; + } + + /* + * (non-Javadoc) + * @see org.osgi.framework.BundleActivator#start(org.osgi.framework.BundleContext) + */ + public void start(BundleContext bundleContext) throws Exception { + Title.context = bundleContext; + } + + /* + * (non-Javadoc) + * @see org.osgi.framework.BundleActivator#stop(org.osgi.framework.BundleContext) + */ + public void stop(BundleContext bundleContext) throws Exception { + Title.context = null; + } +} +``` +## A little tidying up + +By convention we should put any class etc. we don't want exposed outside outside of the Plug-in in a package ending in internal. In this example the package would be called org.csstudio.isis.title.internal. + +## Binding the View to the Model + +The simplest way to do this is to add a method for creating the binding to the View: +```java +private DataBindingContext bindingContext; + +private void doBinding(Label lblTitle){ + bindingContext = new DataBindingContext(); + bindingContext.bindValue(WidgetProperties.text().observe(lblTitle), BeanProperties.value("title").observe(Title.getInstance().model())); +} +``` +* Then the method can be called from the `createPartControl` method of the View + +## Writing to a PV + +For this example we will start by adding a writeable PV for writing to the title to the TitleVariable class, so it now looks like this: +```java +package org.csstudio.isis.title; + +import uk.ac.stfc.isis.ibex.epics.observing.ForwardingObservable; +import org.csstudio.isis.epics.writing.Writable; +import org.csstudio.isis.instrument.Channels; +import org.csstudio.isis.instrument.channels.CharWaveformChannel; +import uk.ac.stfc.isis.ibex.instrument.InstrumentUtils; + +public class TitleVariable { + + private final ObservableFactory obsFactory = new ObservableFactory(OnInstrumentSwitch.SWITCH); + public final ForwardingObservable titleRBV = + obsFactory.getSwitchableObservable(new CharWaveformChannel(), InstrumentUtils.addPrefix("DAE:TITLE")); + + private final WritableFactory writeFactory = new WritableFactory(OnInstrumentSwitch.SWITCH); + public final Writable titleSP = + writeFactory.getSwitchableWritable(new CharWaveformChannel(), InstrumentUtils.addPrefix("DAE:TITLE:SP")); + + public TitleVariable() { + } +} +``` +Note that the type of the new PV is Writable and uses writable and not reader. + +Next we open the TitleModel interface and add two new methods for working with the set-point, it now looks like this: +```java +package org.csstudio.isis.title; + + +public interface TitleModel { + String getTitle(); + void setTitle(String value); + String getTitleSP(); + void setTitleSP(String value); +} +``` +This interface is implemented in the ObservableTitleModel class, so we need to add implementations for the two new methods. +In fact, the ObservableTitleModel should now be showing an error because these methods are not implemented. +Open the class and hover over the class name, it should give you the option to 'Add unimplemented methods', select this. +Now we need to add code for these new methods: +```java +... +@Override +public String getTitleSP() { + return ""; +} + +@Override +public void setTitleSP(String value) { + titleVar.titleSP.write(value); +} +... +``` +The getter returns an empty string, but this does not matter - we could wire it up to get the current value of the set-point using a BaseObserver in the same way we did for reading the current title earlier, but we will leave that as an exercise for the reader! + +The setter uses the Writable object to write the new value to the IOC via channel access. + +Finally we need to edit the View itself to enable it to bind to titleSP. For this we add a text-box called `txtNewTitle` and set up the binding: +```java +... +bindingContext.bindValue(WidgetProperties.text(SWT.Modify).observe(txtNewTitle), BeanProperties.value("titleSP").observe(org.csstudio.isis.title.Title.getInstance().model())); +... +``` +The key thing to note here is that WidgetProperties.txt takes SWT.Modify; this tells the binding to push the changes back to the model when the text-box is changed. + +If we start the GUI it can be seen that as we type in the text-box the title changes; however, it is not ideal as the title set-point is updated with every keystroke which seems a bit unnecessary. + +The alternative is to have a set button. First add a string property for the title SP called titleSP like so: +```java +... +public class ObservableTitleModel extends ModelObject implements TitleModel { + +private String title; +private String titleSP; +private final TitleVariable titleVar; + +private final BaseObserver titleObserver = new BaseObserver() { +... +``` +Now we modify the getter and setter to use this variable for storing the new title before it is sent to the IOC: +```java +... +@Override +public String getTitleSP() { + return titleSP; +} + +@Override +public void setTitleSP(String value) { + firePropertyChange("titleSP", this.titleSP, this.titleSP = value); +} +... +``` +We also add a method to the TitleModel interface and the implementation to the ObservableTitleModle for sending the string to the IOC: +```java +... +@Override +public void sendTitleSP() +{ + titleVar.titleSP.write(titleSP); +} +... +``` +The final step is to add a button to the View's `createPartControl` method: +```java +... +Button btnSet = new Button(parent, SWT.NONE); +btnSet.setText("Set"); +btnSet.addSelectionListener(new SelectionAdapter() { + @Override + public void widgetSelected(SelectionEvent e){ + org.csstudio.isis.title.Title.getInstance().model().sendTitleSP(); + } +}); +... +``` + +## Using a Read-Write control + +It is fine to have separate controls on the View for reading and writing, but sometimes it is more convenient to have one control for both. + +This is relatively straightforward as there is already a helper class called WritableObservableAdapter that does most of the work. + +The first step though is to create a View Model class in our UI plug-in, so create a new class in org.csstudio.isis.ui.title called ViewModel. +This class is where we will connect up the WritableObservableAdapter, but first we need to make some changes to ObservableTitleModel +and to do that we start by modifying TitleModel to add a methods for accessing the Writeable object and a CachingObservable object (explained later): +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.epics.observing.CachingObservable; +import org.csstudio.isis.epics.writing.Writable; + + +public interface TitleModel { + String getTitle(); + void setTitle(String value); + String getTitleSP(); + void setTitleSP(String value); + void sendTitleSP(); + CachingObservable title(); + Writable titleSetter(); +} +``` +Now we implement these methods in ObservableTitleModel: +```java +... +@Override +public CachingObservable title() { + return titleVar.titleRBV; +} + +@Override +public Writable titleSetter() { + return titleVar.titleSP; +} +... +``` +Next we add some code to ViewModel to connect things up: +```java +package org.csstudio.isis.ui.myperspective; + + +import org.csstudio.isis.title.ObservableTitleModel; +import org.csstudio.isis.ui.widgets.observable.WritableObservableAdapter; + +public class ViewModel { + + public final ObservableTitleModel model = org.csstudio.isis.title.Title.getInstance().model(); + + public final WritableObservableAdapter title = new WritableObservableAdapter(model.titleSetter(), model.title()); + + public ViewModel() { + + } +} +``` +Some error messages will appear relating to WritableObservableAdapter; to fix, hover over WritableObervableAdaptor and select "Add 'org.csstudio.isis.ui.widgets' to required bundles". +Errors will still remain on model.titleSetter() and model.title(), so hover over one of those and select "Add 'org.csstudio.isis.epics' to required bundles". + +Now we need to adjust the View to use the ViewModel, first we strip out all of the code relating to the previous label, text-box and button (including the bindings), and then we add a ViewModel object and a WritableObservingTextBox: +```java +package org.csstudio.isis.ui.myperspective; + +import org.csstudio.isis.ui.widgets.observable.WritableObservingTextBox; +import org.eclipse.swt.SWT; +import org.eclipse.swt.widgets.Composite; +import org.eclipse.ui.part.ViewPart; +import org.eclipse.swt.layout.GridData; +import org.eclipse.swt.layout.GridLayout; + +public class MyView extends ViewPart { + public static final String ID = "org.csstudio.isis.ui.myperspective.myview"; + + private final ViewModel viewModel = new ViewModel(); + private WritableObservingTextBox rbNumber; + + public MyView() { + } + + @Override + public void createPartControl(Composite parent) { + parent.setLayout(new GridLayout(1, false)); + rbNumber = new WritableObservingTextBox(parent, SWT.NONE, viewModel.title); + rbNumber.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, true, false, 1, 1)); + + } + + @Override + public void setFocus() { + } + +} +``` +Hopefully once the GUI is started that all works as expected. + +So what is going on and what are these new bits we are using? + +* WritableObservingTextBox is a custom ISIS control for displaying and editing a value where they are different PVs for reading and writing +* WritableObservableAdapter is the object for linking up the two PVs. Essentially it does the same as what we did earlier with reading one PV and writing to another +* View Model - a class for providing presentation logic and state information for a View. The code could live in the View itself, but it is cleaner to put it in a separate class. + +http://blogs.msdn.com/b/dphill/archive/2009/01/31/the-viewmodel-pattern.aspx provides a good explanation of View Models and how they fit in with the View and Model. + +As you have probably noticed there are a number of methods in the ObservableTitlemodel that is no longer used, these can be removed (don't forgot to remove them from the interface as well!). The BaseObserver can also be removed. +```java +package org.csstudio.isis.title; + +import org.csstudio.isis.epics.observing.CachingObservable; +import org.csstudio.isis.epics.writing.Writable; +import org.csstudio.isis.model.ModelObject; + +public class ObservableTitleModel extends ModelObject implements TitleModel { + + private final TitleVariable titleVar; + + public ObservableTitleModel(TitleVariable titleVar) { + this.titleVar = titleVar; + //titleVar.titleRBV.subscribe(titleObserver); + } + + @Override + public CachingObservable title() { + return titleVar.titleRBV; + } + + @Override + public Writable titleSetter() { + return titleVar.titleSP; + } +} +``` \ No newline at end of file diff --git a/doc/client/coding/Databinding---common-mistakes.md b/doc/client/coding/Databinding---common-mistakes.md new file mode 100644 index 000000000..89f022e7b --- /dev/null +++ b/doc/client/coding/Databinding---common-mistakes.md @@ -0,0 +1,40 @@ +# Databinding - common mistakes + +Start with [An introduction to databinding](Databinding) if you are new to this. + +### Missing getter or setter (or incorrectly named) + +The getter and setters (if the value can be set from the UI) must exist and be named correctly. For example: `getName` and `setName` for `BeanProperties.value("name")`. + +Behind the scenes the databinding library automatically changes the first character to upper-case and then prepends "get" and "set". + +### Incorrectly cased string in binding + +With something like the following it is import that the name of the property to bind to is cased correctly: +```java +ctx.bindValue(WidgetProperties.text(SWT.Modify) + .observe(txtAge), BeanProperties.value("age").observe(person)); +``` + +This assumes there is a getter and setter called `getAge` and `setAge`. + +For something like `getFedId` the binding code would look like: + +```java +ctx.bindValue(WidgetProperties.text(SWT.Modify) + .observe(txtId), BeanProperties.value("fedId").observe(person)); +``` +**The important point to note is the 'f' of "fedId" is lower-case. It will not work if it is upper-case.** + +### The getter or setter "silently" throws an exception + +If any code in the getter throws an unhandled exception then the binding won't work because the value cannot be read. +If a setter throws an unhandled exception before the firing the property change then the listeners will not receive the change signal. Both result in the binding being broken. + +If a binding seems to work intermittently then there might be something in the getter or setter causing this, e.g. an object used in a getter that switches between being null and initialised based on something else. + +**The exceptions will appear in the console inside Eclipse and IBEX but won't cause an error pop-up to appear.** + +### The binding works but the initial value is not put in the widget + +When the binding first happens it will call the getter to set the widget properties to whatever is in the model. If this doesn't happen the getter is probably non-existent or misspent. \ No newline at end of file diff --git a/doc/client/coding/Databinding.md b/doc/client/coding/Databinding.md new file mode 100644 index 000000000..515a8500c --- /dev/null +++ b/doc/client/coding/Databinding.md @@ -0,0 +1,387 @@ +# Databinding + +Note: there are some helpful tips to common errors/mistakes [here](Databinding---common-mistakes). + +Note: an explanation of how to connect a view to a PV is [here](Connecting-a-View-to-a-PV) + +## Create the example plug-ins ## + +First create an Eclipse plug-in for the back-end: + +* Start the plug-in wizard via File->New->Plug-in Project + +* On the first page of the wizard enter `org.databinding.example.backend` as the name + +* On the second page: + + * Deselect "Generate an activator, a Java class that controls the plug-in's life cycle" + * Deselect "This plug-in will make contributions to the UI" + * Select "No" for "Would you like to create a 3.x rich client application?" + * Click "Finish" + +Next create a plug-in for the UI side: + +* Start the plug-in wizard via File->New->Plug-in Project + +* On the first page of the wizard enter `org.databinding.example.frontend` as the name + +* On the second page: + + * Select "Generate an activator, a Java class that controls the plug-in's life cycle" + * Select "This plug-in will make contributions to the UI" + * Select "Yes" for "Would you like to create a 3.x rich client application?" + * Click "Next" + * Select "RCP application with a view" + * Click "Finish" + +## Creating the Person class (the model) + +In the backend plug-in create two new classes, one called `ModelObject` and one called Person. +Enter the following code for the `ModelObject`: +```java +package org.databinding.example.backend; + +import java.beans.PropertyChangeListener; +import java.beans.PropertyChangeSupport; + +public class ModelObject { + private final PropertyChangeSupport changeSupport = new PropertyChangeSupport(this); + + public void addPropertyChangeListener(PropertyChangeListener listener) { + changeSupport.addPropertyChangeListener(listener); + } + + public void removePropertyChangeListener(PropertyChangeListener listener) { + changeSupport.removePropertyChangeListener(listener); + } + + protected void firePropertyChange(String propertyName, Object oldValue, + Object newValue) { + changeSupport.firePropertyChange(propertyName, oldValue, newValue); + } +} +``` +The ModelObject contains the code that is essential for data-binding. + +Behind the scenes when you create a binding the data-binding framework automatically calls `addPropertyChangeListener` for the UI widget being bound +(not strictly true - it is a bean object that is associated with the widget). The `removePropertyChangeListener` is called when the widget binding is disposed, e.g. when the GUI is closed. + +The `firePropertyChange` method is used to inform the listeners when the value has changed. +Note: if `oldValue` and `newValue` are the same then the event is not raised. + +The Person class is the class that is going to be used as the model the UI binds to. Add the following code: +```java +package org.databinding.example.backend; + +public class Person extends ModelObject { + private String name; + private int age; + private String idNumber; + + public String getName() { + return name; + } + + public void setName(String name) { + firePropertyChange("name", this.name, this.name = name); + } + + public int getAge() { + return age; + } + + public void setAge(int age) { + firePropertyChange("age", this.age, this.age = age); + } + + public String getIdNumber() { + return idNumber; + } + + public void setIdNumber(String id) { + firePropertyChange("idNumber", this.idNumber, this.idNumber = id); + } + + public Person(String name, int age) { + setName(name); + setAge(age); + setIdNumber("0000"); + } +} +``` +Whenever a setter is called the `firePropertyChange` is called which informs all the listeners that the value has changed. +Other than that it is a pretty standard Java class. + +## Basic data-binding + +In the frontend plug-in's MANIFEST.MF add the backend plug-in to the list of dependencies. Also, add the following dependencies: + +* `org.eclipse.core.databinding` +* `org.eclipse.core.databinding.beans` +* `org.eclipse.core.databinding.property` +* `org.eclipse.jface.databinding` + +Open the View class and change it to the following: +```java +package org.databinding.example.frontend; + +import org.databinding.example.backend.Person; +import org.eclipse.swt.SWT; +import org.eclipse.swt.layout.GridData; +import org.eclipse.swt.layout.GridLayout; +import org.eclipse.swt.widgets.Composite; +import org.eclipse.swt.widgets.Label; +import org.eclipse.swt.widgets.Text; +import org.eclipse.ui.part.ViewPart; +import org.eclipse.core.databinding.Binding; +import org.eclipse.core.databinding.DataBindingContext; +import org.eclipse.core.databinding.observable.value.IObservableValue; +import org.eclipse.jface.databinding.fieldassist.ControlDecorationSupport; +import org.eclipse.jface.databinding.swt.WidgetProperties; +import org.eclipse.core.databinding.beans.BeanProperties; + +public class View extends ViewPart { + public static final String ID = "org.databinding.example.frontend.view"; + + private Person person; + + public View() { + person = new Person("John", 25); + } + + public void createPartControl(Composite parent) { + GridLayout glParent = new GridLayout(2, false); + glParent.marginHeight = 5; + glParent.marginWidth = 5; + glParent.horizontalSpacing = 1; + glParent.verticalSpacing = 0; + parent.setLayout(glParent); + + Label lblName = new Label(parent, SWT.NONE); + lblName.setText("Name: "); + lblName.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, false, false, 1, 1)); + + Text txtName = new Text(parent, SWT.BORDER); + GridData gdText = new GridData(SWT.LEFT, SWT.CENTER, true, false, 1, 1); + gdText.minimumWidth = 100; + txtName.setLayoutData(gdText); + + Text txtReadName = new Text(parent, SWT.BORDER); + GridData gdReadText = new GridData(SWT.LEFT, SWT.CENTER, true, false, 1, 1); + gdReadText.minimumWidth = 100; + txtReadName.setLayoutData(gdReadText); + txtReadName.setEditable(false); + + Label lblAge = new Label(parent, SWT.NONE); + lblAge.setText("Age: "); + lblAge.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, false, false, 1, 1)); + + Text txtAge = new Text(parent, SWT.BORDER); + GridData gdAge = new GridData(SWT.LEFT, SWT.CENTER, true, false, 1, 1); + gdAge.minimumWidth = 100; + txtAge.setLayoutData(gdAge); + + // DATABINDING START + DataBindingContext ctx = new DataBindingContext(); + + IObservableValue target = WidgetProperties.text(SWT.Modify).observe(txtName); + IObservableValue model = BeanProperties.value("name").observe(person); + ctx.bindValue(target, model); + + // Bind on one line + ctx.bindValue(WidgetProperties.text(SWT.Modify) + .observe(txtReadName), BeanProperties.value("Name").observe(person)); + + // Use default validation - binding code on one line + Binding bindValue = ctx.bindValue(WidgetProperties.text(SWT.Modify) + .observe(txtAge), BeanProperties.value("age").observe(person)); + + ControlDecorationSupport.create(bindValue, SWT.TOP | SWT.RIGHT); + // DATABINDING END + } + + public void setFocus() { + + } +} +``` + +The binding of the name is a simple as it gets; an `IObservableValue` is created for both the widget and the name in the Person class +and these are bound together in the `DataBindingContext` object. + +For age, because it is an int it is possible to make the widget show an error message when is contains a value that is not an int +using `ControlDecorationSupport` as shown. + +If the UI is started then it is possible to see the data-binding in action, by using the debugger it is possible to see that the +`setName` method is called every time a change is made to the widget text, i.e. every time a letter is added or removed. + +## Validators + +In the previous example, a warning was shown when the age entered was invalid - this is basic validation, it might be that more advanced validation is desired. +For example, it might that we want to ensure a string is only made up of digits. + +Create a new class called `NumbersOnlyValidator` and add the following code: +```java +package org.databinding.example.frontend; + +import java.util.regex.Pattern; + +import org.eclipse.core.databinding.validation.IValidator; +import org.eclipse.core.databinding.validation.ValidationStatus; +import org.eclipse.core.runtime.IStatus; + +public class NumbersOnlyValidator implements IValidator { + + private final Pattern numbersOnly = Pattern.compile("\\d*"); + + public IStatus validate(Object value) { + if (value != null && numbersOnly.matcher(value.toString()).matches()) { + return ValidationStatus.ok(); + } + return ValidationStatus.error(value + " contains a non-numeric!"); + } +} +``` +All custom validators implement `IValidator`. In this example, pattern matching is used to check that the value (in this case a string) contains only digits. +Otherwise it returns an error status. + +Now modify View by adding the following directly before DATABINDING END: +```java +Label lblId = new Label(parent, SWT.NONE); +lblId.setText("ID: "); +lblId.setLayoutData(new GridData(SWT.FILL, SWT.CENTER, false, false, 1, 1)); + +Text txtId = new Text(parent, SWT.BORDER); +GridData gdId = new GridData(SWT.LEFT, SWT.CENTER, true, false, 1, 1); +gdId.minimumWidth = 100; +txtId.setLayoutData(gdId); + +IValidator validator = new NumbersOnlyValidator(); + +UpdateValueStrategy strategy = new UpdateValueStrategy(); +strategy.setBeforeSetValidator(validator); +Binding bindValue2 = ctx.bindValue(WidgetProperties.text(SWT.Modify) + .observe(txtId), + BeanProperties.value("idNumber").observe(person), strategy, + null); +ControlDecorationSupport.create(bindValue2, SWT.TOP | SWT.RIGHT); + +``` +This code creates a label and text-box for the ID field of the Person class. +An instance of `NumbersOnlyValidator` is created and added to a `UpdateValueStrategy` object. +The `UpdateValueStrategy` object defines how a data-binding responses to value changes - in this case it calls the validator before setting the value. +The strategy is added as the third parameter of the `bindValue` method call, this is because the third parameter represents the strategy for updating the model based on the widget. +The fourth parameter is null because there is currently no update strategy for updating the widget based on the model (the default is used). + +By running this example it can be seen that a warning appears if a non-numeric character is entered for the ID. + +## Converters + +Converters are similar to validators in that they are used as part of the update strategy; however, they are used to convert either the data from the model or the data send to the model by the widget. +In this example, two converters are created: one to convert the name to upper-case; and, one to convert to lower-case. + +Create two new classes called `ToLowerConverter` and `ToUpperConverter`. + +Add the following code to `ToLowerConverter`: +```java +package org.databinding.example.frontend; + +import org.eclipse.core.databinding.conversion.Converter; + +public class ToLowerConverter extends Converter { + + public ToLowerConverter() { + super(String.class, String.class); + } + + public Object convert(Object fromObject) { + return fromObject.toString().toLowerCase(); + } +} +``` + +Likewise for ToUpperConverter: +```java +package org.databinding.example.frontend; + +import org.eclipse.core.databinding.conversion.Converter; + +public class ToUpperConverter extends Converter { + + public ToUpperConverter() { + super(String.class, String.class); + } + + public Object convert(Object fromObject) { + return fromObject.toString().toUpperCase(); + } +} + +``` +Now replace the code between the DATABINDING START and the DATABINDING FINISH with: +```java +DataBindingContext ctx = new DataBindingContext(); + +UpdateValueStrategy strategy1 = new UpdateValueStrategy(); +strategy1.setConverter(new ToLowerConverter()); + +UpdateValueStrategy strategy2 = new UpdateValueStrategy(); +strategy2.setConverter(new ToUpperConverter()); + +IObservableValue target = WidgetProperties.text(SWT.Modify).observe(txtName); +IObservableValue model = BeanProperties.value("name").observe(person); + +// First strategy goes towards the model, second goes towards the widget (target) +ctx.bindValue(target, model, strategy1, strategy2); + +``` +Note: The data-binding for age has been deleted, so the age text-box will be empty when this is run. + +Running this example should show the name in upper-case and it ran through the debugger it can be seen that if the value is modified the `setName` method in Person receives a lower-case string. + +## Update strategies + +There are other things that can be done with update strategies, in this example the model is only updated when the update button is clicked (unlike in the first example where the person's name changes each time it is modified). + +Replace the code between the DATABINDING START and the DATABINDING FINISH with: +```java +final DataBindingContext ctx = new DataBindingContext(); + +UpdateValueStrategy strategyOnPress = new UpdateValueStrategy(UpdateValueStrategy.POLICY_ON_REQUEST); + +IObservableValue target = WidgetProperties.text(SWT.Modify).observe(txtName); +IObservableValue model = BeanProperties.value("name").observe(person); +ctx.bindValue(target, model, strategyOnPress, null); + +Button btnUpdate = new Button(parent, SWT.NONE); +btnUpdate.setText("Update"); +txtAge.setLayoutData(new GridData(SWT.LEFT, SWT.CENTER, true, false, 1, 1)); + +btnUpdate.addSelectionListener(new SelectionAdapter() { + @Override + public void widgetSelected(SelectionEvent arg0) { + ctx.updateModels(); + } +}); + +``` +In this example, the Update strategy is initialised with POLICY_ON_REQUEST. This means the updates are only send to the model (the Person object) when told to. +The models are updated when `updateModels` is called by the `DataBindingContext` which in this example is on a button click. + +Running this example in the debugger shows that the `setName` method is only called when the button is clicked rather than on every modification. + +## Binding a Java List to a List Control + +Most of this is taken care of by the databinding library via `ListViewer` and `ObservableListContentProvider`. Here is an example of it in action: + +```java +ListViewer myViewer = new ListViewer(parent, SWT.BORDER | SWT.V_SCROLL | SWT.SINGLE); +ObservableListContentProvider contentProvider = new ObservableListContentProvider(); +myViewer.setContentProvider(contentProvider); +myViewer.setInput(BeanProperties.list("names").observe(myViewModel)); + +// To get the List Control itself - it is org.eclipse.swt.widget.List not a standard Java List +List myList = myViewer.getList(); +``` + +`ListViewer` is a wrapper around the List control that provides extra functionality and `ObservableListContentProvider` make the databinding work. \ No newline at end of file diff --git a/doc/client/coding/GUI-Coding-Conventions.md b/doc/client/coding/GUI-Coding-Conventions.md new file mode 100644 index 000000000..e9a4e11e2 --- /dev/null +++ b/doc/client/coding/GUI-Coding-Conventions.md @@ -0,0 +1,258 @@ +# Conventions + +Contains the style and coding conventions for the IBEX GUI. + +**New conventions should not be added to this document without first being discussed at a 'Code Chat'.** + +## Style Conventions # + +Unless stated otherwise below we should follow the standard Java conventions for style where possible. + +## Code documentation ## + +Classes and public methods should be documented using the Javadoc syntax. For example: + +```java +/** + * A make-your-own pizza. + * + * This class allows a custom pizza to be order by the customer. + * Note: it does not calculate the price. + */ +public class CustomPizza extends Pizza { + private final static int MAX_TOPPINGS = 6; + private List toppingList; + + /** + * The default constructor. + */ + public CustomPizza() { + toppingList = new ArrayList(); + } + + /** + * Add a topping to the pizza. + * + * @param topping the topping to add + * @return true if the topping was added + */ + public boolean addTopping(Topping topping) { + if (toppingList.size() < MAX_TOPPINGS) { + toppingList.add(topping); + return true; + } + + return false; + } +} +``` +For Getters it is okay to skip the first sentence and do the following as it saves repetition: +```java +/** + * @return true if the block is enabled + */ +public boolean getEnabled() { + return enabled; +} +``` + +## Code formatting ## + +For Java use the standard conventions built in to the IBEX developer's version of Eclipse. + +An example of what it looks like: +```java +void foo2() { + if (true) { + return; + } + + if (true) { + return; + } else if (false) { + return; + } else { + return; + } +} +``` +In Eclipse, a quick way to auto-format the code correctly is to use Ctrl+Shift+F. + +## Code comments ## + +Comments should have a space between the // and the text, and start with a capital letter: +```java +// This is a good comment + +//this is a bad comment +``` + +## Checkstyle ## + +Code should be run through Checkstyle via Eclipse and corrected (within reason) before being committed. +The Checkstyle plug-in is installed as part of the IBEX developer's version of Eclipse. + +The Checkstyle configuration file for Eclipse is more picky that the one used on Jenkins as it warns about 'magic numbers' and 'Java-docs'. + +By right-clicking on a file one can tell Checkstyle to check that file. + +Warnings must be fixed where possible. + +Checkstyle has a suppress warning flag that tells it to ignore certain warning; warnings that are allowed to be suppressed are: + +* Magic numbers - if it is related to a GUI layout then suppress them. + +* Name must match pattern - ignore GUI names that don't match the recommended pattern (e.g. `gd_switchToCombo`) + +Suppression example: + +```java +@SuppressWarnings({ "checkstyle:magicnumber", "checkstyle:localvariablename" }) +public void getSecondsInHours(int hours) { + int seconds_per_hour = 60 * 60; // Magic numbers and a bad variable name + return hours * seconds_per_hour; +} +``` +## Naming conventions ## + +### General ### + +In general we use the standard Java naming conventions, e.g.: + +* Class names are CamelCase and usually nouns, e.g. `FileReader` not `ReadsFile` + +* Method names are Mixed Case (also known as Lower CamelCase), e.g. `sayHello` + +* Package names are lowercase, e.g. `uk.ac.stfc.isis.dae` + +* Variable names are Mixed Case, e.g. `calculateMeanSpeed` + +* Constants are uppercase spaced by underscores, e.g. `SECONDS_PER_FORTNIGHT` + +### Getters and Setters ### + +Getters and Setters should follow the JavaBeans convention, namely: + +* **Getter** - starts with "get" + +* **Boolean Getter** - can start with "is" or "get" + +* **Setter** - starts with "set" + +For example: +```java +class Point { + private double x; + private double y; + private boolean visible; + + public Point(double x, double y) { + this.x = x; + this.y = y; + this.visible = true; + } + + public double getX() { return x; } + public void setX(double x) { this.x = x; } + + public double getY() { return y; } + public void setY(double y) { this.y = y; } + + public boolean isVisible { return visible; } + public void setVisible(boolean visible) { this.visible = visible; } +} +``` + +## Coding Conventions ## + +A mix of IBEX specific and general Java coding conventions. + +### GUI code must use a View Model ### + +This maintains a separation between pure GUI code and the back-end. It also makes it easier for us to test our code. +See the previous GUI Chat slides for more information. + +Some legacy code does not have a View Model, this is on the list to fix. + +### Use data-binding for GUI items ### + +~~For connecting UI elements to data from the View Model use data-binding. +It seems that if a mix of data-binding and more traditional SWT wiring up is used (e.g. AddPropertyChangeListener) then the data-binding will stop working*, so always using data-binding should avoid this problem.~~ + +~~*This does need more investigation to find out why it occurs.~~ + +This no longer seems to apply. + +### Don't mess with finalizers ### +From the Google Java Style Guide: + +``` +It is extremely rare to override Object.finalize. + +Tip: Don't do it. If you absolutely must, first read and understand Effective Java Item 7, "Avoid Finalizers," very carefully, and then don't do it. +``` + +As of Java 9, `finalize` is officially deprecated in java. The IBEX checkstyle configuration is configured to disallow finalizers - this check should not be disabled or overridden. + +The remaining options for supported clean-up mechanisms (in preference order) are: +- Implement `autocloseable` and use the class in a try-with-resources statement to ensure the relevant resource is closed +- Use a `closeable` and manually call `close` to ensure the relevant resource is closed +- Refactor to avoid needing to close the resource at all +- Use a *supported* automatic clean-up mechanism such as `PhantomReference` or `Cleaner` only as a last resort. While these are *better* than finalizers, they still suffer from high complexity and are tricky to get right. In particular it is easy to accidentally create reference loops meaning that objects will never be cleaned. + +### Return a empty collection or stream, not null ### +For methods that return arrays/lists/maps/sets/streams etc. don't return null. It is cleaner to return an empty instance as the calling code does not need to check for null. + +```java +// Avoids this extra check in the caller +if (cars != null && cars.contains("mustang") { + ... +} +``` +See Effective Java Item 43 "Return empty arrays or collections, not nulls" + +### Prefer StringBuilder for string concatenation ### +Using `+` is fine for, say, joining two or three short strings but it is inefficient for larger numbers of strings and longer strings. Use StringBuilder instead. + +See Effective Java Item 51 "Beware the performance of string concatenation" + +### Prefer `Optional` over `null` + +New APIs should not return null to indicate a missing value. Instead, return an Optional wrapping the value which may not exist. Where external code returns null to indicate a missing value, this should be wrapped in an optional as soon as reasonable. + +To convert a maybe-null value to an optional, use: +```java +String s = null; +Optional stringWhichMightNotExist = Optional.ofNullable(s); +``` + +To convert an Optional to a maybe-null value, use: +```java +Optional mightNotExist = Optional.empty(); +String str = mightNotExist.orElse(null); +``` + +### Streams + +Streams should be used where they make an algorithm clearer. + +When using streams, put each operation on it's own line. + +Good: +```java +public Stream getNames() { + return getThings() + .map(thing -> thing.getName()) + .filter(name -> name != "") + .sorted(); +} +``` + +Bad: +```java +public Stream getNames() { + return getThings().map(thing -> thing.getName()).filter(name -> name != "").sorted(); +} +``` + + diff --git a/doc/client/coding/IBEX_complete_DAE_readonly.png b/doc/client/coding/IBEX_complete_DAE_readonly.png new file mode 100644 index 000000000..6a7c9117f Binary files /dev/null and b/doc/client/coding/IBEX_complete_DAE_readonly.png differ diff --git a/doc/client/coding/IBEX_complete_clean.png b/doc/client/coding/IBEX_complete_clean.png new file mode 100644 index 000000000..e6c9c0fe7 Binary files /dev/null and b/doc/client/coding/IBEX_complete_clean.png differ diff --git a/doc/client/coding/IBEX_complete_perspective_switcher_highlighted.png b/doc/client/coding/IBEX_complete_perspective_switcher_highlighted.png new file mode 100644 index 000000000..e9e857a22 Binary files /dev/null and b/doc/client/coding/IBEX_complete_perspective_switcher_highlighted.png differ diff --git a/doc/client/coding/Instrument-switching.md b/doc/client/coding/Instrument-switching.md new file mode 100644 index 000000000..710b68bba --- /dev/null +++ b/doc/client/coding/Instrument-switching.md @@ -0,0 +1,15 @@ +# Instrument switching + +Instrument switching in the GUI uses an extension point. This means that the switch can take place in a central place but then each plugin which is interested can sign up to the switching event. This keeps a separation between the plugins and the instrument switching module; i.e. a plugin can be removed without changing the instrument switching code. + +The instrument switching in E4 is performed through the E3 compatibility layer, as E4 has no native support for extension points. The equivalent behaviour in E4 is provided through services, which might be a necessary transition as services and extensions points are not cross-compatible. + +This extension point is setup in `uk.ac.stfc.isis.ibex.instrument/META-INF/MANIFEST.MF` (see the extension Points tab). This sets up the name of the extension point and the schema. The schema is in `/uk.ac.stfc.isis.ibex.instrument/schema/uk.ac.stfc.isis.ibex.instrument.info.exsd` (click Schema on previous page). This defines the methods that should be fulfilled by the plugin that want to sign up to this extension. In this case there are three methods: + + `preSetInstrument` - this will be called before the instrument is switched. This is useful for closing resources using the old instrument. + `setInstrument` - this will be called to set the instrument and should actually perform the change. + `postSetInstrument` - this will be called after the instrument is set. It can be used, for example, to perform final clean up of resources or reopening of perspectives. + +The instrument handler is responsible for calling this method on each registered plugin. It does this in the method `uk.ac.stfc.isis.ibex.instrument.Instrument.setInstrumentForAllPlugins` in the private method `updateExtendingPlugins`. Notice that it call all preSetInstruments methods before setInstrument. + +To sign up to this event the plugin must create a class which implements the interface `uk.ac.stfc.isis.ibex.instrument.InstrumentInfoReceiver`. This will be instantiated when the instrument is switched to. This class is registered in the plugin's extensions (this is similar to the perspectives see [adding a perspective](Adding-a-Button-to-the-Perspective-Switcher) ). diff --git a/doc/client/coding/Migrating-or-adding-a-button-to-the-E4-perspective-switcher.md b/doc/client/coding/Migrating-or-adding-a-button-to-the-E4-perspective-switcher.md new file mode 100644 index 000000000..d2210c75e --- /dev/null +++ b/doc/client/coding/Migrating-or-adding-a-button-to-the-E4-perspective-switcher.md @@ -0,0 +1,38 @@ +# Adding a button to the E4 Perspective switcher + +## Migrating an E3 perspective to E4 + +1. Open the Application.e4xmi from `uk.ac.stfc.isis.ibex.e4.client` +1. Go to `Snippets` +1. Click `Add` to add a new perspective +1. Set the perspective up using an existing migrated perspective as a template + 1. Set a sensible ID + 1. Give it a label + 1. If you want your perspective to be invisible toggle the visible checkbox + 1. Set the icon + 1. Add controls. This should be a hierarchy of part sash containers. You can see how it should be set up from the existing perspectives. Don't forget to set the container data where appropriate; it sets the relative size of sibling components. +1. Add the perspective-specific parts + 1. In the alarms perspective, you'll see one part in the final part sash container called alarms. Do the same thing in your new perspective, but give it an appropriate name + 1. Change the ID of your new part to the ID of the view class you want the perspective to open +1. Add the dependency of the view you've added to the `plugin.xml` in the `...e4.client` plugin +1. Add the new dependency to `...feature.xml` (in `uk.ac.stfc.isis.ibex.feature.base`) +1. Synchronize `ibex.product` (in `...e4.client.product`) +1. Open IBEX +1. Check the new perspective scales appropriately and change the layout accordingly if needed + +## Creating a brand new E4 perspective + +Making a brand new E4 perspective would probably look similar to the steps above, minus the E3 steps. However, a new E4 perspective has yet to been attempted. + +## Hiding Perspectives + +Perspectives can be hidden by adding perspective IDs to the Eclipse preference store at the preference key `uk.ac.stfc.isis.ibex.preferences/perspectives_not_shown` (at `/uk.ac.stfc.isis.ibex.e4.client/plugin_customization.ini`). e.g. + +``` +uk.ac.stfc.isis.ibex.preferences/perspectives_not_shown=uk.ac.stfc.isis.ibex.client.e4.product.perspective.scriptGenerator +``` +Note: Multiple perspectives can be hidden using a comma separated list of perspective IDs. e.g. + +``` +uk.ac.stfc.isis.ibex.preferences/perspectives_not_shown=uk.ac.stfc.isis.ibex.client.e4.product.perspective.scriptGenerator,uk.ac.stfc.isis.ibex.client.e4.product.perspective.dae +``` \ No newline at end of file diff --git a/doc/client/coding/PV-Switching.md b/doc/client/coding/PV-Switching.md new file mode 100644 index 000000000..1e0cde652 --- /dev/null +++ b/doc/client/coding/PV-Switching.md @@ -0,0 +1,49 @@ +# PV switching + +If you are only interested in how to create PVs in IBEX with proper switching behaviour go to "Using the PV Switching". + +## Background ## + +PV switching occurs when a user in IBEX changes from viewing one instrument to another. It is then required that PVs 'switch' prefix and instead of, for example, looking at IN:LARMOR:CS:BLOCKSERVER:CONFIGS the GUI will now point to IN:DEMO:CS:BLOCKSERVER:CONFIGS. This means that information about the new instrument will be displayed. + +This was originally done in IBEX by maintaining a PVAddressBook class that kept a map of all PV-connected observables that had been registered with it and reset their prefixes when the instrument was changed. This created a number of issues: + +* Some PVs might not need to be switched (e.g. the beam status), this was solved by bypassing the `PVAddressBook` and creating PVs in other plugins + +* There was no way to remove PVs from this address book, which led to an ever increasing map, and subsequent channel access connection attempts, when configurations or instruments were changed + +* Some PVs will need to be closed all together when instruments are switched, such as those relating to specific configurations or synoptics. There was no mechanism for this to happen within the PVAddressBook + +* PV creation was messy, requiring inheriting from specific classes or some knowledge of how the underlying PVs were created + +This was further complicated by the large number of Observable, Observer, Writable and Writer classes that were used to observe PVs in different situations. + +## Solution Design + +The proposed solution was to remove the central PVAddressBook class and instead use a factory for creating PV Observables, which are passed the switching behaviour as a switcher class. Each of these switcher classes provides a different switching functionality and is switched using the extension point that is globally used for instrument switching in the GUI. + +![Switching](new_switching.jpg) + +1. When an observable PV is required an instance of ObservableFactory is first created and passed an OnInstrumentSwitch enum to describe which switching behaviour is required + +1. On initialisation the factory will then query the InstrumentSwitchers class for the specific Switcher class that handles that type of switching + +1. The original class that wanted the PV will subsequently ask the factory for PVs, providing the channel type and PV address. This will be provided as a `SwitchableObservable` that can be subscribed to for value changes. + +1. Before providing the new Observable object the factory will register it with the relevant Switcher class, each of which holds a list of all `Switchable` objects that it is required to switch. The `SwitchableObservable` is also provided with a reference to the switcher so that it can remove itself from the relevant list if it is closed for any reason. + +1. When the instrument is changed the InstrumentSwitchers class will call the `switchInstruments` method on each of the switchers, which will then go on to perform the relevant switching behaviour. E.g. Changing a `Switchable`'s source, closing it or doing nothing. + +A similar process also occurs when switching writable PVs, as can be seen in the UML diagram below. The differences being that a `WritableFactory` is used, this can create a Writable that inherits from `Switchable` and can write values to PVs. Both the `Switchable` interface and the abstract `PrefixChangingSwitcher` were created so that the switching process is as similar as possible when dealing with `Writables` and `Observables` + +![Writables](new_switching_writables.jpg) + +## Using the PV Switching + +The inner workings of the switching code need not be understood to create PVs that switch with the instrument. Only steps 1 and 3 in the previous list need be performed by a class that wants a new PV. Specifically the following code must be used +```java +ObservableFactory closingObsFactory = new ObservableFactory(OnInstrumentSwitch.CLOSE); +ForwardingObservable pv + = closingObsFactory.getSwitchableObservable(new StringChannel(), "A_PV_ADDRESS")); +``` +The above code will create a String type PV observable that will close when the instrument is changed. Subscriptions can now be attached to the PV and will be called when PVs change value. diff --git a/doc/client/coding/Static-analysis.md b/doc/client/coding/Static-analysis.md new file mode 100644 index 000000000..05e5b2c7d --- /dev/null +++ b/doc/client/coding/Static-analysis.md @@ -0,0 +1,95 @@ +# Static analysis + +We are currently making use of 3 static analysis tools to help ensure the quality of our code: [PMD](https://pmd.github.io/), [FindBugs](http://findbugs.sourceforge.net/), and [CheckStyle](https://checkstyle.sourceforge.io/). These particular tools are mostly intended for use with Java programs and as such our main use for them is the Eclipse GUI project, though they are also used on some Java code in the main EPICS project (specifically the IOC log server). + +All of these tools have maven plugins which allow the analyses to be run as part of the maven build process, with the results being output to an XML file. Additionally, they all have Jenkins plugins which can consume the XML results files, producing reports and graphs within Jenkins and allowing you to track code quality trends over time. The Jenkins reports list all the specific violations which can be categorised in a number of ways and are listed with file name and line number as well as a detailed description of the problem. + +## Checkstyle + +Checkstyle is designed to enforce a set of highly configurable coding standards. It supports a very large number of rules including ones relating to naming conventions, annotations, javadoc comments, poor coding practices, etc. That rules that Checkstyle will check for violations of may be configured in an XML file. + +The style rules imposed by Checkstyle are very demanding and it may be helpful to consider them as guidelines rather than as hard rules. At our first analysis of the Eclipse GUI project, Checkstyle showed over 30000 errors and it probably isn't worth the effort to fix all of these. However moving forward, when writing new code, it is a good idea to make at least some effort to conform to the Checkstyle 'rules'. + +Checkstyle configuration is done by XML file called 'checkstyle.xml' which is located in the Tycho parent directory. This is then set up as a global rule set called IBEX Checks. The rule set used is the default Sun ruleset with some rules disabled (by commenting them out in the XML file). In order to reduce the number of needless Checkstyle warnings, the following specific rules, which are enabled by default, have been disabled for Jenkins: + +* FileTabCharacterCheck - using a tab character anywhere in the code +* RegexpSinglelineCheck - line has trailing white space +* LineLengthCheck - more than 80 characters in a line +* JavaDoc - checks for JavaDoc comments +* LineLength - checks that lines are not longer than 80 chars +* AvoidInlineConditionals - checks for ? : style conditionals +* HiddenField - checks that method parameter names do not mirror class parameter names +* InnerAssignment - checks that inner assignment is not used +* DesignForExtension - enforces a programming style where superclasses provide empty "hooks" that can be implemented by subclasses. +* FinalParameters - checks for method parameters that should be declared "final" +* NewlineAtEndOfFile - checks for a newline at the end of the code file +* MagicNumber - checks for numbers that could perhaps be variables instead + +The ISIS standard Checkstyle rules for the IDE is stored in the repository for our client, so that can just be imported rather than setting the rules up by hand. For the Eclipse IDE the "MagicNumber" rule is not disabled as it does occasionally provide some useful warnings, but can be ignored for cases where it is being pernickety. + +An example of a "useful" warning: +``` + @Override + public int hashCode() { + return displayName.hashCode() ^ (isWritable ? 1231 : 1237); // It is not clear what these numbers represent + } +``` + +An example of an overzealous warning: +``` + @Override + public String getColumnText(Object element, int columnIndex) { + IocState ioc = (IocState) element; + switch (columnIndex) { + case 0: + //The hidden column + case 1: + //The IOC Name column + return ioc.getName(); + case 2: + //The IOC Name column + return ioc.getDescription(); + case 3: // MagicNumber flags this up + //The current state column + //This is handled in getColumnImage + case 4: // MagicNumber flags this up too + // Dummy column that is hidden by the scrollbar + return ""; + default: + return "Unknown column"; + } + } +``` +There is also the option of using the `@SuppressWarnings` qualifier to tell Checkstyle to ignore certain warnings for specific classes or methods. For example: +``` + @SuppressWarnings("checkstyle:magicnumber") + public void getSecondsInHours(int hours) { + return hours * 60 * 60; // Magic numbers! + } +``` +Or for multiple warnings: + +``` + @SuppressWarnings({"checkstyle:magicnumber", "checkstyle:localvariablename"}) + public void getSecondsInHours(int hours) { + int seconds_per_hour = 60 * 60; // Magic numbers and a variable name that does not conform to the recommended style! + return hours * seconds_per_hour; + } +``` +## PMD + +PMD focuses on potential coding problems such as unused or suboptimal code, code size and complexity, and good coding practices. Some typical rules include "Empty If Statement", "Broken Null Check", "Avoid Deeply Nested If Statements", "Switch Statements Should Have Defaults", etc. PMD will produce far fewer warnings/errors than checkstyle, however they are typically far more important, indicating technical problems with the code rather than merely stylistic ones, and should therefore be fixed whenever possible/sensible. + +## FindBugs + +FindBugs detects similar types of problems as PMD, though it can detect many additional serious problems such as potential null pointer exceptions, infinite loops, and unintentional access of the internal state of an object (to name a few). Unlike PMD, which checks through the source code, FindBugs actually checks through the applications bytecode for potential issues. FindBugs tends to find a smaller number of more important issues. + +For both PMD and FindBugs, there are certain rules which, while generally sensible, may not apply to the project at hand. For example, on the Eclipse GUI project, FindBugs throws up a lot of problems involving writing to a static member from an instance method, however the Activator class in most Eclipse plugins make use of this construct. + +## Maven + +The tools may be run as part of a maven build by including them as plugins in the project's pom.xml. In the case of the Eclipse GUI, each plugin is included in the `tycho.parent` pom.xml file, and is therefore used in every other project (Eclipse plugin) during the build. + +* [FindBugs maven plugin](https://gleclaire.github.io/findbugs-maven-plugin/index.html) +* [CheckStyle maven plugin](http://maven.apache.org/plugins/maven-checkstyle-plugin/) +* [PMD maven plugin ](http://maven.apache.org/plugins/maven-pmd-plugin/) \ No newline at end of file diff --git a/doc/client/coding/basic_principle.png b/doc/client/coding/basic_principle.png new file mode 100644 index 000000000..2099e3c3c Binary files /dev/null and b/doc/client/coding/basic_principle.png differ diff --git a/doc/client/coding/eclipse_add_icons_to_build.png b/doc/client/coding/eclipse_add_icons_to_build.png new file mode 100644 index 000000000..6c50c464b Binary files /dev/null and b/doc/client/coding/eclipse_add_icons_to_build.png differ diff --git a/doc/client/coding/eclipse_add_perspective_plugin1.png b/doc/client/coding/eclipse_add_perspective_plugin1.png new file mode 100644 index 000000000..18e285415 Binary files /dev/null and b/doc/client/coding/eclipse_add_perspective_plugin1.png differ diff --git a/doc/client/coding/eclipse_add_perspective_plugin2.png b/doc/client/coding/eclipse_add_perspective_plugin2.png new file mode 100644 index 000000000..acc258b63 Binary files /dev/null and b/doc/client/coding/eclipse_add_perspective_plugin2.png differ diff --git a/doc/client/coding/eclipse_adding_a_View.png b/doc/client/coding/eclipse_adding_a_View.png new file mode 100644 index 000000000..d01077ca0 Binary files /dev/null and b/doc/client/coding/eclipse_adding_a_View.png differ diff --git a/doc/client/coding/eclipse_adding_a_package.png b/doc/client/coding/eclipse_adding_a_package.png new file mode 100644 index 000000000..97f293277 Binary files /dev/null and b/doc/client/coding/eclipse_adding_a_package.png differ diff --git a/doc/client/coding/eclipse_extensions_added1.png b/doc/client/coding/eclipse_extensions_added1.png new file mode 100644 index 000000000..8eff1532a Binary files /dev/null and b/doc/client/coding/eclipse_extensions_added1.png differ diff --git a/doc/client/coding/eclipse_extensions_added2.png b/doc/client/coding/eclipse_extensions_added2.png new file mode 100644 index 000000000..3b9d06a3a Binary files /dev/null and b/doc/client/coding/eclipse_extensions_added2.png differ diff --git a/doc/client/coding/eclipse_extensions_added3.png b/doc/client/coding/eclipse_extensions_added3.png new file mode 100644 index 000000000..5c0c12d10 Binary files /dev/null and b/doc/client/coding/eclipse_extensions_added3.png differ diff --git a/doc/client/coding/eclipse_extensions_added4.png b/doc/client/coding/eclipse_extensions_added4.png new file mode 100644 index 000000000..3fb1442ae Binary files /dev/null and b/doc/client/coding/eclipse_extensions_added4.png differ diff --git a/doc/client/coding/eclipse_extensions_added5.png b/doc/client/coding/eclipse_extensions_added5.png new file mode 100644 index 000000000..87b3c5bc5 Binary files /dev/null and b/doc/client/coding/eclipse_extensions_added5.png differ diff --git a/doc/client/coding/eclipse_no_extensions.png b/doc/client/coding/eclipse_no_extensions.png new file mode 100644 index 000000000..e5c6046af Binary files /dev/null and b/doc/client/coding/eclipse_no_extensions.png differ diff --git a/doc/client/coding/eclipse_perspective_copy_errors.png b/doc/client/coding/eclipse_perspective_copy_errors.png new file mode 100644 index 000000000..e1f79d74f Binary files /dev/null and b/doc/client/coding/eclipse_perspective_copy_errors.png differ diff --git a/doc/client/coding/eclipse_select_extension_point.png b/doc/client/coding/eclipse_select_extension_point.png new file mode 100644 index 000000000..d39514776 Binary files /dev/null and b/doc/client/coding/eclipse_select_extension_point.png differ diff --git a/doc/client/coding/new_switching.jpg b/doc/client/coding/new_switching.jpg new file mode 100644 index 000000000..c3dad33ba Binary files /dev/null and b/doc/client/coding/new_switching.jpg differ diff --git a/doc/client/coding/new_switching_writables.jpg b/doc/client/coding/new_switching_writables.jpg new file mode 100644 index 000000000..62e21117d Binary files /dev/null and b/doc/client/coding/new_switching_writables.jpg differ diff --git a/doc/client/coding/parent_pom.png b/doc/client/coding/parent_pom.png new file mode 100644 index 000000000..f7e9ca651 Binary files /dev/null and b/doc/client/coding/parent_pom.png differ diff --git a/doc/client/css_other/Debugging-CSS.md b/doc/client/css_other/Debugging-CSS.md new file mode 100644 index 000000000..d0746b720 --- /dev/null +++ b/doc/client/css_other/Debugging-CSS.md @@ -0,0 +1,38 @@ +# Debugging CSS Views + +Sometimes we find issues in our code that originate from CSS. This page originates from [ticket #1206](https://github.com/ISISComputingGroup/IBEX/issues/1206) and some of the resources that were used to analyse the problem. This page may grow owing to future issues; please feel free to add to it. + +## Source inspection within ECLIPSE + +Note that whilst in ECLIPSE debug mode, you should be able to see most of the source you're accessing via breakpoints and the normal step into, step over commands, including CSS. However, if the source is being loaded via reflection, as is the case in the above issue, you will only be able to see the code at call time, you won't be able to navigate to specific definitions which can be frustrating. + +Note that you may have to go through numerous layers of system code for which the source is unavailable. The debug stepping commands will still work, you just need to keep at it until you get to an inspection enabled class. + +## Breakpoints + +My experience in ECLIPSE is that breakpoints will work in 3rd party CSS code, provided you can inspect the class in question (see above). The one caveat is that you can toggle a breakpoint at a specific line, and even make it conditional, but the breakpoint icon may not appear alongside the code. If you look at the Breakpoints window, it should still appear there, and it should work. So don't worry! + +## Source on GitHub + +As was the case in the attached issue, we might not be using the most up to date version of CSS. How do you know whether it has been changed in the source repo? Fortunately CSS is all open source. Unfortunately, the repo structure is quite hard to navigate. For instance, the package `org.csstudio.security.ui` is located [here](https://github.com/ControlSystemStudio/cs-studio/tree/master/core/utility/utility-plugins/org.csstudio.security.ui): + +`https://github.com/ControlSystemStudio/cs-studio/tree/master/core/utility/utility-plugins/org.csstudio.security.ui` + +Owing to efforts to clean up the file structure, it also means many files have moved. The Github `history` option for a file doesn't take that into account. I know of no way then to look at the full file history via Github. You have to do it by cloning the repo and then use the command: + +`git log --follow ./path/to/file` + +## Source inspection within the file system + +If you're trying to figure out what is going on in the code and the limitations on source inspection within ECLIPSE is stopping you, you can always look at the source that you're using directly through the file system. The `.jar` files used can be unzipped with most archiving programs (I use 7Zip). Simply navigate to the package in question and look in the `.source*` jar file. For instance, the source for `org.csstudio.security` on my system is located at: + +`C:\Instrument\Dev\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.csstudio.security.source_1.1.0.20140923-1503.jar\` + +## Plugin customisation + +Note that many of the CSS plugins use some degree of customization. This is set in the file `plugin_customization.ini`. For me it's located at: + +`C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.product\plugin_customization.ini` + +Packages with customization options should contain a `*.prefs` file defining the available options. + diff --git a/doc/client/css_other/Malformed-URL-errors.md b/doc/client/css_other/Malformed-URL-errors.md new file mode 100644 index 000000000..08438e982 --- /dev/null +++ b/doc/client/css_other/Malformed-URL-errors.md @@ -0,0 +1,12 @@ +# Malformed URL errors + +It is possible for the project files in the CS Studio application to become out of sync with what you have on disk. In this circumstance when attempting to load OPIs you will receive a malformed URL error message. + +To fix this you can complete the following steps: + +1. Delete the project files from the project using the Navigation view on the right of the screen. Be sure you do NOT check `'Delete project contents from disk'`. +2. Right click in the Navigation view white space (same location the project files were deleted from) and select `Import...`. +3. From the Import view select `Existing Projects into Workspace` and click Next. +4. Browse to your resources directory and click Finish. + +The project files will now be correctly loaded and it is recommended you restart CS Studio at this point to update the GUI menus. diff --git a/doc/client/css_other/PV-Connection-Layer.md b/doc/client/css_other/PV-Connection-Layer.md new file mode 100644 index 000000000..0c9fc1edb --- /dev/null +++ b/doc/client/css_other/PV-Connection-Layer.md @@ -0,0 +1,149 @@ +# PV Connection Layer + +This is what I think I know about how the layers under PVMAnagerObservable work to reconnect to a PV. This was part of the [logging ticket](https://github.com/ISISComputingGroup/IBEX/issues/2162) and led to a logging build [PV Manager and Observers Logging](../troubleshooting/PV-Manager-and-Observers-Logging). + +On disconnect the log is: + +``` +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 27 +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: CAJChannel connect changegov.aps.jca.Channel$ConnectionState[DISCONNECTED=1] +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: MultiplexChannelHandler connect changeJCAConnection [connected: false writeConnected: false channel: CAJChannel = { name = TE:NDW1407:CS:SB:TEMP1, connectionState = DISCONNECTED }] +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: ConnectionCollector connect changefalse +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVDirector connected or exception event false +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVReaderImpl connection +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVManagerObservable connection false +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: DisplayBlock connection false +``` + +The channel then is searched for: + +``` +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: ChannelSearchManager Moved channel search to timer with delay 32 +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.309000: ChannelSearchManager Moved channel search to timer with delay 64 + all = 13:52:47.309000: ChannelSearchManager no work to do not restarting timer. period32 +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.366000: PVReaderImpl valueChange +:NDW1407:CS:SB:TEMP1 = 13:52:47.366000: PVManagerObservable valueChange false +:NDW1407:CS:SB:TEMP1 = 13:52:47.383000: ChannelSearchManager Moved channel search to timer with delay 128 + all = 13:52:47.383000: ChannelSearchManager no work to do not restarting timer. period64 +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.521000: ChannelSearchManager Moved channel search to timer with delay 256 + all = 13:52:47.521000: ChannelSearchManager no work to do not restarting timer. period128 +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.787000: ChannelSearchManager Moved channel search to timer with delay 512 + all = 13:52:47.787000: ChannelSearchManager no work to do not restarting timer. period256 +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:48.309000: ChannelSearchManager Moved channel search to timer with delay 1024 + all = 13:52:48.309000: ChannelSearchManager no work to do not restarting timer. period512 +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:49.343000: ChannelSearchManager Moved channel search to timer with delay 2048 + all = 13:52:49.343000: ChannelSearchManager no work to do not restarting timer. period1024 +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:51.401000: ChannelSearchManager Moved channel search to timer with delay 4096 + all = 13:52:51.401000: ChannelSearchManager no work to do not restarting timer. period2048 +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:55.508000: ChannelSearchManager Moved channel search to timer with delay 8192 + all = 13:52:55.508000: ChannelSearchManager no work to do not restarting timer. period4096 +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: ChannelSearchManager request sent? true + all = 13:52:58.123000: ChannelSearchManager no work to do not restarting timer. period16384 +:NDW1407:CS:SB:TEMP1 = 13:53:03.711000: ChannelSearchManager Moved channel search to timer with delay 16384 + all = 13:53:03.711000: ChannelSearchManager no work to do not restarting timer. period8192 +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: ChannelSearchManager request sent? true +``` + +You can see that as the channel is not found it is sent to progressively longer period searches. Until is reaches the 16s one I have seen it up to a 64s delay. + +Then we reconnect the IOC and we get: + +``` + all = 13:53:05.425000: ChannelSearchManager no work to do not restarting timer. period32 + all = 13:53:05.788000: ChannelSearchManager no work to do not restarting timer. period128 + all = 13:53:06.224000: ChannelSearchManager no work to do not restarting timer. period256 + all = 13:53:08.641000: ChannelSearchManager no work to do not restarting timer. period128 +:NDW1407:CS:SB:TEMP1 = 13:53:20.534000: ChannelSearchManager Moved channel search to timer with delay 32768 + all = 13:53:20.534000: ChannelSearchManager no work to do not restarting timer. period16384 +:NDW1407:CS:SB:TEMP1 = 13:53:20.830000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:20.830000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:20.831000: ChannelSearchManager request sent? true + all = 13:53:38.603000: ChannelSearchManager no work to do not restarting timer. period32 + all = 13:53:39.815000: ChannelSearchManager no work to do not restarting timer. period64 + all = 13:53:39.954000: ChannelSearchManager no work to do not restarting timer. period128 + all = 13:53:46.211000: ChannelSearchManager no work to do not restarting timer. period16384 + all = 13:53:50.802000: ChannelSearchManager no work to do not restarting timer. period256 +:NDW1407:CS:SB:TEMP1 = 13:54:10.818000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:10.818000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:10.819000: ChannelSearchManager request sent? false + all = 13:54:11.286000: ChannelSearchManager no work to do not restarting timer. period512 +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:54:43.589000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 22 +:NDW1407:CS:SB:TEMP1 = 13:54:43.589000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 18 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 22 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 18 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAJChannel connect complete changegov.aps.jca.Channel$ConnectionState[DISCONNECTED=1] +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CATransport resubscribe subscriptions request +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CATransport resubscribe subscriptions request +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAJChannel connect changegov.aps.jca.Channel$ConnectionState[CONNECTED=2] +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: MultiplexChannelHandler connect changeJCAConnection [connected: true writeConnected: false channel: CAJChannel = { name = TE:NDW1407:CS:SB:TEMP1, connectionState = CONNECTED }] +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: ConnectionCollector connect changetrue +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: PVDirector connected or exception event true +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: PVReaderImpl connection +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: PVManagerObservable connection true +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: DisplayBlock connection true +``` + +The GUI gets access rights (22) then a connection (18). I have seen it get a version number too not sure whether this is only on first connect. Then we register for value changes I think. + +``` +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CreateChannel connect complete +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 22 +:NDW1407:CS:SB:TEMP1 = 13:54:43.592000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 18 +:NDW1407:CS:SB:TEMP1 = 13:54:43.592000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.592000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.592000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.592000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 15 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.593000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.602000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.602000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.602000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.602000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 1 +:NDW1407:CS:SB:TEMP1 = 13:54:43.602000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 15 +:NDW1407:CS:SB:TEMP1 = 13:54:43.692000: PVReaderImpl valueChange +:NDW1407:CS:SB:TEMP1 = 13:54:43.692000: PVManagerObservable valueChange true +``` + + +So I think the way it works is that there is a ladder of timers which send a search response for requested PVs onto the network (max delay appears to be 132s). If a connection is established the search is no longer requested if it isn't then it is moved down the ladder. Once the IOC replies the handshake is done and a connection is established the timing for this seems short so I don't think this is the problem. + +The logic of sending the search request seems complicated maybe the problem is in here (ChannelSearchManager.timeout) or maybe if there are many items in this list it fills up the send buffer so it takes a while for it to reconnect along with the delay. +Also we found that if the system gets to many responses it tell the system to stop sending flow control (CATransport.enableFlowControl). This may also cause a problem if this search is ignored because flow control is on and then it has to wait to send it again. Maybe this along with the large number of PVs might explain slow re-connection. I have created another ticket for someone else to look. diff --git a/doc/client/design_docs/Client-Architectural-Design.md b/doc/client/design_docs/Client-Architectural-Design.md new file mode 100644 index 000000000..83ac77e4f --- /dev/null +++ b/doc/client/design_docs/Client-Architectural-Design.md @@ -0,0 +1,32 @@ +# Client Architectural Design + +This document describes the architectural design for the IBEX client. Eventually there should be outlines of the architectural design for each part of the high level design. + +### Client Design Overview + +![Client Architecture](client_architecture.png) + +Notes on the figure: + +* CA - Channel Access +* JDBC - Java Database Connectivity +* JMS - Java Messaging Service + +### Eclipse RCP + +The IBEX client is based on Eclipse RCP (Rich Client Platform). + +### CS-Studio + +The client makes use of CS-Studio components such as: + +* Alarm Perspective +* Data Browser (called Log Plotter in the IBEX Client) +* OPI Views +* PV Manager + +... and others to be listed later. + +### PyDev + +IBEX uses PyDev to provide the Python scripting window in the IBEX client. diff --git a/doc/client/design_docs/Configuration-and-Component-Architecture.md b/doc/client/design_docs/Configuration-and-Component-Architecture.md new file mode 100644 index 000000000..75755bdaa --- /dev/null +++ b/doc/client/design_docs/Configuration-and-Component-Architecture.md @@ -0,0 +1,35 @@ +# Configuration and Component Architecture + + +Going forward, we intend to streamline the interactions between the GUI and the Blockserver regarding configurations and components, particularly making a clearer distinction between reading and editing a configuration, as well as reworking the underlying class architecture in the GUI. + +## GUI Class Architecture + +We propose the following changes: +- Currently, both components and configurations are instances of `Configuration`. Create a `Component`-class, which along with `Configuration` inherits from a common abstract superclass. +- Make `EditableConfiguration` inherit from `Configuration`, simply adding setters for the elements contained in the config. Create a new `ConfigEditingModel` that holds the configuration and handles all editing logic. + +UML of the proposed new architecture: + +![UML](config_architecture.png) +## Blockserver Interactions +We distinguish between the following two types of interaction: + +#### Use Case 1: Reading a configuration +For example when switching to a different active configuration, or displaying config information on the dataweb. What we read from the Blockserver: + +One PV containing a configuration in it's entirety, i.e. all elements(iocs, blocks, groups ...) native to the configuration as well as ones imported from components. +There currently are PVs holding this information. No changes required. + +#### Use case 2: Editing a configuration + +What we read/write to the Blockserver: + +- The configuration to edit including only its native elements plus the names of added components. This is the current write behaviour. We propose to add a read PV with the same information to unify the data formats in this interaction. +- Additionally, read the full information of all components on the instrument. This is currently stored in the `:ALL_COMPONENT_DETAILS` PV. + +The new ConfigEditingModel (see above) is then responsible for matching this information up as needed. + +Required changes: +Add "native-only" config PVs to read from when editing configurations. + diff --git a/doc/client/design_docs/IBEX_UI_New_OPI_View.ep b/doc/client/design_docs/IBEX_UI_New_OPI_View.ep new file mode 100644 index 000000000..f7faa3a10 --- /dev/null +++ b/doc/client/design_docs/IBEX_UI_New_OPI_View.ep @@ -0,0 +1,276 @@ + +Untitled Page1454065083065_9201945883#ffffff + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Alarms + + + + + Beam Status + + + + + DAE + + + + + Expt. Details + + + + + IOC Log + + + + + Log Plotter + + + + + Motors + + + + + Scripting + + + + + Synoptic + + + + + Web Links + + + + + +
Device Screens
Eurotherm
Pinhole
Jaws 1
Jaws 2
Jaws 3
+
+ + + + +  Eurotherm]]> + + + + +  Pinhole]]> + + + + +  Jaws 3
]]> + + + + + + + + + + + + + + + + + + + + + + Device Screens + + + + + x + + + + + x + + + + + x + + + + + + + + OK + + + + + Cancel + + + + + + + + text + + + + + + + +
Eurotherm
Rotating Bench
Mk3 Chopper
Pixleman
Sample changer
Beam-stop
+ + + + + + + + + + + + + Add + + + + + Remove + + + + + + + +
Device Screens
Eurotherm
Pinhole
Jaws 1
Jaws 2
Jaws 3
+ + + + + + + + + Edit List + + + + + Add + + + + + Remove + + + + + Open + + + + + + ]]> + + + +

+ \ No newline at end of file diff --git a/doc/client/design_docs/IBEX_UI_New_OPI_View.png b/doc/client/design_docs/IBEX_UI_New_OPI_View.png new file mode 100644 index 000000000..7cdd07f5a Binary files /dev/null and b/doc/client/design_docs/IBEX_UI_New_OPI_View.png differ diff --git a/doc/client/design_docs/Opening-an-OPI-Outside-of-a-Synoptic.md b/doc/client/design_docs/Opening-an-OPI-Outside-of-a-Synoptic.md new file mode 100644 index 000000000..61491633f --- /dev/null +++ b/doc/client/design_docs/Opening-an-OPI-Outside-of-a-Synoptic.md @@ -0,0 +1,93 @@ +# Opening an OPI Outside of a Synoptic + +## Names + +The suggested name for this is 'Device Screens'. We will change the terminology currently used in the synoptic from 'Component' to 'Device' too, for consistency (this avoids ambiguity with a component as part of a configuration). + +Other ideas for names were: +* Component Viewer +* Components +* Devices +* Beam/SE Devices +* Beam/SE Components +* Target View +* Component Targets +* Target View +* OPIs +* Screens + +## GUI Design + +The figure below shows a proposed layout for the GUI. This is designed as a first attempt layout, to be similar to the existing synoptic view and hence straightforward to implement. We need to consider how best to open up multiple OPIs on one screen, but this does not necessarily need to exist in the first iteration. + +![OPI View](IBEX_UI_New_OPI_View.png) + +Note that the 'Edit List' dialogue will look different with the changes for automatically setting defaults and the new way of setting properties. + +## BlockServer Storage + +The BlockServer will be responsible for saving a list of 'Device Screens'. It is proposed that these are stored as a separate XML file in the configuration or component: `NDXXXXX/configurations/my_config/device_screens.xml`. + +The XML structure will be similar to: +```xml + + + Eurotherm 1 + Eurotherm + OPI + + + EURO + EUROTHERM1 + + + + +``` + +A schema for the device screens will be required: + +```xml + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +The device screens will be served up in the same way as the synoptics, JSON over Channel Access. Note the way PVs are named needs to take [#1053](https://github.com/ISISComputingGroup/IBEX/issues/1053) +into account. + +## Synoptic changes + +Changes should be made once this is complete to use the device screens within the synoptics. This should not be done until the device screens editor and BlockServer changes are completed and well tested. When implementing the changes this should be taken into account though, to make sure synoptics can easily make use of a device screen. diff --git a/doc/client/design_docs/client_architecture.graphml b/doc/client/design_docs/client_architecture.graphml new file mode 100644 index 000000000..88fba7f85 --- /dev/null +++ b/doc/client/design_docs/client_architecture.graphml @@ -0,0 +1,735 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + 2 + + + + + + + + + + IBEX Client (Eclipse RCP) + + + + + + + + + + + + + + + + + IBEX Project Code + + + + + + + + + + + + + + + + + Alarm View +(from CS-Studio) + + + + + + + + + + + + + + + + + Log Plotter +(Data Browser +from CS-Studio) + + + + + + + + + + + + + + + + + OPIs +(from CS-Studio) + + + + + + + + + + + + + + + + + Alarm Server + + + + + + + + + + + + + + + + + Log Plotter +(MySQL) + + + + + + + + + + + + + + + + + + + + IOCs +(on NDX machine) + + + + + + + + + + + + + + + + + Block Server + + + + + + + + + + + + + + + + + genie_python +(scripting window +from PyDev) + + + + + + + + + + + + + + + + + PV Manager +(from CS-Studio) + + + + + + + + + + + + + + + + + DatabaseServer + + + + + + + + + + + + + + + + + Exp. Details DB +(MySQL) + + + + + + + + + + + + + + + + + + + + IOC DB +(MySQL) + + + + + + + + + + + + + + + + + + + + Message DB +(MySQL) + + + + + + + + + + + + + + + + + + + + + IOC Log Server +(Avtive MQ) + + + + + + + + + + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + JMS + + + + + + + + + + + + + + + + + + + + JDBC (MySQL) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + + + + + + + + + + + CA/JSON over CA + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + JSON over CA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + JDBC + + + + + + + + + + + + + + + + + + JDBC + + + + + + + + + + + + + + + + + + + JMS + + + + + + + + + <?xml version="1.0" encoding="utf-8"?> +<svg version="1.1" + xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" + x="0px" y="0px" width="41px" height="48px" viewBox="-0.875 -0.887 41 48" enable-background="new -0.875 -0.887 41 48" + xml:space="preserve"> +<defs> +</defs> +<linearGradient id="SVGID_1_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-979.1445" x2="682.0508" y2="-979.1445" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_1_)" d="M19.625,36.763C8.787,36.763,0,34.888,0,32.575v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,34.888,30.464,36.763,19.625,36.763z"/> +<linearGradient id="SVGID_2_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-973.1445" x2="682.0508" y2="-973.1445" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_2_)" d="M19.625,36.763c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.927-18.396,3.927 + c-9.481,0-17.396-1.959-18.396-3.927l-1.229,2C0,34.888,8.787,36.763,19.625,36.763z"/> +<path fill="#3C89C9" d="M19.625,26.468c10.16,0,19.625,2.775,19.625,2.775c-0.375,2.721-5.367,5.438-19.554,5.438 + c-12.125,0-18.467-2.484-19.541-4.918C-0.127,29.125,9.465,26.468,19.625,26.468z"/> +<linearGradient id="SVGID_3_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-965.6948" x2="682.0508" y2="-965.6948" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_3_)" d="M19.625,23.313C8.787,23.313,0,21.438,0,19.125v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,21.438,30.464,23.313,19.625,23.313z"/> +<linearGradient id="SVGID_4_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-959.6948" x2="682.0508" y2="-959.6948" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_4_)" d="M19.625,23.313c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.926-18.396,3.926 + c-9.481,0-17.396-1.959-18.396-3.926l-1.229,2C0,21.438,8.787,23.313,19.625,23.313z"/> +<path fill="#3C89C9" d="M19.476,13.019c10.161,0,19.625,2.775,19.625,2.775c-0.375,2.721-5.367,5.438-19.555,5.438 + c-12.125,0-18.467-2.485-19.541-4.918C-0.277,15.674,9.316,13.019,19.476,13.019z"/> +<linearGradient id="SVGID_5_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-952.4946" x2="682.0508" y2="-952.4946" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_5_)" d="M19.625,10.113C8.787,10.113,0,8.238,0,5.925v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,8.238,30.464,10.113,19.625,10.113z"/> +<linearGradient id="SVGID_6_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-946.4946" x2="682.0508" y2="-946.4946" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_6_)" d="M19.625,10.113c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.926-18.396,3.926 + c-9.481,0-17.396-1.959-18.396-3.926L0,5.925C0,8.238,8.787,10.113,19.625,10.113z"/> +<linearGradient id="SVGID_7_" gradientUnits="userSpaceOnUse" x1="644.0293" y1="-943.4014" x2="680.8223" y2="-943.4014" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<ellipse fill="url(#SVGID_7_)" cx="19.625" cy="3.926" rx="18.396" ry="3.926"/> +<path opacity="0.24" fill="#FFFFFF" enable-background="new " d="M31.04,45.982c0,0-4.354,0.664-7.29,0.781 + c-3.125,0.125-8.952,0-8.952,0l-2.384-10.292l0.044-2.108l-1.251-1.154L9.789,23.024l-0.082-0.119L9.5,20.529l-1.65-1.254 + L5.329,8.793c0,0,4.213,0.903,7.234,1.07s8.375,0.25,8.375,0.25l3,9.875l-0.25,1.313l1.063,2.168l2.312,9.645l-0.521,1.416 + l1.46,1.834L31.04,45.982z"/> +</svg> + + + + diff --git a/doc/client/design_docs/client_architecture.png b/doc/client/design_docs/client_architecture.png new file mode 100644 index 000000000..50816146f Binary files /dev/null and b/doc/client/design_docs/client_architecture.png differ diff --git a/doc/client/design_docs/config_architecture.png b/doc/client/design_docs/config_architecture.png new file mode 100644 index 000000000..fbfa60d0e Binary files /dev/null and b/doc/client/design_docs/config_architecture.png differ diff --git a/doc/client/eclipse/Checkstyle-setup.md b/doc/client/eclipse/Checkstyle-setup.md new file mode 100644 index 000000000..5872bdfc7 --- /dev/null +++ b/doc/client/eclipse/Checkstyle-setup.md @@ -0,0 +1,47 @@ +# Checkstyle setup + +Check style is set up by: + +## Install checkstyle plugin. +You may need to install an older version to be compatible with our checkstyle config in case our dependencies are not up to date. + +1. Help --> Install New Software +1. Add new site (Name: Checkstyle, Location: https://checkstyle.org/eclipse-cs-update-site) +1. Uncheck "Show only latest versions of available software" +1. Uncheck "Hide Items that are already installed" +1. Install the correct version of "Eclipse Checkstyle Plug-in"(at the moment this is v`10.0.0`). If you see another version in the list that is already installed, you need to uninstall it first: Go to `Help > About Eclipse IDE > Installation Details`, search for "checkstyle" in the list of installed software and click `Uninstall` + +## Configure Checkstyle to use external file. + +1. In Eclipse open checkstyle settings via `Window` > `Preferences` > `Checkstyle` +1. If you have an existing "IBEX Checks" entry using an internal configuration, remove it. If it complains about the configuration currently being used in projects, you may have to remove and re-import all projects before you remove this config. +1. Add a new check configuration by clicking `New...` next to the table +1. Set the following + 1. File: External configuration file + 1. Name: IBEX Checks + 1. Location: `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.client.tycho.parent\checkstyle.xml` + 1. `Additional Properties` button + 1. `Add...` button + 1. Set: + - Name: `checkstyle.suppressions.file` + - Value: `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.client.tycho.parent\suppressions.xml` +1. Apply and close: you should now be setup + 1. If you have issues with the highlight colour on a dark theme you can go to: Window -> Preferences, General -> Editors -> Text Editors -> Annotations to change it. + +## To skip checkstyle processing for a method or all elements of a class. + +1. Suppose you run build.bat and there are checkstyle violations. +2. Open the file checkstyle-result.xml in the particular project you have issues, it will show you the error, for example: ` +3. In the example above the error is generated by the checker `JavadocVariableCheck which implies that the variable defined in line#29 doesn't have a javadoc`. +4. It is possible that for this variable javadoc isn't necessary. For examples it is a file with lots of constants identifying magic numbers and strings. It is impractical to provide javadoc for all of them. +5. In such case we might want to suppress the checkstyle inspection for this file. There are two ways of doing it: + + 5.1 - Locate the suppressions.xml file and mention the file name or pattern to skip processing. This is useful when you have a class of files you wish to skip, for examples test files. Different patterns can be separated by '|' as it is a regular expression. + + 5.2 It may be more relevant to only suppress for a specific file or method or variable. In this case we can use the annotation @SuppressWarnings("checkstyle:"). Recall the example above where the error was from the checker `JavadocVariableCheck`. The tag to use in this case is `javadocvariable` - take the checker class name, remove the word Check at the end and replace all capital letters by small hand letters. + +6. Run the build again, the checkstyle violations will be suppressed. + +## Useful links + +https://checkstyle.org/eclipse-cs/#!/custom-config \ No newline at end of file diff --git a/doc/client/eclipse/Common-Eclipse-Issues.md b/doc/client/eclipse/Common-Eclipse-Issues.md new file mode 100644 index 000000000..d89637e61 --- /dev/null +++ b/doc/client/eclipse/Common-Eclipse-Issues.md @@ -0,0 +1,129 @@ +# Common Eclipse issues + +Sometimes the error messages that Eclipse gives are a little opaque, so here are some possible solutions for some of the more common issues. + +## Plugin execution not covered by lifecycle configuration + +Go to window -> preferences -> Maven -> Errors/Warnings. Change "Plugin execution not covered by lifecycle configuration" to ignore in the drop down. + +## The type XXXXXXX cannot be resolved. It is indirectly referenced from required .class files + +### Errors appear for single plugin +Possible solutions (in order of desperation): + +* Check the error message: you may need to add a dependency to the plugin that contains XXXXXXX. For example: if the error message is ```The type org.eclipse.ui.plugin.AbstractUIPlugin cannot be resolved. It is indirectly referenced from required .class files``` then the org.eclipse.ui plugin requires adding.bundle to the required bundle in the manifest file of your plugin. +* Delete the complaining plugin from the workspace and then re-import it. +* Remove JRE System Library from the complaining plug-in's Java Build Path then re-add it. +* This can also occur in similar cases to the **Invalid class hierarchy**. + +### Errors appear across entire project + +Suggests that eclipse has got itself into a funny state. Try `Project > Clean` first, if that does not get rid of the problems, try: +1. Reset the targetplatform: Open file in `uk.ac.stfc.isis.ibex.targetplatform`, reload dependencies if they are in error, then click `Reload Target Platform` top right +2. Synchronize the runnable product: open `ibex.product` in `uk.ac.stfc.isis.ibex.e4.client.product`, click `Synchronize this configuration with the product's defining plug-in.` +3. Confirm the problems have disappeared. (you may need to repeat step 1 & 2 a couple of times) +4. Launch application from `ibex.product` (little green arrow on the top right) and confirm the client starts + +## Invalid class hierarchy + +* Make sure that you don't have any circular dependencies. That is package A imports package B imports package A. +* If you have several layers of derived dependencies, particularly containing CSStudio or eclipse classes, make sure the correct bundles have been imported. Dependencies are not necessarily re-exported from the intermediate layers by default. There are two solutions: + * Add the base bundle to the list of dependencies of your new class via the manifest file + * Go through the classes you derive from. In their bundles go to the dependencies menu and click on the plugin you're deriving from. Click properties and check the box that says "Reexport this dependency". Once the intermediate layers all reexport their dependencies then it should become available to your class. + + +## Product XXXXXXXXXXX.product could not be found + +Typically is followed by a lot of errors relating to bundle resolution, for example: + +``` + !MESSAGE Product uk.ac.stfc.isis.ibex.product.product could not be found. + + !ENTRY org.eclipse.osgi 2 0 2015-09-16 15:25:45.343 + !MESSAGE One or more bundles are not resolved because the following root constraints are not resolved: + !SUBENTRY 1 org.eclipse.osgi 2 0 2015-09-16 15:25:45.343 + !MESSAGE Bundle reference:file:/C:/CodeWorkspaces/GitHub/ibex_gui/base/uk.ac.stfc.isis.ibex.ui.perspectives/ was not resolved. + !SUBENTRY 2 uk.ac.stfc.isis.ibex.ui.perspectives 2 0 2015-09-16 15:25:45.343 + !MESSAGE Missing required bundle uk.ac.stfc.isis.ibex.ui.statusbar_1.0.0. + !SUBENTRY 1 org.eclipse.osgi 2 0 2015-09-16 15:25:45.343 + !MESSAGE Bundle reference:file:/C:/CodeWorkspaces/GitHub/ibex_gui/base/uk.ac.stfc.isis.ibex.product/ was not resolved. + !SUBENTRY 2 uk.ac.stfc.isis.ibex.product 2 0 2015-09-16 15:25:45.343 + !MESSAGE Missing required bundle uk.ac.stfc.isis.ibex.ui.statusbar_0.0.0. + !SUBENTRY 1 org.eclipse.osgi 2 0 2015-09-16 15:25:45.343 + !MESSAGE Bundle reference:file:/C:/CodeWorkspaces/GitHub/ibex_gui/base/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/plugins/uk.ac.gda.common_1.2.0.v20140919-1144.jar was not resolved. + !SUBENTRY 2 uk.ac.gda.common 2 0 2015-09-16 15:25:45.344 + !MESSAGE Missing native code match lib/linux-x86/libgda_common.so; processor=x86; osname=linux, lib/linux-x86_64/libgda_common.so; processor=x86_64; osname=linux. + + !ENTRY org.eclipse.osgi 2 0 2015-09-16 15:25:45.389 + !MESSAGE The following is a complete list of bundles which are not resolved, see the prior log entry for the root cause if it exists: + !SUBENTRY 1 org.eclipse.osgi 2 0 2015-09-16 15:25:45.389 + !MESSAGE Bundle uk.ac.stfc.isis.ibex.ui.weblinks_1.0.0.qualifier [460] was not resolved. + !SUBENTRY 2 uk.ac.stfc.isis.ibex.ui.weblinks 2 0 2015-09-16 15:25:45.393 + !MESSAGE Missing required bundle uk.ac.stfc.isis.ibex.ui.perspectives_1.0.0. + !SUBENTRY 1 org.eclipse.osgi 2 0 2015-09-16 15:25:45.393 + ... + + ... + !MESSAGE Bundle uk.ac.stfc.isis.ibex.ui.perspectives_1.0.0.qualifier [465] was not resolved. + !SUBENTRY 2 uk.ac.stfc.isis.ibex.ui.perspectives 2 0 2015-09-16 15:25:45.393 + !MESSAGE Missing required bundle uk.ac.stfc.isis.ibex.ui.statusbar_1.0.0. +``` + +From this example it seems to be related to uk.ac.stfc.isis.ibex.ui.statusbar as that appears multiple times. + +Possible solution: + +* Check the offending plug-in has been added to one of the feature projects as a plug-in + +## New plugin is not available (and possibly crashing existing plugins) + +This might be difficult to find as an error, and it may not exist for other developers - even on the same branch! +Adding the plugin directly to your configuration solves this. + +## Plugins compile fine but dependencies cause errors at runtime + +If you can run the application fine but are seeing a `java.lang.NoClassDefFoundError` at runtime, try re-setting the target platform, synchronizing `client.product` (under "testing" in the product overview), and doing a clean. + +## Menu items are missing + +This is usually because a dependency is missing. See "I really cannot work out why the GUI won't start!" below. + +## No application ID + +Starting the GUI via Eclipse just doesn't seem to work and I cannot see why! + +One useful thing to try is to use the run configuration to check all the correct plug-ins are included: + +* Right-click on ibex.product in uk.ac.stfc.isis.ibex.client.product and select Run As->Run Configurations... +* Select ibex.product under Eclipse Application +* From the tabs select "Plug-ins" and click the "Validate Plug-ins" button +* If it displays errors because plug-ins are missing: + + * Close the dialogs and add the missing plug-ins to the feature.xml file in uk.ac.stfc.isis.ibex.feature.base or one of the other features + * Repeat this whole process again from the beginning + +* If there are no errors then click the "Run" button to see if it works + +Sometimes eclipse will tell you that you have errors in various projects when you open it. The following operations (may) help. +- Refresh, clean and build all projects. Select all projects, press F5 to refresh, then go to `Project -> Clean` to clean all projects. +- Go to `Run -> Run Configurations -> Plugins` and press "Add required plugins". You can now validate/apply your choice. +- In `uk.ac.stfc.isis.ibex.targetplatform`, open `uk.ac.stfc.isis.ibex.targetplatform.target` and click "set as target platform". +- If all else fails, delete all the projects from eclipse's workspace and reimport them. + +## The GUI starts but looks strange and some items (e.g. menu bar, perspective buttons) are missing + +This can happen if the wrong ".product" file is run and not all plugins are defined in the application. + +* Make sure you *only* run ibex.product in uk.ac.stfc.isis.ibex.**client**.product *not* any other ".product" file in the workspace e.g. uk.ac.stfc.isis.ibex.product + +## I can't see changes in a new branch + +Sometimes when you check out a new branch in the ibex GUI repository, you won't see any changes in the GUI. You can fix this by following the steps below: + +1. In Eclipse, remove all the projects from the workspace and close Eclipse. +1. Then, while checked out to the branch you want to see the changes on, perform a `git clean -fdx && git reset HEAD --hard` to reset the git repository. This will remove any changes you have made to the branch. +1. Restart Eclipse and repeat the steps to get the GUI working from [Building the GUI](../getting_started/Building-the-GUI). + +## Build fails with `[ERROR] Failed to resolve target definition ... targetplatform.target: Could not find "org.eclipse.e4.tools.spies.feature.feature.group..." in the repositories of the current location` + +This probably means that our pinned target has gone out of date and needs updating. Open the Eclipse IDE and click on `org.eclipse.tools/latest` and click `update` this should fail and show `Unable to locate installable unit ...`. To fix this remove the plugin and then add it from the `software site `, work with `org.eclipse.e4.core.tools.update - http://download.eclipse.org/e4/snapshots/org.eclipse.e4.tools/latest` then you add `Eclipse E4 - All Spies`. diff --git a/doc/client/eclipse/Common-Eclipse-Tasks.md b/doc/client/eclipse/Common-Eclipse-Tasks.md new file mode 100644 index 000000000..aeb62df24 --- /dev/null +++ b/doc/client/eclipse/Common-Eclipse-Tasks.md @@ -0,0 +1,302 @@ +# Common Eclipse tasks + +## Add A New Plugin + +1. In Eclipse IDE, File Menu > New > Plugin Project. +1. Give the Project a name, e.g. ``uk.ac.stfc.isis.ibex.foo``. If it is a UI plugin, it should have a "ui" in the top part of the name e.g. ``uk.ac.stfc.isis.ibex.ui.foo`` +1. For location, select the base source code folder, e.g., ``C:\Instrument\Dev\Client\base\uk.ac.stfc.isis.ibex.foo``, rather than the default option (which will probably be the workspace). +1. Click next. +1. Make sure 'Generate an activator, a Java...' is checked. +1. If this is to be a UI plugin, check 'This plug-in will make contributions to the UI'. This will cause the Activator to extend ``AbstractUIPlugin`` rather than ``BundleActivator``. Plugins that contribute to the UI in any way such as by adding a preference page or menu item, need to be UI plugins +1. Click finish. +1. In the plugin ``uk.ac.stfc.isis.ibex.feature.base``, open ``feature.xml`` and go to the 'Plug-ins' tab. Add your new plugin to the plug-ins list. + +1. Add a ``pom.xml file`` to the plugin so that it can be built with maven. This should be the same as the pom file in every other project; the only thing you'll need to change is the plugin's name (the ``artifactId``). The contents should be as below. Other sections of the pom follow will be inherited from ``org.csstudio.isis.tycho.parent`` and so don't need to be explicitly included: + ```xml + + 4.0.0 + uk.ac.stfc.isis.ibex.foo + eclipse-plugin + + CSS_ISIS + 1.0.0-SNAPSHOT + uk.ac.stfc.isis.ibex.client.tycho.parent + ../uk.ac.stfc.isis.ibex.client.tycho.parent + + 1.0.0-SNAPSHOT + + ``` +1. In the plugin ``uk.ac.stfc.isis.ibex.client.tycho.parent``, add your new plugin to the list of modules (you may have to do this manually in the XML view). + + +## Easy Plugin Access + +One option to enable easy access to your plug-in object is to use the singleton pattern. Declare a static variable in your plug-in class for the singleton. Store the first (and only) instance of the plug-in class in the singleton when it is created. Then access the singleton when needed through a static ``getDefault()`` method. Your Activator class should look something like this: +```java +public class Activator extends AbstractUIPlugin +{ + public static final String PLUGIN_ID = "org.csstudio.isis.ui.logger"; + + private static Activator plugin; + + public Activator() { + } + + public static Activator getDefault() { + return plugin; + } + + public void start(BundleContext context) throws Exception { + super.start(context); + plugin = this; + } + + public void stop(BundleContext context) throws Exception { + plugin = null; + super.stop(context); + } +} +``` + +## Force Plugin To Start At Eclipse Startup + +Normally plugins are loaded lazily, i.e., the first time they are actually needed. It can be useful to force a plugin to start when the program launches. + +In your plugin: + +1. Make a class called, e.g., ``org.csstudio.isis.foo.FooStartup`` that extends the Eclipse interface, ``IStartup``. Override the public method ``earlyStartup()``, with blank implementation. +1. Open ``META-INF`` > ``MANIFEST.MF`` > ``Extensions`` tab. Make sure "Show only extension points from required plug-ins" is unchecked. +1. Click ``Add`` and type: ``org.eclipse.ui.startup``. +1. In the Extension element details, select the startup, ``FooStartup``, class you created earlier. + + +## Running A Background Job + +In Eclipse, Jobs are units of runnable work that can be scheduled to be run with the job manager. Once a job has completed, it can be scheduled to run again (jobs are reusable). You might want to use a job to prevent UI freeze, i.e., if clicking a button performs a long operation or calculation, you still want the UI to respond while performing the operation. You might also use a job if a task needs to run continuously in the background for the lifetime of the program (as is the case for the JMS handler thread that receives messages from the IOC log server). + +A job can be prepared as follows: +```java +Job fooJob = new Job("My Foo Job") +{ + @Override + protected IStatus run(IProgressMonitor monitor) + { + fooCalculator.performLongTask(); + return Status.OK_STATUS; + } +}; +``` +The string passed to the constructor will be the name of the thread that you will see if you are debugging the application. + +The job can be started with: +```java +fooJob.schedule(); +``` +More details can be found in the [Vogella tutorial](http://www.vogella.com/tutorials/EclipseJobs/article.html). + + +## Updating The UI (Without Data Binding) + +Where possible, you should use the Eclipse data binding framework to update UI elements (see [Vogella databinding tutorial](http://www.vogella.com/tutorials/EclipseDataBinding/article.html); however in some cases this is not convenient or possible. + +The display of UI elements is not handled in the main execution thread but in a separate UI thread. Consequently, if you attempt to alter the display of any UI element from the main thread, you will get an Invalid Thread Access exception. We can overcome this limitation by calling the ``Display.asyncExec()``, which passes a runnable command to the UI thread and asks for it to be run at the next available opportunity. + +As an example, if you had a UI class, FooDisplay, which had a method, ``setLabelText()``, you might implement it as follows: +```java +public class FooDisplay extends Canvas +{ + private Label fooLabel; + + private void setLabelText(final String text) + { + Display.getDefault().asyncExec(new Runnable() + { + @Override + public void run() { + fooLabel.setText(text); + } + }); + } +} +``` + +## Add A New Perspective + +Instructions on adding a new perspective can be found [here](../coding/Adding-a-Button-to-the-Perspective-Switcher). + +Add A Preference Page +--------------------- + +Many plugins may have options that you want the user of the client to be able to configure. The simplest way to achieve this is to have the plugin contribute a preference page to the client's preference window. + +1. Make sure your plugin contributes to the UI, i.e., that its activator class extends ``AbstractUIPlugin``, not ``BundleActivator``, and that it has a static ``getDefault()`` method. +1. In your plugin, add a new preference package, called e.g., ``org.csstudio.isis.foo.preferences``. + +1. Create a new class called e.g., ``FooPreferenceConstants``, which will store tags and default values for each preference in your plugin. In the below example, we specify tags and defaults for a String preference called name and a integer preference called count. The tags are used internally by eclipse to refer to the preference and will not be displayed to the user: + ```java + public class FooPreferenceConstants + { + public static final String TAG_NAME = "fooName"; + public static final String TAG_COUNT = "fooCount"; + + public static final String DEFAULT_NAME = "This is my foo name!"; + public static final int DEFAULT_COUNT = 5; + } + ``` +1. Create a new class called e.g., ``FooPreferenceInitializer``, that extends ``AbstractPreferenceInitializer``, which will set the default values of each preference: + ```java + public class FooPreferenceInitializer + extends AbstractPreferenceInitializer { + + public void initializeDefaultPreferences() { + IPreferenceStore store + = Activator.getDefault().getPreferenceStore(); + + store.setDefault(FooPreferenceConstants.TAG_NAME, + FooPreferenceConstants.DEFAULT_NAME); + store.setDefault(FooPreferenceConstants.TAG_COUNT, + FooPreferenceConstants.DEFAULT_COUNT); + } + } + ``` +1. Add a new preference page class called, e.g., ``FooPreferencePage``, and have it extend the Eclipse class ``FieldEditorPreferencePage`` and implement the interface ``IWorkbenchPreferencePage``. +1. Add a constructor and implementations of the functions ``creatFieldEditors()`` and ``init()``: + ```java + public class FooPreferencePage extends FieldEditorPreferencePage + implements IWorkbenchPreferencePage + { + public FooPreferencePage() { + super(GRID); + setPreferenceStore(Activator.getDefault().getPreferenceStore()); + setDescription("Settings for Foo."); + } + + @Override + public void createFieldEditors() { + addField(new StringFieldEditor(FooPreferenceConstants.TAG_NAME, + "Foo Name", getFieldEditorParent())); + addField(new StringFieldEditor(FooPreferenceConstants.TAG_COUNT, + "Foo Count", getFieldEditorParent())); + } + + @Override + public void init(IWorkbench workbench) { + } + } + ``` +1. Open the plugins ``plugin.xml`` and navigate to the Extensions tab. + + * Add the extension ``org.eclipse.core.runtime.preferences``. + * To this, add a new ``Initializer`` and set its class as ``FooPreferenceInitializer``. + * Add the extension ``org.eclipse.ui.preferencePages``. + * To this, add a new ``Page`` and set its class to ``FooPreferencePage``. You can also set the ``name``, which will displayed in the UI. + +The ``plugin.xml`` should look like the following: +```xml + + + + + + + + + + + + +``` +When you start the client, the Foo preference page should now appear in the Eclipse Preferences window. Changes made by the user will be persisted to file automatically by the Eclipse framework and will be reloaded next time the user starts the client. + +For more details see the [Vogella preferences tutorial](http://www.vogella.com/tutorials/EclipsePreferences/article.html). + + +## Add A Menu + +Sometimes it may be necessary to add a new menu item to the menu bar in the Eclipse client so you can open some sort of dialog window or perform some other action. If you want the menu item to open a dialog, make sure you already have a ``Dialog`` class prepared in your plugin. + +1. Create a class that extends ``org.eclipse.core.commands.IHandler``; call it something like ``FooHandler``. Add the unimplemented methods. +1. Make ``isEnabled()`` and ``isHandled()`` return ``true``. +1. Make ``execute()`` instantiate and open your dialog (or perform whatever other action you have in mind): + ```java + public class FooHandler implements IHandler + { + @Override + public void addHandlerListener(IHandlerListener handlerListener) { + } + + @Override + public void dispose() { + } + + @Override + public Object execute(ExecutionEvent event) throws ExecutionException { + Shell shell = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell(); + FooDialog dialog = new FooDialog(shell); + dialog.open(); + return null; + } + + @Override + public boolean isEnabled() { + return true; + } + + @Override + public boolean isHandled() { + return true; + } + + @Override + public void removeHandlerListener(IHandlerListener handlerListener) { + } + } + ``` +1. Open the plugins ``plugin.xml`` and navigate to the Extensions tab. + + * Add the extension ``org.eclipse.ui.commands``. + * To this, add a new ``command``. Set the ``id``, give it a ``name``, and set the ``defaultHandler`` to the ``FooHandler`` class that you made earlier. + * Add the extension ``org.eclipse.ui.menus``. + * To this, add a new ``menuContribution``; set the ``locationURI`` to ``menu:org.eclipse.ui.main.menu``. + * To this, add a new ``menu``. Give it an ``id`` and ``label`` (this will be displayed on the menu bar). + * To the ``menu``, add a new ``command``. Set the ``commandId`` to be the ``id of the ``command`` you created earlier and give it a label that will be displayed in the menu on the UI; + +The ``plugin.xml`` should now resemble: +```xml + + + + + + + + + + + + + + + + +``` +The menu should now be visible in the client UI. For more details see the [Vogella menus tutorial](http://www.vogella.com/tutorials/EclipseCommands/article.html). + +## Add A Third Party Library To A Plugin + +To do \ No newline at end of file diff --git a/doc/client/eclipse/Creating-the-IBEX-Developer-Version-of-Eclipse.md b/doc/client/eclipse/Creating-the-IBEX-Developer-Version-of-Eclipse.md new file mode 100644 index 000000000..a11029cf4 --- /dev/null +++ b/doc/client/eclipse/Creating-the-IBEX-Developer-Version-of-Eclipse.md @@ -0,0 +1,3 @@ +# Creating the IBEX developer version of Eclipse + +The recommended IBEX developer version of Eclipse can be found at `\\isis\inst$\Kits$\CompGroup\ICP\Developer Tools`. diff --git a/doc/client/eclipse/Dictionary-setup.md b/doc/client/eclipse/Dictionary-setup.md new file mode 100644 index 000000000..ad471db59 --- /dev/null +++ b/doc/client/eclipse/Dictionary-setup.md @@ -0,0 +1,9 @@ +# Dictionary setup + +Spelling error highlights fix + +1. In Eclipse open `Window > Preferences > General > Editors > Text Editors > Spelling` +1. Change the "User defined dictionary" to `/code_dictionary.txt` + + + diff --git a/doc/client/eclipse/Eclipse-RCP.md b/doc/client/eclipse/Eclipse-RCP.md new file mode 100644 index 000000000..ac7cc60cb --- /dev/null +++ b/doc/client/eclipse/Eclipse-RCP.md @@ -0,0 +1,7 @@ +# Eclipse RCP + +We have two books which should serve as a reasonable introduction to the Eclipse RCP platform: 'Eclipse Rich Client Platform' by McAffer, Lemieux and Aniszczyk, and 'Eclipse 4 Application Development' by Vogel. Most of the contents of the latter book can be found on the Vogella website along with a large number of additional Eclipse RCP and JFace tutorials. + +### E4 and compatibility views (e.g. `org.eclipse.ui.console`) + +Although we run an E4 product, we are able to load E3-style plugins via the eclipse compatibility layer. However, [any UI element referred to via the compatibility layer must be defined as a shared element in the client E4XMI, not defined in a snippet](https://www.eclipse.org/forums/index.php/t/358552/) as this does not work correctly with certain UI actions - for example, resetting layouts. \ No newline at end of file diff --git a/doc/client/eclipse/Eclipse-logging.md b/doc/client/eclipse/Eclipse-logging.md new file mode 100644 index 000000000..9df35c26b --- /dev/null +++ b/doc/client/eclipse/Eclipse-logging.md @@ -0,0 +1,71 @@ +# Eclipse logging + +## Log4j +We are using the [log4j2](http://logging.apache.org/log4j/2.x/manual/index.html) framework for our logging. + + +## End Users +Recent log messages can be viewed by clicking 'Console > Console Log'. The messages shown are those that are currently in the rolling log file (see below). + +![Log UI](log_ui.png) + +### Configuration +In the GUI, the user can access logging settings from the 'ISIS Logger' page in the preferences window. + +![Log Pref](log_pref.png) + +The log directory is the root directory where all log files will be stored. + +New log messages are written to the 'rolling log file' which is stored in the log directory. Older log messages are moved into dated archive log files with file names which follow the pattern given by the 'log4j archive pattern', e.g., + +``` +%d{yyyy-MM}/isis-%d{yyyy-MM-dd}--%i.log +``` + +This means log files will be sorted into per-month folders named as 'yyyy-MM', e.g., '2014-08', and individual files will be named according to the date: 'isis-yyyy-MM-dd--i.log'. 'i' is the sequence number, the number of log files generated so far on that day. You can also configure the maximum size of an individual log file and the maximum number of files that will be generated per day. In the event that a very large number of log messages is generated on a particular day and the space in the available log files is exceeded, older messages from that day will be dropped to make room for new ones. For more information on how log4j2 handles file roll-over see [the log4j2 manual](http://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender). + +The preference screen also allows you to configure the format of the actual log messages that are displayed in the log file through the 'log4j message pattern' setting. + +``` +%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n +``` + +For more information, see the [pattern layout](http://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout) section of the log4j2 manual. + + +### Logging levels +In the preferences page, you may configure the logging level. Selecting a particular level will cause all log messages of that level and above that are generated in the program to be logged to file. In order of increasing severity, the levels are: + +``` +TRACE < DEBUG < INFO < WARN < ERROR < FATAL +``` + +## Using the Logger in Code +To use the logger in your plugin/class, first add the packages 'org.csstudio.isis.logger' and 'org.apache.logging.log4j' to the plugin's imported packages list. Next have the class get a static instance of the logger. + +``` +private static Logger logger = IsisLog.getLogger(MyClass.class); +``` + +To do this you will need to import two classes: + +``` +import org.csstudio.isis.logger.IsisLog; +import org.apache.logging.log4j.Logger; +``` + +The argument to the static `getLogger()` method can be either a Class type or a string. If you wanted all the messages from a given plugin to be listed as coming from the same source, you could supply the name of the plugins activator class (for example) to the `getLogger()` call in every class in the plugin. + +Finally to actually log a message, simply call the appropriate method on your 'Logger' object: + +``` +logger.warn("A minor issue has occurred..."); +``` + +The logger has a separate method for each level of logging listed above. + +## Turning on Extra debug logging + +Sometimes we may want the ability to log very fine events, however leaving this turned on permanently could be an issue with disk space. + +The solution is to use `LoggerUtils.logIfExtraDebug(logger, message)`. This only logs if the environment variable `IBEX_GUI_EXTRA_DEBUG` was set to `1` *at the time the GUI was started*. diff --git a/doc/client/eclipse/Eclipse-preferences.md b/doc/client/eclipse/Eclipse-preferences.md new file mode 100644 index 000000000..c5ed13150 --- /dev/null +++ b/doc/client/eclipse/Eclipse-preferences.md @@ -0,0 +1,17 @@ +# Eclipse preferences + +There are two sorts of settings. + +`System.getProperty("x")` will read a setting supplied on the command line as `-Dx=value`. To set in Eclipse go to "Run/Run Configurations..." and add to the "VM arguments" on the Arguments tab + +The second sort are Eclipse preferences. On the plugin's extensions tab under "org.eclipse.core.runtime.products/???? (product)" there should be a preferenceCustomization proper whose value is the name of the preferences file. In that file set a property like: + +``` +org.csstudio.isis.product/prefix = NDW1298:sjb99183: +``` + +To access it from org.csstudio.isis.product: + +``` +prefix = DefaultScope.INSTANCE.getNode(Activator.PLUGIN_ID).get("prefix", "no found"); +``` diff --git a/doc/client/eclipse/log_pref.png b/doc/client/eclipse/log_pref.png new file mode 100644 index 000000000..30771ac19 Binary files /dev/null and b/doc/client/eclipse/log_pref.png differ diff --git a/doc/client/eclipse/log_ui.png b/doc/client/eclipse/log_ui.png new file mode 100644 index 000000000..32346e8d3 Binary files /dev/null and b/doc/client/eclipse/log_ui.png differ diff --git a/doc/client/getting_started/Building-the-GUI.md b/doc/client/getting_started/Building-the-GUI.md new file mode 100644 index 000000000..812d4c90b --- /dev/null +++ b/doc/client/getting_started/Building-the-GUI.md @@ -0,0 +1,94 @@ +# Building the GUI + +Before building and running the GUI please make sure you have followed the steps from here: [First time installing and building (Windows)](/overview/First-Time-Build). In particular make sure Git and genie_python are installed. + +## Eclipse RCP resources + +We have a book which should serve as a reasonable introduction to the Eclipse RCP platform: 'Eclipse Rich Client Platform' by McAffer, Lemieux and Aniszczyk. There is also a basic introduction at http://www.vogella.com/tutorials/EclipseRCP/article.html. + +## Checking out the GUI + +Create a directory for where you want your IBEX GUI to reside (e.g. `C:\Instrument\Dev`). From an appropriate Git console (e.g. Git Bash) navigate to your directory and run: + +`git clone https://github.com/ISISComputingGroup/ibex_gui.git` + +## Building via Eclipse ## + +### Eclipse + +You must use a version of eclipse >= 2019-06, earlier versions will fail to build the client. An appropriate version is available at `\\isis\inst$\Kits$\CompGroup\ICP\Developer Tools`, simply unzip the the latest version folder to your chosen location for Eclipse (e.g. `C:\Tools\eclipse`) and use the provided workspace. + +You can choose to download Eclipse directly from [the eclipse website](http://www.eclipse.org/downloads/packages/), choose the package for "RCP and RAP developers". If you choose to download eclipse directly from the eclipse website, and you get errors about failing to resolve `jython` manifests, you will need to add `-Djdk.util.zip.disableZip64ExtraFieldValidation=true` to `eclipse.ini` (located right next to the eclipse.exe executable). + +### Building + +These are the steps needed to run the GUI via Eclipse: + +1. First, create a new workspace (example name: ibex_workspace_E4). +1. Then `git checkout master` from where the IBEX code was cloned to (if it's a fresh clone, it should already be on this branch). +1. Install the [pre-commit hook](https://github.com/ISISComputingGroup/IBEX/issues/4786). To install this hook you must run `install_pre_commit_hook.bat` from the `\build` subdirectory. +1. Start Eclipse IDE and select the workspace and use "Browse" to create and select the new workspace folder. +1. From the menu bar choose File->Import->General->Existing Projects into Workspace. Choose "Select root directory" and browse to `\base`. You should now see a list of plugins to import with names like `uk.ac.stfc.isis.ibex.*`. Eclipse should automatically select everything so you just need to click "Finish" to add them to the project. If a "Marketplace solutions available" dialogue appears, click cancel as these will be obtained later. +1. If there is a "Welcome" tab open in Eclipse, close it. From the "Project Explorer" tab on the left, expand the target platform folder (labelled as ``uk.ac.stfc.isis.ibex.targetplatform``), double click on the target file and choose "Set as Active Target Platform". This may take some time as parts of CS-Studio and DAWN are downloaded. It may also be required to update the Locations in use should some packages appear to be missing. +1. Then select Project->Clean from the menu bar. +1. To run the application from within Eclipse: open "ibex.product" from the ``uk.ac.stfc.isis.ibex.e4.client.product`` folder, select "Launch an Eclipse application". +1. Next, From the menu bar choose Run->Run configurations and select "ibex.product" from the left hand list under "Eclipse Application". +1. In the "Main" tab in **Run configurations** tick the "Clear" tick box and untick the "Ask for confirmation before clearing". In the "Configuration" tab tick the "Clear the configuration area before launching" tick box. Click "Apply" and select "Run". + +Note: If there is a `ava.lang.UnsupportedClassVersionError` error then go to Window -> Preferences -> Java -> Compiler -> Set compliance level to 21 + +IBEX should now build but there will probably be some errors. You can clear them following the procedure below. + +**Important Notes:** +* you will need JDK 21 installed to launch the IBEX GUI successfully. +* you should also [set up the checkstyle](../eclipse/Checkstyle-setup), do it sooner than later as it may prevent potential errors + +### If you see a “Plugin execution not covered by lifecycle configuration” error +1. From the menu bar choose: Window->Preferences +1. Expand Maven and choose Errors/Warnings +1. Set "Plugin execution not covered by lifecycle configuration" to Warning or Ignore + +### Configuring Eclipse to show current Git repository and branch in the Package Explorer + +e.g. `uk.ac.stfc.isis.ibex.targetplatform [ibex_gui_e4 master]` + +Not for new starters: this should already have been done by eclipse so unless you do not see `[ibex_gui_e4 master]` next to the name of a plugin/package you do not need to do any of the following. +1. Select all plugins (ctrl-a) +1. Right-click on any one plugin and select `Team -> Share Project` +1. Click `Finish` +1. The repository and branch names should now be displayed after each plugin as above + + +## Building via Maven ## + +1. Ensure your maven version is >= 3.6.0, excluding 3.6.1 as that version has a bug +1. Double check that your maven is the correct version by running `mvn -v` in a new command window. Older versions will give you very hard to diagnose build errors +1. From the command line, navigate to the `.\build\` directory in the IBEX code (one directory under the root, which should be `ibex_gui\`). +1. Run `build.bat` +1. Wait for a few minutes while it builds +1. After the build finishes it can be found in `.\built_client\` + +## Troubleshooting ## + +If the GUI loads up but items are the wrong size, you may need to change your display settings. The exact settings that you need may vary from computer to computer. This is a common issue on Windows 10 machines due to the OS' scaling setting for text, icons etc. that is meant to ensure they do not look too small on high screen resolutions. + +Generally a resolution of 1920 x 1080 with a scaling factor of 100% should look correct on standard screens. You can increase both of those settings slightly if you feel like the display elements look uncomfortably small. + +Errors can occur if the wrong version of Java is installed for your OS. For example, the `x86` version can differ from the `x64` version and can cause issues when building the GUI. + +## Eclipse troubleshooting ## + +Sometimes eclipse will tell you that you have errors when you open it. The following operations (may) help. +- Refresh, clean and build all projects. Select all projects, press F5 to refresh, then go to `Project -> Clean` to clean all projects. +- Go to `Run -> Run Configurations -> Plugins` and press "Add required plugins". You can now validate/apply your choice. +- In `uk.ac.stfc.isis.ibex.targetplatform`, open `uk.ac.stfc.isis.ibex.targetplatform.target` and click "set as target platform". +- If you have done all these steps and it still doesn't work, there is more troubleshooting information [here](../eclipse/Common-Eclipse-Issues). +- If all else fails, delete all the projects from eclipse's workspace and reimport them. + +Eclipse can automatically set the Java standard for some projects to `1.8`. If you are seeing errors such as `var cannot be assigned to a type` on certain projects, navigate to the project in the explorer, then right-click and choose Properties->Java Compiler->Configure Workspace Settings, and then set the required Java standard to `11`. + +If you have a lot of errors make sure that `-Djdk.util.zip.disableZip64ExtraFieldValidation=true` is included in `eclipse.ini` (located right next to the eclipse.exe executable). + +## Further Troubleshooting ## + +Additional support can be found [here](../GUI-Troubleshooting). diff --git a/doc/client/getting_started/GUI-Development-Workflow.md b/doc/client/getting_started/GUI-Development-Workflow.md new file mode 100644 index 000000000..cd436b51f --- /dev/null +++ b/doc/client/getting_started/GUI-Development-Workflow.md @@ -0,0 +1,348 @@ +# Development workflow + +## Quick overview of workflow + +1. **Create a branch for the ticket (based on master) via GitHub (developer)** + * Use the git issue number for the branch name followed by a brief description, e.g. Ticket780_SelectNewTarget +1. **Clone the branch locally (developer)** +1. **Modify the code (developer)** + * Update any copyright notices to the current year to read ``Copyright (C) - ...`` +1. **Push back the changes (developer)** + * Make sure you have merged in any changes to master that have occurred since your ticket was created - makes reviewer's life easier + * Please follow commenting guidelines below +1. **Create a pull request (developer)** + * Reference the git issue in the comments, for example: ```ISISComputingGroup/IBEX/issues/1016``` + * Assign the correct milestone + * Add a record of the changes to [Release Notes](https://github.com/ISISComputingGroup/IBEX/wiki/ReleaseNotes_Dev) under "Changes in software but still on a branch" +1. **Review (reviewer)** + * Assign the pull request to yourself to indicate you are reviewing it and label`under review` + * Check that the pull request has been built by Jenkins with no errors (if there is a Jenkins build) + * Check that the ticket fixes the associated issue + * If it is okay then move to the next step, otherwise inform the developer of the problem and add a comment to the pull request +1. **Merge the changes (reviewer or developer)** + * If it is a simple merge the reviewer can do it, otherwise ask the developer to do it + * Move the relevant entry in the [Release Notes](https://github.com/ISISComputingGroup/IBEX/wiki/ReleaseNotes_Dev) from "Changes in software but still on a branch" to "Changes merged into master" +1. **Delete the branch (reviewer or developer)** +1. **Close the ticket in Trac (reviewer or developer)** + +Notes: + +* You don't have to create branches on GitHub first, often it is more convenient to create them locally first instead (which you must do if you intend to rebase) +* Deleting a branch on github does not delete it from you local repository clone + +## Cloning the repository (first time only) + +* From the repository page copy the "HTTPS clone URL": + +![GitHub](clone.png) + +* Open Git Bash or Git Shell from the Windows Start Menu + +* In the console enter the following (paste the address from the previous step) to create a local copy of the repository: + +``` +user@MACHINE /c/Instrument/Dev/ (master) +$ git clone https://github.com/ISISComputingGroup/ibex_gui.git +``` + +* Now enter the ibex_gui directory: +``` +user@MACHINE /c/Instrument/Dev/ibex_gui/base (master) +$ cd ibex_gui +``` +* The command line shows in brackets which branch you are on - probably master at this time + +* Use the git branch command to list the available branches: + +``` +user@MACHINE /c/Instrument/Dev/ibex_gui/base (master) +$ git branch -a +* master + remotes/origin/HEAD -> origin/master + remotes/origin/master +``` + +* In the console enter `git checkout master` to change to a working copy of the E4 (latest) revision of the GUI + +``` +user@MACHINE /c/Instrument/Dev/ibex_gui/base (master) +$ git checkout master +Checking out files: 100% (994/994), done. +Switched to a new branch 'master' +Branch 'master' set up to track remote branch 'master' from 'origin'. +``` + +* The branch name in brackets after the working directory should change to `master`: +``` +user@MACHINE /c/Instrument/Dev/ibex_gui/base (master) +$ +``` +## Creating a branch + +* Go to the IBEX GUI GitHub repository https://github.com/ISISComputingGroup/ibex_gui +* The page should look something like this: + +![GitHub](start.png) + +* Click the branch drop-down button which will list all the current branches. Type a name in the box and then click "Create branch" to create a new branch: + +![GitHub](create_branch.png) + +Note: Use the Trac ticket number for the branch name followed by a brief description, e.g. Ticket780_SelectNewTarget + +* The page should switch to the new branch: + +![GitHub](new_branch.png) + +## Cloning the branch locally + +It is necessary to clone the new branch locally if you don't already have it: + +* First get an updated list of remote branches: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git remote update +Fetching origin +From https://github.com/ISISComputingGroup/ibex_gui + * [new branch] Ticket768 -> origin/Ticket768 +``` +* The new branch should appear in the branch list: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git branch -a +* master + remotes/origin/HEAD -> origin/master + remotes/origin/Ticket768 + remotes/origin/master +``` +* Switch to the new branch created earlier: +``` +$ git checkout Ticket768 +``` +* Now you can start coding! + +## Pushing the changes back + +Now the code changes have been made, it is time to push the changes back to the repository on GitHub: + +* From the command line we can see what changes we have made using the git status command like so: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git status -s + M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/BannerComposite.java + M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +?? .metadata/ +?? uk.ac.stfc.isis.ibex.client.tycho.parent/workspace/ +?? workspace/ +?? ../runtime-ibex.product/ +?? ../surefire-reports/ +``` + +We can see that I have modified two .java files (indicated by the M) and there are a few files not under source control (indicated by ??). +I am not interested in the files not under source control as they are default files created by Eclipse. Notice that there is a space before the M on the modified files. + +* To be able to commit the changes back they have to be staged using the git add command like so: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git add -u +``` + +The -u flag tells git to stage any modified files that are already in source control. To add new files or stage only specific files use the file name instead like so: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git add some_file_name.txt +``` +* If we repeat the git status command we get something like this: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git status -s +M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/BannerComposite.java +M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +?? .metadata/ +?? uk.ac.stfc.isis.ibex.client.tycho.parent/workspace/ +?? workspace/ +?? ../runtime-ibex.product/ +?? ../surefire-reports/ +``` +Notice that there is no longer a space before the M, this indicates that the file is staged. + +* Now we commit the changes locally using the git commit command. + +The commit will require a comment and the format for comments should be as per https://robots.thoughtbot.com/5-useful-tips-for-a-better-commit-message i.e. +50 char title, blank line, further details wrapped at 72 characters per line. +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git commit +``` +* Check for upstream commits + +Before we finally push to the server we will merge into our branch any changes that have been pushed to master on github, this +is to make the job easier for a reviewer as we will see potential merge conflicts and handle them ourselves. We update our local master branch with +changes on GitHub and then merge them into our copy: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git pull origin master +From https://github.com/ISISComputingGroup/ibex_gui + * branch master -> FETCH_HEAD + Already up-to-date. +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git merge master +Already up-to-date. +``` +(Advanced usage note: if your branch does not yet exist on GitHub, and has not been otherwise shared with another developer, then you may wish to consider doing a rebase rather than a merge) + +* Next we push the changes back to GitHub using the git push command like so: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (Ticket768) +$ git push origin Ticket768 +Counting objects: 31, done. +Delta compression using up to 8 threads. +Compressing objects: 100% (8/8), done. +Writing objects: 100% (15/15), 965 bytes | 0 bytes/s, done. +Total 15 (delta 6), reused 0 (delta 0) +To https://github.com/ISISComputingGroup/ibex_gui.git + 86f5162..8b9814f Ticket768 -> Ticket768 +``` +* If we navigate back to the branch on the GitHub page we can see that the changes have been pushed back: + +![GitHub](pushed_branch.png) + +## Create a pull request + +* Create a pull request by clicking the 'Compare & pull request' button while on the correct branch: + +![GitHub](pull_request_start.png) + +* The new page allows you to add comments and to review the modifications before creating the pull request. +Notice that for my changes it says "Able to merge". This means that my changes don't clash with any other changes that have been made on the master while I have been working on the branch. + +![GitHub](open_a_pull_request.png) + +* Clicking the "Create pull request" button will create the pull request + +* Reference the git issue in the comments by paste a link to the issue. For example ```ISISComputingGroup/IBEX/issues/1016``` + +* Assign the correct milestone + +* Ensure that the code has built and tested correctly, this can be seen by a message on the bottom of the PR. Note that any increase in checkstyle issues will result in a failed build. If you believe these to be unavoidable discuss with the reviewer. + +![GitHub](checked_pr.png) + +* That is us done for now as the ticket now needs to be reviewed +* Add a summary of your changes to the table under "Changes in software but still on a branch" in the [Release Notes](https://github.com/ISISComputingGroup/IBEX/wiki/ReleaseNotes_Dev). If your changes require modifications to files on some or all of the instruments to work correctly, document these in the relevant section. + +## Reviewing a pull request + +Before reviewing the pull request it is necessary to copy the branch locally if you don't already have it: + +* First get an updated list of remote branches: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git remote update +Fetching origin +From https://github.com/ISISComputingGroup/ibex_gui + * [new branch] Ticket768 -> origin/Ticket768 +``` +* The new branch should appear in the branch list: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git branch -a +* master + remotes/origin/HEAD -> origin/master + remotes/origin/Ticket768 + remotes/origin/master +``` +* Now checkout the branch: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git checkout Ticket768 +Branch Ticket768 set up to track remote branch Ticket768 from origin. +Switched to a new branch 'Ticket768' +``` + +* The code can now be loaded into Eclipse and reviewed + +* Once the code has been reviewed either you can merge the changes yourself via GitHub or you can pass it back to the developer to do it + +## Merging changes + +Basically there are two types of merges: one where the code changes don't clash with other changes on master; and, one where it does clash. + +If it does not clash then it can be merged via the "Merge pull request" button on the pull request page on GitHub. + +Otherwise, GitHub will say "We can't automatically merge this pull request" on the pull request page. +This requires manual intervention: + +* From the command line switch to the the master branch if not already on it + +* Next fetch the most up-to-date version of master: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git fetch origin + +* Merge the master with the branch: + +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git merge Ticket768 +Auto-merging base/uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +CONFLICT (content): Merge conflict in base/uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +Automatic merge failed; fix conflicts and then commit the result. +``` +* The merge has failed (as expected) but we can now view the code conflict in Eclipse: +```java +... +<<<<<<< HEAD + showBanner("No groups to display!"); +======= + // Leave text blank + showBanner(""); +>>>>>>> Ticket768 +... +``` + +* For this example I decide to keep my changes and ignore the master, so the next step is merge the new changes and update on GitHub: +``` +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master|MERGING) +$ git status -s +M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/BannerComposite.java +UU uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +?? .metadata/ +?? uk.ac.stfc.isis.ibex.client.tycho.parent/workspace/ +?? workspace/ +?? ../runtime-ibex.product/ +?? ../surefire-reports/ + +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master|MERGING) +$ git add uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java + +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master|MERGING) +$ git status -s +M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/BannerComposite.java +M uk.ac.stfc.isis.ibex.ui.blocks/src/uk/ac/stfc/isis/ibex/ui/blocks/groups/GroupsPanel.java +?? .metadata/ +?? uk.ac.stfc.isis.ibex.client.tycho.parent/workspace/ +?? workspace/ +?? ../runtime-ibex.product/ +?? ../surefire-reports/ + +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master|MERGING) +$ git commit -m "Resolved conflict with Ticket768" +[master 2aaaf10] Resolved conflict with Ticket768 + +mjc23@NDW1373 /c/CodeWorkspaces/GitHub/ibex_gui/base (master) +$ git push origin master +Username for 'https://github.com': matt.clarke@stfc.ac.uk +Password for 'https://matt.clarke@stfc.ac.uk@github.com': +Counting objects: 1, done. +Writing objects: 100% (1/1), 229 bytes | 0 bytes/s, done. +Total 1 (delta 0), reused 0 (delta 0) +To https://github.com/ISISComputingGroup/ibex_gui.git + 06cecee..2aaaf10 master -> master +``` +* Now if you look at the pull request on GitHub it should say it has been merged and closed. +* Move the changes documented in the [Release Notes](https://github.com/ISISComputingGroup/IBEX/wiki/ReleaseNotes_Dev) down into the table in the "Changes merged into master" section, as well as any required instrument-specific modifications related to the same ticket. + +## Deleting the branch + +Once the branch has been merged into master it can be deleted via the pull request page on GitHub. Don't worry it is not permanently deleted! +You will also have to delete the branch locally. \ No newline at end of file diff --git a/doc/client/getting_started/Maven-and-Tycho.md b/doc/client/getting_started/Maven-and-Tycho.md new file mode 100644 index 000000000..1510d0080 --- /dev/null +++ b/doc/client/getting_started/Maven-and-Tycho.md @@ -0,0 +1,368 @@ +# Maven and Tycho + +The IBEX GUI is built using Tycho which is an extension of Maven designed for building Eclipse RCP products. + +This document provides a brief overview of both Maven and Tycho. + +## Maven ## + +The official description: + +_Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information._ + +In simple terms: + +_Maven is a build automation tool used primarily for Java projects._ + +### What is a POM file? ### +An XML file that describes the software project being built, its dependencies on other external modules and components, the build order, directories, and required plug-ins. + +In many ways it can be consider the equivalent of a C-style Makefile as it defines how the project is built. Here is an example: + +```xml + + + + 4.0.0 + com.maven_example.test + test + 1.0-SNAPSHOT + jar + + + + junit + junit + 4.8.2 + test + + + + + UTF-8 + + + + + + org.sonatype.install4j + install4j-maven-plugin + + + compile-installers + package + + compile + + + ${project.basedir}/src/main/installer/myproject.install4j + + + + + + + + +``` + +Let's explain the various parts shown: +* Near the top we have the following information: + * The `groupId` is usually the name of the overarching project and is the same style as a Java package ID, e.g. org.apache.maven + * The name that the project is often known by, e.g. maven + * The version number. SNAPSHOT is automatically replaced by a date-time indicating the build time + * Packaging defines what is built, in this case a standard jar file +* The dependencies section defines any external dependencies required for the project, in this case only junit is required +* The properties section is used to define other properties relevant to the build +* The build section defines specific information for the build such as the plugins to use. In this example, we use install4j to create an executable file + +See https://maven.apache.org/pom.html for more information on the various sections. + +### A Maven project ### + +Maven is very picky about how a project is laid out, it requires the various directories to be named and positioned correctly. +Luckily, most IDEs will take care of this for us. + +The layout is as follows: +``` +|- The top-level of the project + |- src + |- main + |- java + |- The packages directories for the source e.g. com.myexample + |- resources + |- test + |- java + |- The packages directories for the tests + - pom.xml +``` + + +The POM described above is sufficient for us to build a project from the command line using Maven. The commonly used commands used are: + +* `mvn compile` - this creates the JVM bytecode +* `mvn test` - this runs any unit tests +* `mvn package` - create the jar and runs the tests +* `mvn clean` - removes any previous build artifacts + +### Multiple modules ### + +If we have a project made up of a number of different packages (or modules) with dependencies then Maven can handle that. +For example, let's say we have two separate packages: com.example.library; and, com.example.application where application depends on library. The POM for application would need to indicate this dependency like so: + +```xml + + + + 4.0.0 + com.maven_example + myApp + 1.0-SNAPSHOT + jar + + + + com.example.library + library + 1.0-SNAPSHOT + compile + + + com.google.code.gson + gson + 2.6.2 + compile + + + +``` +It also has a dependency on GSON for the hell of it. The dependencies have a scope, this determines how they are treated. The most common options are: + +* `compile` - dependencies are available in all classpaths (default) +* `test` - only needed for testing +* `provided` - provided at run-time, e.g. from the JDK +* `runtime` - not needed for compilation +* `system` - have to provide the containing JAR explicitly + +There are other options and more detailed explanations on the [Maven website] (https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html) + +Now application has no idea where to find library, so we create a new POM file in a higher directory which will be our parent POM. The parent provides the information for both modules to work together. The POM looks something like: + +```xml + + + + 4.0.0 + com.example.parent + parent + 1.0-SNAPSHOT + pom + + + com.example.application + com.example.library + + +``` +The modules tags specify the location of the modules to build. And the modules' POMs need to have a reference to the parent, like so: + +```xml + + + + 4.0.0 + com.example + library + 1.0-SNAPSHOT + jar + + + com.example.parent + parent + 1.0-SNAPSHOT + + +``` +Running the build using the parent POM (i.e. from the same directory) will build both modules and link them appropriately. + +### Profiles ### +Using profiles it is possible to configure the build for different circumstances. For example, say I want to install the final product in a certain directory sometimes then I can add a profile for this, like so: + +```xml + + + + 4.0.0 + com.example + library + 1.0-SNAPSHOT + jar + + + com.example.parent + parent + 1.0-SNAPSHOT + + + + + release + + c:\\release\ + + + + +``` +Running the build with the profile specified will result in the final product being installed into the specified directory. To run a profile, specify the profile using -P like so: +```mvn package -Prelease``` + +Profiles can be used for a number of things, such as specifying the JDK version number or the OS to build for. + +For more information see the [Maven website] (http://maven.apache.org/guides/introduction/introduction-to-profiles.html) + +## Tycho ## + +Tycho is a set of Maven plugins and extensions for building Eclipse-based applications with Maven. Tycho allows Maven to support building bundles, fragments, features, P2 repositories, RCP applications etc. + +Tycho is used to build the IBEX GUI application that is deployed on the instruments. + +### The parent POM ### +Maven allows a parent POM to be defined which contains references to the other POM files for the other projects that make up an application. This is used in the IBEX GUI because it allows the centralisation of the settings for building the application. + +The IBEX POM contains a lot of information, so here is a stripped down version to show the key points. + +```xml + + 4.0.0 + CSS_ISIS + uk.ac.stfc.isis.ibex.client.tycho.parent + 1.0.0-SNAPSHOT + pom + + + 0.20.0 + https://oss.sonatype.org/content/groups/public/ + http://download.eclipse.org/releases/kepler + http://download.eclipse.org/eclipse/updates/3.8 + UTF-8 + + + + + kepler-updates + p2 + ${kepler-updates-repo.url} + + + + + + + org.eclipse.tycho + tycho-maven-plugin + ${tycho.version} + true + + + + + + + + ../uk.ac.stfc.isis.ibex.client.product + ../uk.ac.stfc.isis.ibex.feature.base + + + + ../uk.ac.stfc.isis.ibex.ui.mainmenu.tests + + + +``` +Let's explain the various parts shown: +* At the top we have the standard Maven boiler plate as discussed above +* The properties section is used to define properties relevant to the build, in this case, the URLs for the various repositories we are using +* The repositories section defined the version of Eclipse we are building against. +* The build section defines the plugins that will be used to build the application. Most of the ones we used have been removed for clarity. The Tycho plugin entry tells Maven to download Tycho and use that to define the build +* The modules section lists the locations of the POM files for the various projects that make up the application (remember that this is the parent POM) + +### A child POM ### +A typical child POM is shown below + +```xml + + 4.0.0 + uk.ac.stfc.isis.ibex.banner + eclipse-plugin + + CSS_ISIS + uk.ac.stfc.isis.ibex.client.tycho.parent + 1.0.0-SNAPSHOT + ../uk.ac.stfc.isis.ibex.client.tycho.parent + + 1.0.0-SNAPSHOT + +``` + +Basically it is very simple - it points at the parent POM to get most of its information. + +The packaging type is defined a eclipse-plugin. This is a packaging type defined by Tycho, Maven itself has no idea what this means. Other Tycho types used in IBEX are eclipse-feature for features,eclipse-test-plugin for fragment projects that define unit tests and eclipse-repository for configuring builds. + +### Creating an application ### + +In IBEX we define a separate project called `uk.ac.stfc.isis.ibex.client.product` which defines how the product is built. It has a POM file of packaging type eclipse-repository. + +```xml + + 4.0.0 + 1.0.0-SNAPSHOT + uk.ac.stfc.isis.ibex + eclipse-repository + + CSS_ISIS + uk.ac.stfc.isis.ibex.client.tycho.parent + 1.0.0-SNAPSHOT + ../uk.ac.stfc.isis.ibex.client.tycho.parent + + + + + + org.eclipse.tycho + tycho-p2-director-plugin + ${tycho.version} + + + + materialize-products + + materialize-products + + + + + archive-products + + archive-products + + + + + + + +``` + +The key area is the materialize-products section, this tells Tycho to actually create the product. Without this the build runs and checks everything is okay, but does not produce an executable. diff --git a/doc/client/getting_started/checked_pr.png b/doc/client/getting_started/checked_pr.png new file mode 100644 index 000000000..820c03980 Binary files /dev/null and b/doc/client/getting_started/checked_pr.png differ diff --git a/doc/client/getting_started/clone.png b/doc/client/getting_started/clone.png new file mode 100644 index 000000000..a2ece1b0f Binary files /dev/null and b/doc/client/getting_started/clone.png differ diff --git a/doc/client/getting_started/create_branch.png b/doc/client/getting_started/create_branch.png new file mode 100644 index 000000000..6ce56926b Binary files /dev/null and b/doc/client/getting_started/create_branch.png differ diff --git a/doc/client/getting_started/new_branch.png b/doc/client/getting_started/new_branch.png new file mode 100644 index 000000000..45e17664c Binary files /dev/null and b/doc/client/getting_started/new_branch.png differ diff --git a/doc/client/getting_started/open_a_pull_request.png b/doc/client/getting_started/open_a_pull_request.png new file mode 100644 index 000000000..ba75a612b Binary files /dev/null and b/doc/client/getting_started/open_a_pull_request.png differ diff --git a/doc/client/getting_started/pull_request_start.png b/doc/client/getting_started/pull_request_start.png new file mode 100644 index 000000000..7daf31315 Binary files /dev/null and b/doc/client/getting_started/pull_request_start.png differ diff --git a/doc/client/getting_started/pushed_branch.png b/doc/client/getting_started/pushed_branch.png new file mode 100644 index 000000000..910341ec5 Binary files /dev/null and b/doc/client/getting_started/pushed_branch.png differ diff --git a/doc/client/getting_started/start.png b/doc/client/getting_started/start.png new file mode 100644 index 000000000..858317998 Binary files /dev/null and b/doc/client/getting_started/start.png differ diff --git a/doc/client/misc/Designing-for-Colour-Blindness.md b/doc/client/misc/Designing-for-Colour-Blindness.md new file mode 100644 index 000000000..31f52a347 --- /dev/null +++ b/doc/client/misc/Designing-for-Colour-Blindness.md @@ -0,0 +1,13 @@ +# Designing for Colour Blindness +Some people are colour blind (to varying degrees). About 7% of men and <1% of women are colour blind. The most common form of colour-blindness is red-green colour blindness (inability to distinguish red from green). A smaller number of people have blue-yellow colour blindness. + +This page provides some tips on designing displays that are suitable for people with normal vision and those with colour blindness. + +## Tips +1. Don't rely on colour alone to distinguish between elements of the display. Combine colours with different shapes (with outlines, in black, say) or text. +1. In graphs, combine colours with line styles (solid lines, dashed lines, alternative dots-dashes, etc.). +1. Avoid colour combinations that colour-blind people find difficult to distinguish. + 1. [Colour Combinations](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Colour_Combinations.pptx) illustrates some poor and some good colour combinations. + 1. As a simple rule (for red-green colour blindness): replace red with magenta or green with turquoise. +1. Download the [ColorOracle](http://colororacle.org/index.html) tool. It temporarily changes the colour balance of your screen to show how it appears to a colour blind person. Use it when designing displays & graphs. NOTE: when running the ColorOracle exe file, a small icon will appear on your taskbar (next to the date and time); if it's not there, check your hidden icons. +1. The [ColorOracle](http://colororacle.org/design.html) site has more comprehensive design advice. \ No newline at end of file diff --git a/doc/client/misc/Embedding-JavaFX-views-into-IBEX.md b/doc/client/misc/Embedding-JavaFX-views-into-IBEX.md new file mode 100644 index 000000000..9cb2f9364 --- /dev/null +++ b/doc/client/misc/Embedding-JavaFX-views-into-IBEX.md @@ -0,0 +1,54 @@ +# Embedding JavaFX views into IBEX + +This is useful for e.g. embedding native views from Phoebus such as displaybuilder, databrowser etc. + +### EFXclipse / compatibility layer setup + +In order to be able to embed JavaFX views, the efxclipse compatibility layer must be configured on the branch you're attempting to use. The easiest way to check this is to check for the presence of an `uk.ac.stfc.isis.ibex.javafx` package - if this package is not present, you will not be able to embed javafx views. Currently this package is only available on the Phoebus proof-of-concept branch. + +### Patching Java to include JavaFX support + +To enable JavaFX support on a java installation, it must be patched with the JavaFX overlay. See [here](/deployment/deploy/Upgrade-Java) for details of how to do this. + +If you don't do this, you will get an error of the form: + +``` +java.lang.ExceptionInInitializerError +    at uk.ac.stfc.isis.ibex.ui.alarm.AlarmView.createPartControl(AlarmView.java:44) +    at org.eclipse.ui.internal.e4.compatibility.CompatibilityPart.createPartControl(CompatibilityPart.java:158) +    at org.eclipse.ui.internal.e4.compatibility.CompatibilityView.createPartControl(CompatibilityView.java:155) +    at org.eclipse.ui.internal.e4.compatibility.CompatibilityPart.create(CompatibilityPart.java:365) +[...] +Caused by: java.lang.RuntimeException: No toolkit found +    at com.sun.javafx.tk.Toolkit.getToolkit(Toolkit.java:280) +    at com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:291) +    at javafx.application.Platform.startup(Platform.java:113) +    at javafx.embed.swt.FXCanvas.lambda$initFx$4(FXCanvas.java:356) +    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) +    at java.base/java.security.AccessController.doPrivileged(AccessController.java:460) +    at javafx.embed.swt.FXCanvas.initFx(FXCanvas.java:355) +    at javafx.embed.swt.FXCanvas.(FXCanvas.java:272) +``` + +### Creating a JavaFX view + +They key bit of code to do this is actually rather simple: + +```java + @Override + public void createPartControl(final Composite parent) { + FXCanvas fxCanvas = new FXCanvas(parent, SWT.NONE); + fxCanvas.setScene(new Scene(new AlarmTreeView(new AlarmClient("127.0.0.1:12345", "kafka_topic", null)))); + } +``` + +Where `AlarmTreeView` is a JavaFX UI class. `Scene` is a class provided by JavaFX (you can get this by depending on `uk.ac.stfc.isis.ibex.javafx` in the plugin's `MANIFEST.MF`). `FXCanvas` is the efxclipse class that enables SWT <-> JavaFX interoperability - again it is exported from `uk.ac.stfc.isis.ibex.javafx` - in particular it comes from `javafx-swt.jar`. + +### JavaFX library setup + +Note: we cannot currently get JavaFX from maven - although the dependency looks like it's there, it doesn't work for us due to the absence of `javafx-swt.jar`. + +- Download javafx SDK from https://gluonhq.com/products/javafx/ +- Put the jars in `uk.ac.stfc.isis.ibex.javafx/lib` +- Reference the jars on the plugin classpath and runtime-export any `com.sun.javafx.*` and `org.openjfx.*` packages from the plugin. + diff --git a/doc/client/misc/Icon-Licences.md b/doc/client/misc/Icon-Licences.md new file mode 100644 index 000000000..e45e60e11 --- /dev/null +++ b/doc/client/misc/Icon-Licences.md @@ -0,0 +1,7 @@ +# Icon Licences + +Icon licences are available to view in the IBEX GUI from the `Help -> Icon Licences` dialog on the main window. + +## Crediting icon licences + +When adding new icons ensure that the correct licence is included in the icon licences file. This is located at `uk.ac.stfc.isis.ibex.e4.ui.help.resources/iconlicences.txt`. The icon licences file is structured so that each package has a subsection with a path to the icons resource folder. The icons name must be listed and the associated licence. diff --git a/doc/client/misc/Local-files-and-start-up.md b/doc/client/misc/Local-files-and-start-up.md new file mode 100644 index 000000000..2c621acc0 --- /dev/null +++ b/doc/client/misc/Local-files-and-start-up.md @@ -0,0 +1,18 @@ +# Local Files + +Some processes require files to be created in user directory, which can be by default found at this path: + +`C:\Users\USERNAME\AppData\Local\IBEX` + +This path can be changed using preferences. +This folder can contain data like temporary files for use of the IBEX GUI client. +Note that this folder can be safely deleted as it is automatically generated when necessary. + +## Startup Processes +There are some processes happening during startup of IBEX client as part of application's lifecycle. +As part of ticket [#6577](https://github.com/ISISComputingGroup/IBEX/issues/6577) of particular interest +are classes: `ApplicationWorkbenchWindowAdvisor` and `ApplicationWorkbenchAdvisor` in package `uk.ac.stfc.isis.ibex.e4.product`. + +In order to detect that multiple instances of IBEX client are running, a file in the folder is locked using a Java `FileLock`. Every new instance of the client will check if the file is locked. If the file is locked then it means that another instance is already running and user should be prompted for confirmation to start another client instance. See [#7381](https://github.com/ISISComputingGroup/IBEX/issues/7381). + +Currently only the first instance locks the file. So as an example if two instances are started, the first locking instance is closed, and a third instance is started, the user will not be prompted and the third instance will re-lock the file. \ No newline at end of file diff --git a/doc/client/misc/Synoptic-Icons.md b/doc/client/misc/Synoptic-Icons.md new file mode 100644 index 000000000..7fde9716d --- /dev/null +++ b/doc/client/misc/Synoptic-Icons.md @@ -0,0 +1,37 @@ +# Synoptic icons + +If you need to add new icons for the IBEX GUI Synoptic Perspective, these are under `uk.ac.stfc.isis.ibex.ui.devicescreens/icons` (both full size icons for the actual Synoptic and thumbnails for the Synoptic Editor and Device Screens Perspective). Please follow these guidelines when creating new synoptic icons (e.g. `icon.png`) and ensure thumbnail icons have a `_tb` suffix (e.g., `icon_tb.png`). +(Note: this page will not discuss the creative process involved in creating an icon) + +### Getting started + +[GIMP](http://www.gimp.org/) is a powerful, free image editing program that can be used to create icons for Synoptics. There are also some [helpful tutorials](https://www.gimp.org/tutorials/) on their website. + +An alternative is [Inkscape](https://inkscape.org/) which is a vector graphics editor and possibly more suited to icon creation and editing. Please see the [Basic Tutorial](https://inkscape.org/en/doc/tutorials/basic/tutorial-basic.html) for an introduction to the program. + +First, create a new file. Then, find the "Layers"-window in the GIMP taskbar under `Windows > Dockable Dialogs > Layers`. Delete the background layer from the list. Create a new layer by right-clicking the layers window or from the `Layer` tab in the taskbar. Make sure the Layer Fill Type is set to "Transparency". Draw your Icon on this layer. + +Once you've created the icon image, you will need to add the .png file of the image to the `uk.ac.stfc.isis.ibex.ui.devicescreens/icons` You can specify which icon should appear in the IBEX GUI Synoptic Perspective with the file `uk.ac.stfc.isis.ibex.ui.devicescreens/ComponentIcons.java`. This file has a large switch statement to cover all of the component types, and link their icons to them. The case name in the switch statement should match the `` tag for the device in `opi_info.xml`. e.g. `PUMP` and ```case PUMP: return "pump";``` where the returned value (e.g. pump) is the icon's base filename, pump.png + + + +### Icon Conventions + +* Synoptic icon image files have a maximum height of 100px and a width of (icon width + 20px) including margins. +* The icon image should be a `.png` file. +* The fill colour of the icons is `0xAAA19C`. The border colour is black. +* The icon background should be transparent. +* You will need to make a thumbnail image of the icon as well, for smaller display windows in the IBEX GUI like the Synoptic Editor. Thumbnail icon image files have height of 28px and a width of 28px including margins. These can usually be created by scaling down the icon image in `Image > Scale Image` + +### Centred Icons +To centre your new (or a pre-existing) icon, open the file in gimp and follow these steps: +* Crop the image with `Image > Autocrop Image`. This removes all empty space around the Icon. +* The icon should now have a height of 100px or less. If this is not the case, scale it down under `Image > Scale Image` +* Open `Image > Canvas Size`. Unlink Width and Height by clicking the little chain next to the number entry fields. +* Set Height to 100px and width to (current width) + 20px. +* Click on `center` and finish by clicking `Resize`. The icon should now sit right in the centre of the canvas. and have some margins around it. + +### Non-Centred Icons +If you don't want the icon to be centred on the beamline in the synoptics window: +* Click and drag on the ruler above the canvas to produce a guideline. Position it at a vertical position of 50 px (see ruler on left). This simulates where the beamline will appear relative to the icon in the synoptics view. +* Select and move your Icon in the desired vertical position. diff --git a/doc/client/opis/OPI-Creation.md b/doc/client/opis/OPI-Creation.md new file mode 100644 index 000000000..d19491a36 --- /dev/null +++ b/doc/client/opis/OPI-Creation.md @@ -0,0 +1,168 @@ +# OPI creation + +OPIs live in `ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources` + + - Start CS-Studio to create/edit an OPI (see `EPICS\CSS\master\start_css.bat`) + - Load in the project in `ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources`: File -> Import -> General -> Existing Projects into Workspace. + + - Set the correct path for the Fonts and Colours files: Edit -> Preferences -> CSS Applications -> Display -> BOY. For the colour file choose `/resources/Share/isis_colours.def`; for the font file choose `/resources/Share/isis_fonts.def`. + + - Create a new OPI with File -> New -> BOY -> OPI File + - Change to the "OPI Editor" perspective to allow easier editing. + - You need to use "OPI Editor" to edit your files, **not** "Display Editor". The later creates a display builder file and not an OPI. + - Edit the OPI - note that the `template.opi` will give you an idea of the style to use. It is best to copy controls from this template to keep the same behaviour. + - Also add an entry to the `opi_info.xml` file which can be found in `resources\opi_info.xml`. Note that some Macros are provided (See below). + +If you are stuck on how to do something with an OPI there are a number of in-built examples, which you can view by going to File -> Import -> BOY-> BOY Examples. + +## OPI Editor vs Display editor +When editing an existing OPI or creating a new one, it is important to use OPI editor over Display editor, as the display editor is not currently compatible with the rest of IBEX. If you have opened an OPI and can't find the ISIS colour and font schemes, this possibly indicates that you are using the wrong toolset. You may also find that the file extension is being changed to a .bob file when the file is saved in a Display editor window. + +Re-opening the file in an OPI editor window (right click OPI -> Open With -> OPI Editor) may solve these issues. + +## Consistency - Template OPI +In order to promote consistency in look and feel between different OPIs, we have created a template OPI in `ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources\template.opi`. Please follow the guidelines provided in this template, including ALL fonts and colours (it's probably easier to just copy the widgets from this template). If you're adding new widgets in your OPI which are not covered by this template, please add them to the template. + +Please also make sure your OPI works well for colour blind users, following [these guidelines](../misc/Designing-for-Colour-Blindness). + +There is a script called `check_opi_format.py` in `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis` that can help you check certain aspects of a new OPI. This script does not replace checking the OPI visually, but should help catch some style errors in OPIs. This will iterate over all OPIs in the `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources` directory, and write logs to the `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis\check_OPI_format_logs` directory by default. Other behaviours are described below: + +Usage: +`python check_opi_format.py [-h] [-file FILE] [-directory DIRECTORY] [-logs_directory LOGS_DIRECTORY]` + +e.g. +`python check_opi_format.py -directory resources/Lakeshore336 -file Lakeshore336.opi -logs_directory my_logs_directory` +will scan a single OPI in `resources/Lakeshore336/Lakeshore336.opi` and save the logs in `/my_logs_directory/` + +There is a script called `update_values.py` in `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis`. This will iterate over all OPIs in the `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources` directory and update named colour and font tags to match the definitions given in `C:\Instrument\Apps\EPICS\CSS\master\Share\isis_colours.def` and `C:\Instrument\Apps\EPICS\CSS\master\Share\isis_fonts.def`, respectively. + +Usage: +`python update_values.py [-file FILE] [-directory DIRECTORY] [-attribute ]` + +e.g. +`python update_values.py -directory resources/DG645 -file dg645.opi -attribute color` +will scan a single OPI in `resources/DG645/dg645.opi` and update the `color` attributes as necessary. + +## Macros +When an OPI is opened from the synoptic (via OpiTargetView.java) you get at least the following macros automatically set: + +- `NAME`: the OPI title, as defined by the synoptic component editor (e.g. "Slit 2"). When setting up the OPI this macro should be in the Name property of the OPI. +- `OPINAME`: name of the OPI file opened (e.g. Slit.opi) +- `P`: the instrument pv prefix (e.g. IN:LARMOR:) +Others macros may also be available, passed down from higher screens (also defined via the synoptic editor component) + +One convention we have used is to define a macro within the opened OPI called PV_ROOT which we then use to make full PV names. So at the top of the OPI XML would be something like + + + true + $(P)$(EUROTHERM) + + +here EUROTHERM would be a parameter passed from the synoptic and it is combined with P to create the prefix for all PVs referenced within the OPI screen. The macro name should be something like the un-shortened device name, and its value would typically be the IOC name used in the PVs, e.g. EUROTHERM_01. + +In case an OPI's PV_root is defined in its parent, you need to go where to the parent OPI, select the section that represents the child OPI, and in the Macros row in the Properties tab you can find the value for PV_ROOT. Example: The psu_summary_indicate_changeover OPI can be found in the parent OPI `riken_front_end_overview`. If you open the former in OPI Editor, PV_ROOT ca not be found in the Macros row. However, if you go to `riken_front_end_overview`, which is the parent OPI, and click a linking container that links to psu_summary_indicate_changeover, you will find that PV_ROOT is defined in the Macros row. + +### Other Standard Macro Names + +| Name | Meaning | +| ---- | ------- | +| PV_ROOT | Root for all PVs within an OPI | + +## Completing the OPI + +Add the IOC name with appropriate information, including macros to the xml file + + ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources\opi_info.xml + +In the xml file: +* `key`: unique name for component +* `value.type`: the component type, same as name of constant in ComponentType (uk.ac.stfc.isis.ibex.synoptic.model) + This enables the correct component target details to be added when component details are selected +* `value.path`: the path to the opi file (relative to the resources folder) +* `value.description`: description which is shown in target details +* `value.macros.macro.name`: name of macro in OPI +* `value.macro.macro.description`: description of macro value which appears in target details when user clicks on the macro. For a macro that contains the IOC name please provide an example with the actual IOC name in brackets. +* `value.macros.macro.default`: default macro value. This will be placed into the appropriate cell of the macro table if one isn't already present. + +Please follow [these guidelines](../misc/Synoptic-Icons) when specifying and creating new synoptic icons for the OPI. + +The final step should be to send the OPI to the instrument scientists. If they don't approve it that is fine but it is good to get their response. + +## Testing + +To start and interact with a testing IOC see [Running (and testing) IOC](/iocs/testing/Running-IOCs) +The easiest way to test this is to run IBEX-gui through the eclipse editor. Create a device screen and use this to test. The device screen can be refreshed (right click -> refresh) to display the latest changes made in CSS editor. + +## Debugging: No scrollbars + +Check if you have stray widgets. An example of a stray widget might have: + +``` +-2147483647 +2147483496 +``` + +If you have a stray widget, it may prevent scrollbars from appearing on your OPI. It might be possible to solve the issue by selecting and copying the elements to a new OPI. You can then diff the old version and the copied version using the following tools: +- XML sorter +- Diff tool e.g. notepad++, winmerge + +## Writing to char waveform PVs and displaying char waveforms as strings + +CSS text input fields can write to char waveform PVs but needs to be set up in a special way. If you don't do this, the `NORD` field of the waveform will not be set correctly on write and the PV value will only be written successfully if NORD already has a large enough value. +- Postfix the PV name with ` {"longString":true}` (note the space before the JSON). The final value in "PV Name" should look similar to the following: `$(PV_ROOT):ARBITRARY:SP {"longString":true}` +- Ensure "Format Type" is "String" + +Note: If waveform PV is being attached to a multi-line text box then user will have to `ctrl-enter` to save their changes. + +## Data browser Graph Creation + +To make a databrowser graph on an OPI: + +1. In CSS go to the data browser perspective +1. Click button "Add new databrowser plot" which is graph with a plus on in the tool bar +1. Right click -> Add a PV/PVs +1. PV should start with `$(PV_ROOT)` and end in `.VAL`, e.g. `$(PV_ROOT)POS.VAL` +1. Alter colours, remove/add title and legend, rename axis +1. Save in the resources folder, probably in a subfolder for the device (extension will be added automatically as .plt) +1. Add a `Data Browser` monitor to your opi +1. In properties in file set the file you have created + +## Activate tab + +If it's required to activate a tab from a synoptic via a macro: (See SKF G5 Chopper OPI as an example.) + +1. Put macro in local PV +1. Create script that reads value +1. Set active tab on tab container using `setActiveTabIndex` + +## Create a Local Enum + +AS OF 05/02/2020 DOES NOT WORK. + +Create a local pv in an opi with labels (From Tech talk article not tested). + + loc://demo(2, "A", "B", "C", "D") + +If you cannot use an actual enum PV because there are too many labels, and you can't use a local enum PV because that doesn't allow changes to the enum labels, then don't use an enum PV. Instead, on the combo box, un-check the option to get "Items from PV" and directly enter the items, as many as you want. +Or set the items from a script: +``` + from org.csstudio.display.builder.runtime.script import ScriptUtil + combo = ScriptUtil.findWidgetByName(widget, "Name Of My Combo") + combo.setItems( [ "Ene", "Mene", "Muh" ] ) +``` + +## Enable and disable controls + +If the enablement state of a widget (i.e. `enabled` property) is controlled as part of OPI logic, it can be made more obvious by also controlling its `transparent` property. + +See `\base\uk.ac.stfc.isis.ibex.opis\resources\mercuryiTC\enablement_of_controls.py` for examples of setting properties based on the value of a PV. + +Initial investigations centred on alternating the background colour of the control between e.g. `ISIS_Textbox_Readonly_Background` and `ISIS_Textbox_Background`. However, although the logic and syntax were sound, the OPI didn't faithfully reproduce the requested colours. + +## Using OPI rules and external scripts + +When implementing display logic for OPIs (such as toggling an objects visibility based on a PV value) our standards are: +1. Rules take preference over scripts for trivial logic +1. Scripts if needed should be an external file + +Bear in mind that both scripts and rules run **in the GUI thread** this can lead to unresponsiveness if you are doing a lot of work in them. You can write multithreaded scripts, see the jaws_display.opi for an example of this. \ No newline at end of file diff --git a/doc/client/opis/OPI-programming-tips-and-limitations.md b/doc/client/opis/OPI-programming-tips-and-limitations.md new file mode 100644 index 000000000..0d66c7867 --- /dev/null +++ b/doc/client/opis/OPI-programming-tips-and-limitations.md @@ -0,0 +1,273 @@ +# OPI Programming tips & limitations + +## Opening a new Opi +CSS can display one or more windows, each of which can be divided into multiple views, each of +which can contain multiple tabs, so there is huge flexibility but CSS has its own ideas about +how to do things, which introduce some annoying limitations. + +CSS allows you to open a new opi in many locations: +* In place of the current Opi +* In a new window +* As a new tab in the current view +* As a new tab in the top, bottom, left or right views of the current window + +What it does not allow you to do is to replace the contents of another view. +Therefore you cannot have a window with navigation controls in the top view +that change the opi shown +in the main view. They can only create new tabs in the main view + +Alternatively the opi can be shown in a linking container + +## Linking Containers + +The contents of the linking container can be changed dynamically by setting the 'opi_file' property. + +The macros seen by the contents of the container can be set by: + +``` +from org.csstudio.opibuilder.scriptUtil import DataUtil +... +macros = DataUtil.createMacrosInput(True) +macros.put('NAME', 'VALUE') +container.setPropertyValue('macros', macros) +``` + +but the new macros only take effect when a new opi is loaded. If the new opi has the same name as the previous one, nothing is loaded and the new macros do not take effect. To force a reload, set 'opi_file' to the empty string then back to its old value. + +(Isabella: from within CSS scripts, it works by simply re-setting the 'opi_file' property with forcing the fire option set to true: + +`widgetController.setPropertyValue("opi_file", widgetController.getPropertyValue("opi_file"), true)` + +) + +When an opi is displayed in a linking container, scripts on the opi display widget do not get run because the display widget is not loaded. The contents of the display widget instead become children of the linking container. If you want a script to run in these circumstances it has to hang off one of the child widgets. + +## The CSS object model as viewed from Python +This is largely undocumented. The help in CSS/BOY says little and Google can find little. +The only way I was able to find out how to manipulate the actions on a button was to view the java source on [GitHub](https://github.com/ControlSystemStudio/cs-studio/tree/master/applications). The Python objects simply expose the public Java methods. + +It is possible to use any java module or method from python code in an OPI. For example, instead of: + +```python +# Note: this is BAD - do not do this! +sleep(1) +widget.doSomething() +``` + +which would block the UI thread for 1 second, we can import some CS-Studio and core Java classes from python directly, and do: + +```python +# Correct approach +from org.csstudio.ui.util.thread import UIBundlingThread +from java.lang import Thread, Runnable +from org.eclipse.swt.widgets import Display +currentDisplay = Display.getCurrent() + +class WorkerThread(Runnable): + def run(self): + Thread.sleep(1000) + + class UITask(Runnable): + def run(self): + widget.doSomething() + UIBundlingThread.getInstance().addRunnable(currentDisplay, UITask()) + +thread = Thread(WorkerThread(), "some_thread_name") +thread.start() +``` + +Which will *not* block the UI thread for 1 second, as it's 1 second wait is in a different thread (it then uses `UIBundlingThread.addRunnable` to get back onto the UI thread once the wait is finished). + +## Changing actions +The list of actions on a button is: + +``` +actionList = myButton.getPropertyValue('actions').getActionsList() +``` + +This is a Java list which can be subscripted and has methods like add() and clear(). + +If you have an existing action, new write PV actions can be created: + +``` +action = sourceList[0].__class__() +action.setPropertyValue('pv_name', 'loc://myPV') +action.setPropertyValue('value', valueToSet) +action.setPropertyValue('description', 'What appears in the menu') +actionList.add(action) +``` + +For some reason new open opi action created in this way do not work, but existing open opi actions can be altered and added to a different button: + +``` +action.setPropertyValue('replace', 1) +action.setPropertyValue('description', 'What appears in the menu') +action.setPropertyValue('path', 'my.opi') +actionList.add(action) +``` + +In the synoptic view I have a hidden button with actions than can be reused. + +## Running a script on page load +You can make a script run on page load by setting it to trigger from a PV called `loc://$(DID)_CONST_1(1)` and making sure it executes even if there are errors. + +![image](opi_script_options_1.png) + +![](opi_script_options_2.png) + +## Storing State +Occasionally it makes sense to store some state in the OPI, e.g. which traces the user has made visible on the OPI. To do this you can use local PVs of the form `loc://MY_PV`. These can be written and read to like normal PVs. + +`loc://` PVs get initialized to null, but you can provide an explicit initialization value by including it in parentheses e.g. `loc://my_pv(123)` would initialize `loc://my_pv` to `123`. + +Local PVs (prefixed with `loc://`) usually have *application level* scope, i.e. a local PV with the same name will have the same value from all OPIs within IBEX. While this can be useful (e.g. to share state between OPIs), it can also be problematic for OPIs where we expect to have multiple instances of the OPI open. + +The `$(DID)` macro is a unique-per-opi-view identifier which can be inserted into local PV names to ensure that multiple instances of the same OPI do not conflict with each other. + +## Creating new widgets + +Given an existing widget of the correct type, new widgets can be created and added to a container. + +The following will create a new widget of the same type as Label0 and set some of its properties: + +``` +label = group.getWidget('Label0') +source = label.getModel() +model = source.__class__() +model.setName('NewLabel') +model.setX(6) +model.setY(142) +model.setWidth(163) +group.addChild(model) +``` + +The model has set and get methods for the widget properties, or `group.getWidget('NewLabel')` can be used to obtain the widget so its properties can be set using `setPropertyValue()`. + +The widget can be removed using `group.removeChildByName('NewLabel')` + +Note that for dynamically create widgets that do not normally have click actions (e.g. the grouping containers that I use to fake transparent buttons), it seems to be impossible to hook an action to a left click but it is possible to add them to the right click menu. + +## Threading + +Bear in mind that both scripts and rules run **in the GUI thread** this can lead to unresponsiveness if you are doing a lot of work in them. You can write multithreaded scripts, see the jaws_display.opi for an example of this. + +Note that it is a programming error to call `sleep` or other similar functions directly in an OPI script - these will completely block the UI thread every time the script executes. Instead, use a threading pattern and only call `sleep` in a non-UI thread if you need an arbitrary wait in OPI scripts. `UIBundlingThread.addRunnable` can be used to get back onto the UI thread after the sleep is complete. + +## Utilities + +Seven utility classes are provided, but there seems to be no way to get to some of them in the online help other than to know their names and search for them: + +* ConsoleUtil has methods for writing debug messages to the console. + ``` + from org.csstudio.opibuilder.scriptUtil import ConsoleUtil + ConsoleUtil.writeInfo(value ) + ``` +* `DataUtil` has the `createMacrosInput` method. +* `FileUtil` has methods for reading and writing files and displaying an open file dialog. There is also a `saveFileDialog` method that has been omitted from the online help. +* `GUIUtil` has a yes/no dialog and methods for going to full screen or compact mode +* `PVUtil` +* !ScriptUtil +* !WidgetUtil + +## Performance limitations + +We have found that OPIs perform poorly on initial opening once you get past a few hundred widgets being rendered simultaneously. + +At the time of writing, here are some representative numbers of widgets on a variety of OPIs. Note that widgets in tabs which are not shown don't count as they don't get rendered immediately. + +- Eurotherm: 58 widgets +- ICE fridge (with complex schematic): 92 widgets +- Reflectometry front panel: 135 widgets +- CS-Studio "large" performance test: 200 widgets +- Table of motors OPI attempt: 674 widgets - took ~15 seconds to open on NDXSCIDEMO + +See also section below on improving the performance of rules, if your OPI contains significant numbers of rules (e.g. more than 10). If your OPI contains significant numbers of rules, you need to ensure that all rule conditions have an associated fast-path handler. + +## Implementing your own widgets in CS-Studio code + +*Note: this is a significant undertaking; check with other developers that this is the correct solution before going down this route* + +Firstly, you will need to import the CS-Studio code into eclipse by following the instructions [here](https://github.com/ISISComputingGroup/isis_css_top/blob/master/README.md) + +The code for most widgets is defined in `cs-studio\applications\opibuilder\opibuilder-plugins\org.csstudio.opibuilder.widget`, you will need to import this project into eclipse if you haven't already. + +The code follows an MVVM-like pattern, where the mapping is: +- `xxxModel` -> model +- `xxxEditPart` -> viewmodel +- `xxxFigure` -> view + +As an easy way to get started, I suggest copying the above 3 classes from the most similar widget type and then editing them to suit your needs. + +To get your new widget to show up in CSS' menus, it needs to be added to the extension point `org.csstudio.opibuilder.widget` in `MANIFEST.MF`. + +To use a PV value, use a string with the special value `$(pv_value)`. This gets substituted for the actual value at runtime. + +Most figures are implemented using the `draw2d` framework, which is a graphics toolkit. In general it allows a higher level of flexibility than RCP for drawing arbitrary shapes etc, with the disadvantage that this additional flexibility makes it more complex. It is also possible to implement a figure using an RCP widget - see for example `WebBrowserFigure` in `cs-studio\applications\opibuilder\opibuilder-plugins\org.csstudio.opibuilder.widgets.rcp`. + +## Getting a macro value in a script + +`display.getMacroValue("MY_MACRO")` + +However, if your macro name is constant, it is better to use a local pv as a trigger for the script, which happens to be initialized to the value of the relevant macro. e.g. use a local PV like `loc://$(DID)_some_name("$(MACRO)")` - you can then use this as `pvStr0` as usual in your script/rule. + +## Hotfixing an OPI + +OPIs can be hotfixed on an instrument PC, with a built client, by modifying the OPI files in: +`\\plugins\uk.ac.stfc.isis.ibex.opis_1.0.0.SNAPSHOT\resources` - this means you do not have to copy the entire client over, just the `.opi` file. You may also need to modify `opi_info.xml`, also in this folder, if you are adding a new OPI or modifying macros passed to an OPI. + + +## Improving the runtime performance of rules in OPIs + +As of [ticket 7212](https://github.com/ISISComputingGroup/IBEX/issues/7212), we are using a custom `ScriptStore` implementation in cs-studio called `RhinoWithFastPathScriptStore`. By default, CS-Studio uses a script store called `RhinoScriptStore`. + +The primary difference is that `RhinoWithFastPath` will *attempt* to implement rules in pure java, without needing to call into javascript. This saves a significant amount of memory and CPU time. However, there are a number of limitations: +- All conditions in a rule must be listed either in `FAST_PATH_EXPRESSIONS` in `RhinoWithFastPathScriptStore` (in CS-Studio), or be added by the ibex client in `addFastPathHandlers` in `/uk.ac.stfc.isis.ibex.opis/src/uk/ac/stfc/isis/ibex/opis/Opi.java` + * Note that fast path expressions are an exact string match, so `pvInt0 == 0` is a different fast-path condition to `pvInt0==0`. +- If a rule uses the "output expression" feature of CSS, then all output expressions must *also* be listed in `FAST_PATH_EXPRESSIONS` (or added by the ibex client in `Opi.java` as above). This is currently limited to output expressions of boolean type. +- Fast-path expressions can only use their linked PVs as variables - it is not currently possible to depend on the `widget` or macros directly. For example it is not currently possible to write a fast-path expression for `widget.getValue() == 1`. + +If the above conditions are true, javascript will be bypassed and a pure java implementation used instead. The pure java implementation is significantly more performant in terms of CPU and memory use. The javascript implementation (Rhino) is used as a fallback if the above conditions are not true. This approach lets us be fully compatible with all CS-Studio rules & `.opi` files (which can use completely arbitrary javascript fragments as their conditions), while still getting significant performance gains on most rules in practice. + +You can generate a list of all rule expressions used in IBEX by running: + +```shell +grep -roP "bool_exp=\".*?\"" | cut -d ":" -f 2 | sort | uniq -c | sort -nr +``` + +from a git bash terminal in `ibex_gui/base/uk.ac.stfc.isis.ibex.opis/resources`. The vast majority of our rules use a small number of simple expressions - such as `pvInt0 == 0` - these are the expressions which are good candidates for a "fast-path" expression. + +On OPIs which have had significant performance issues, such as the reflectometry OPI, *all* rule expressions should have an associated fast-path handler, and should not use the "output expression" feature of CS-Studio. This ensures that all rules in this OPI do not use the javascript rules implementation. + +### Checking which rules execute in JS + +In `/uk.ac.stfc.isis.ibex.opis/src/uk/ac/stfc/isis/ibex/opis/Opi.java`, call `RhinoWithFastPathScriptStore.setLogScriptsUsingJS(true)`. This will log the script content the first time a rule is loaded and executes via the slow (JS interpreter) path. + +### Forcing JS execution of all rules + +If you need to execute *all* rules in JS (e.g. a major bug is found in `RhinoWithFastPathScriptStore`), it can be entirely disabled by editing `org.csstudio.opibuilder/java_script_engine=RHINO_WITH_FAST_PATH` to `org.csstudio.opibuilder/java_script_engine=RHINO` in `/uk.ac.stfc.isis.ibex.e4.client/plugin_customization.ini`. + +Note that you will need to increase the GUI heap size parameter if this is done, particularly on reflectometers, as the reflectometry OPI uses huge numbers of rules which cause excessive memory consumption under javascript. + +## Importing scripts into other scripts + +If you need complicated scripts with imports - first consider the design of your OPI. Most OPIs should aim not to require any scripts at all, for example by moving logic into the IOC layer. + +However, in some cases it is useful to be able to create generic scripts and import them into other scripts. The CS-Studio Jython instance does not allow access to `__file__`, so it is not always possible to add a known import directory to python's path. + +The workaround is to use something like: + +```python +from java.lang.System import getProperty as getJavaSystemProperty + +import sys +import os +resources_dir = getJavaSystemProperty("ibex.opis.resources_directory") +sys.path.append(os.path.join(resources_dir, "HV", "Scripts")) +# ... import from HV.Scripts.something +``` + +The `ibex.opis.resources_directory` property is set from Java before any OPIs load in `base/uk.ac.stfc.isis.ibex.opis/src/uk/ac/stfc/isis/ibex/opis/Opi.java` + +Note that hard-coding a path like `c:\instrument\dev\ibex_gui\...\resources` would fail on an instrument as the client will be installed in a different location - so this system property **must** be used instead. + +Note that bi and mbbi records that you are trying to use with an LED in an OPI will not work unless the values are 0 and 1. This is because the LED ignores the bit and ENUM settings and On state and Off State and goes of ONLY the PVName attribute. \ No newline at end of file diff --git a/doc/client/opis/opi_script_options_1.png b/doc/client/opis/opi_script_options_1.png new file mode 100644 index 000000000..93841fab4 Binary files /dev/null and b/doc/client/opis/opi_script_options_1.png differ diff --git a/doc/client/opis/opi_script_options_2.png b/doc/client/opis/opi_script_options_2.png new file mode 100644 index 000000000..ff8fb45a1 Binary files /dev/null and b/doc/client/opis/opi_script_options_2.png differ diff --git a/doc/client/testing/Adding-Unit-Tests.md b/doc/client/testing/Adding-Unit-Tests.md new file mode 100644 index 000000000..11e2e3fa7 --- /dev/null +++ b/doc/client/testing/Adding-Unit-Tests.md @@ -0,0 +1,35 @@ +# Adding tests + +For more detailed information see [an_introduction_to_unit_testing.rst](An-Introduction-to-Unit-Testing). + +It is relatively simple to add unit tests for a plug-in in such a way that maven can run them as part of the build. + +Here are the steps required in Eclipse: + +* Create a new Fragment Project + * File > New > Project... > Plug-in Development > Fragment Project + * Set the project name to `\.tests` + * Change the location to the repository rather than the workspace: `xxx\ibex_gui\base\\\` (don't forget the project name!!) + * Click "Next" + * Make sure the Execution Environment points at the correct version of Java (currently JavaSE-11) + * Click the "Browse" button next to "Plug-in ID" + * Select the plug-in to test and click "OK" + * Finish + +* In the newly created plug-in, add a new Package with the same name as the plug-in or something equally sensible. + * Select the plug-in + * File > New > Package + * Enter the name and click "Finish" + +* In the new Package create a class for adding test + * Select the Package + * File > New > Class + * The class name **must** end in Test to be picked up by the automated build + +* Add tests to the class + * Add `org.junit` and `org.mockito` (if required) to the 'Required Plug-ins', under the Dependencies tab for the manifest + +* Add the test plug-in to the Maven build by [following these steps](../coding/Adding-a-Plugin-or-Feature-to-Maven-Build) + +* Running the Maven build should now also run the tests + diff --git a/doc/client/testing/An-Introduction-to-Unit-Testing.md b/doc/client/testing/An-Introduction-to-Unit-Testing.md new file mode 100644 index 000000000..ea496f936 --- /dev/null +++ b/doc/client/testing/An-Introduction-to-Unit-Testing.md @@ -0,0 +1,255 @@ +# An introduction to unit testing + +To create unit tests for an Eclipse plug-in a Fragment Project is used. When creating a Fragment Project we assign the plug-in we wish to test as a Host Plug-in. +Eclipse automatically gives the Fragment access to the classes in the original plug-in. +In the Fragment Project we create classes to test the classes in the original plug-in. + +## A simple example + +Open the wizard for creating a standard plug-in in Eclipse (File->New->Plug-in Project) and complete the following steps (if values not specified then use the defaults): + +* Set "Project name" to org.myexample.plugin +* Click "Next" +* Uncheck "Generate an activator, a Java class that controls the plug-in's life cycle" +* Uncheck "This plug-in will make contributions to the UI" +* Put "No" for "Would you like to create a rich client application?" +* Click "Finish" + +The will create the plug-in. Inside the src folder create a package called org.myexample.plugin.classes and add a class called StringManipulator. +Add the following code to the class: + +``` + package org.myexample.plugin.classes; + public class StringManipulator { + + public String addStrings(String one, String two) { + return one; + } + } +``` + +Now to create the Fragment Project. Open the Fragment Project wizard under File->New->Other->Plug-in Development->Fragment Project and complete the following steps: + +* Set "Project name" to org.myexample.plugin.tests (i.e the original plug-in name plus ".tests" - this is our naming convention) +* Click "Next" +* Under "Host Plug-in" click the "Browse" button and select the original plug-in +* Click "Finish" + +Eclipse will now create the Fragment Project. We need to manually add the JUnit plug-in as a dependency for the Fragment Project, to do this: + +* Open the MANIFEST.MF file +* Select the "Dependencies" tab +* Under "Required Plug-ins" click the "Add" button +* In the dialog, type org.junit and select the plug-in listed (it should be version 4+) +* Click "OK" +* Save the changes + +In the src directory of the Fragment Project create a package called "org.myexample.plugin.tests". +Add a class called "StringManipulatorTest" - the name **MUST** end in Test for the build system to recognise it. + +Okay let's create a test, add the following code to the StringManipulatorTest class: + +``` + package org.myexample.plugin.tests; + + import static org.junit.Assert.*; + + import org.junit.Test; + import org.myexample.plugin.classes.StringManipulator; + + public class StringManipulatorTest { + @Test + public void add_hello_to_world() { + // Arrange + StringManipulator strMan = new StringManipulator(); + + // Act + String ans = strMan.addStrings("Hello", "World"); + + // Assert + assertEquals("HelloWorld", ans); + + } + } +``` + +To run the test right-click on the Fragment Project and select Run As->JUnit Test. The test should run and fail like so: + +![Failed test](failed_test.png) + +Clearly there is something wrong with the original code in addStrings, so let's fix that by changing: + +``` + public String addStrings(String one, String two) { + return one; + } +``` + +to: + +``` + public String addStrings(String one, String two) { + return one + two; + } +``` + +Now the test should pass if it is run again: + +![Passed test](passed_test.png) + +JUnit has many useful features, here are a select few. + +* other asserts such as `assertTrue`, `assertArrayEquals`, `assertNotEqual` and `assertNotNull` + +* assert that an error is thrown: + +``` + @Test(expected=IndexOutOfBoundsException.class) + public void raises_IndexOutOfBoundsException() { + ArrayList emptyList = new ArrayList(); + Object o = emptyList.get(0); + } +``` + +* set-up and teardown methods - these are code snippets that are run before and after **each** test: + +``` + private List names; + + @Before + public void setUp() { + // Called before each test + names = new ArrayList(); + names.add("Tom"); + names.add("Dick"); + names.add("Harry"); + } + + @After + public void tearDown() { + // Called after each test + names.clear(); + } + + @Test + public void concatenate_names() { + // Arrange + StringManipulator strMan = new StringManipulator(); + + // Act + String ans = strMan.concatenateNames(names); + + // Assert + assertEquals("Tom, Dick and Harry", ans); + + } +``` + +Note: Each test should be independent of the other tests as there is no guarantee of the order they are run in. + +* BeforeClass and AfterClass - these are run once before the first test and after the last test in a class respectively: + +``` + @BeforeClass + public static void oneTimeSetUp() { + // Perhaps create a dummy file or something shared by more than one test + } + + @AfterClass + public static void oneTimeTearDown() { + // Clean up + } +``` + +## Naming conventions for unit tests + +See [test naming](Test-naming). + +## Mockito + +Mockito is a framework for creating mock objects that can be substituted for real objects to make testing easier and more specific. +For example: writing tests that don't rely on a database, file or network connection being present. + +Like JUnit is can be used inside a Fragment Project after the dependency is added (`org.mockito`). + +An example of using Mockito would be to mock a database wrapper so that a real database is not required: + +``` + @Test + public void get_row_data() { + // Arrange + // Create a mock database wrapper as we are not testing that + DatabaseWrapper wrapper = mock(DatabaseWrapper.class); + + // Create a mock "response" + List data = new ArrayList(); + data.add("John"); + data.add("Smith"); + data.add("01/01/1955"); + when(wrapper.getRowData(0)).thenReturn(data); // This is the key line + + // This is the object we are really testing + DataHolder dataHolder = new DataHolder(wrapper); + + // Act + List ans = dataHolder.getFirstRow(); + + // Assert + assertEquals(data, ans); + } +``` + +For more detail on Mockito see [here](Using-Mockito-for-Testing-in-the-GUI). + +## Code coverage + +It is useful to see what parts of a plug-in's code are used or not used by the unit tests. +If a piece of code is not used by the unit tests then that may mean that an extra test is required + +Unit test code coverage can be examined inside Eclipse using EclEmma which can be installed via the Eclipse Marketplace (under the "Help" menu). +Once EclEmma is installed the coverage of the unit tests can be examined. Right-click on the test project and select Coverage As->JUnit Test. This will run the tests and calculate the coverage, the results should look something like this: + + +![Coverage result](coverage_result.png) + +From the results it can be seen that 63.2% of the StringManipulator code is used by the unit tests. +The code that isn't used is highlighted in red - for this example we can see that we need to write a test that tests the reverseString method. + + +## Troubleshooting + +### ClassNotFoundException + +Running the tests in Eclipse might crash with an error like: + +``` + Class not found org.myexample.plugin.tests.StringManipulatorTest + java.lang.ClassNotFoundException: org.myexample.plugin.tests.StringManipulatorTest + at java.net.URLClassLoader.findClass(Unknown Source) + at java.lang.ClassLoader.loadClass(Unknown Source) + at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) + at java.lang.ClassLoader.loadClass(Unknown Source) + ... +``` + +This is a known bug and there is a workaround: + +* Right-click on the test plug-in, and under "Run As" select "Run Configurations" +* In the new dialog click on the "Classpath" tab +* Select "User Entries" and click the "Advanced" button +* Select "Add Folders" and click "OK" +* In the new dialog, expand the test plug-in and select the "bin" folder and click "OK" +* On the original dialog, click "Apply" and then "Run" +* Hopefully, the tests will now work and you should be able to re-run them in the normal way + +### Eclipse is not picking up new tests + +If Eclipse is not picking up changes when you add tests you may need to change the default output folder for tests for Maven to pick it up. + +* Right-click on the tests plug-in, go to properties, Java build path +* Change the output folder to target/test-classes (you may need to create this folder first by clicking browse, selecting target and adding the test-classes folder) +* If this does not work try deleting the target/test-classes folder first, if it existed already, and do a clean rebuild of the workspace + +### IncompatibleClassChangeError + +If the tests are failing because of an IncompatibleClassChangeError error then the solution is to delete the bin and target folders for both the main plug-in and the corresponding test plug-in \ No newline at end of file diff --git a/doc/client/testing/System-Testing-with-Squish-BDD.md b/doc/client/testing/System-Testing-with-Squish-BDD.md new file mode 100644 index 000000000..1393ef35e --- /dev/null +++ b/doc/client/testing/System-Testing-with-Squish-BDD.md @@ -0,0 +1,56 @@ +# System Testing with Squish BDD Tools + +## BDD and Gherkin Overview + +Behaviour-Driven Development (BDD) is a software development process that builds on the lessons learnt from Test-Driven Development (TDD). BDD focuses on developing a common understanding of the behaviour of an application. More details can be found at https://cucumber.io/docs/bdd/ and https://www.agilealliance.org/glossary/bdd/. + +BDD works by describing the behaviours of an application in a language that can be understood by developers and users, the Gherkin language https://cucumber.io/docs/gherkin/reference/, enabling a conversation on detailed application behaviour. This language helps to accurately describe behaviour and provides a way to carefully consider what to build. Gherkin is most effective for designing software when combined with other techniques such as low-fidelity prototypes. + +Gherkin also helps to automate testing. The steps in a `.feature` (gherkin) file can be linked to the code that runs the test step on the application. Squish supports this with its BDD tools https://www.froglogic.com/squish/features/bdd-behavior-driven-development-testing/. We have made use of this in the script generator for all its Squish testing. These tests can now act as documentation of the behaviour of the application and can be used to discuss the intricacies of the behaviour with scientists. + +## Structure of our tests + +Squish is split up into test suites, the script generator tests are all in the `suite_script_gen_tests`. The test cases in this suite are the `.feature` files that describe the behaviour in Gherkin. In general, we define one or more features in each file with the `Feature:` tag and each feature has a collection of scenarios denoted by the `Scenario:` tag. The feature will have a title and a description and a scenario will have a title that acts as its description. + +Each scenario is made up of a set of `Given`, `When`, `Then` steps. These steps can take parameters including whole tables and can be ordered in lots of different ways. `Given` generally describes the state the application should be in before a user action, `When` describes a user action and `Then` describes verification of the state of the application after a user action. More details can be found at https://cucumber.io/docs/gherkin/ and in the Squish tutorials on https://www.froglogic.com/squish/features/bdd-behavior-driven-development-testing/. + +The `Given`, `When` and `Then` steps are linked to code which is stored in the test suite resources steps area. For example, the `then_tooltip.py` file contains `Then` steps related to tooltip behaviour. Steps are defined like this: + +```python +@Then("the following actions have a tooltip |word|") +def step(context, status): + actions_that_we_expect_have_a_tooltip = context.table + do_test_code() +``` + +This step takes a table as a parameter (accessed through `context.table`) and a word parameter (described by the decorator with `|word|` and passed to the step function as `status`). A step can have multiple decorators of `@Given`, `@When` and `@Then`. There are also more abilities such as using `From` sections and passing data between tests through the context variable - details at https://doc.froglogic.com/squish/latest/api.bdt.functions.html. + +We do not generally edit the test.py file in the test case resources scripts. This script is created by squish and handles starting up tests and setting up the hooks. This could be used to start up the client instead of using `@OnFeatureStart` to speed up tests and enable us to better structure our features without concern for lengthening our tests. + +## Scenario and Feature hooks + +In the script section of the test suite resources, there is a file named `bdd_hooks.py`. This file contains a number of functions hooked into the tests to run at the beginning and end of features. For example: + +```python +@OnFeatureStart +def hook(context): + do_hook_code() +``` + +Before the scenarios of a feature are run this hook is called by Squish. We utilise `@OnFeatureStart`, `@OnScenarioStart`, `@OnScenarioEnd` and `@OnFeatureEnd` to carry out the test setup and cleanup activities. More details at `6.19.10. Performing Actions During Test Execution Via Hooks` of https://doc.froglogic.com/squish/latest/api.bdt.functions.html. + +## Implementing a new test + +Have you already agreed to a gherkin description of the behaviour required? If so then add it to the test cases either as a new feature (there's a little button labelled BDD in the test cases section of the Squish GUI) or if your feature fits well into a feature already in the test suite then include the scenarios in there - this avoids running feature start and end code which restarts the client and lengthens the tests, though don't be afraid to add a new feature if it doesn't fit. + +If you haven't already agreed on the behaviour, consider creating a design for the feature with gherkin and a low-fidelity prototype and reviewing it with other developers and script generator users - this depends on how major and well defined the feature is. If you're not sure, ask the team! + +Now that you have your feature, scenario and steps laid out it's time to write the test code. Are any of the steps already defined in the test suite resources? They may not have the same name but they may have the same functionality - you can add a new `@Given`, `@When` or `@Then` decorator to the step or rename the step in the test case. Any steps that don't have an already defined test function for you will need to implement, these steps will be annotated by Squish in the test case. + +To add the steps you want you can either code them directly by writing a test step function or you can record it. If you right-click on a feature or scenario in the test case and click `Record Missing Steps in Feature/Scenario` then the test will execute the steps it knows and then pauses on the steps it doesn't know to record any button clicks and do any verification steps as according to the squish recording tools - see the tutorial on https://www.froglogic.com/squish/features/recording-and-playback/. After recording, this will insert the step function into a file in the steps section of the test suite resources - often this is in a file that doesn't make sense so please move it to somewhere it does make sense. + +Although the recording is useful, it often produces brittle step functions, please make use of the utilities in the global scripts area to improve the robustness of the test and see [here](System-Testing-with-Squish) for some hints, tips and gotchas. + +## Running the tests + +You can run the whole test suite with the play button located above the test cases next to the new BDD test case button. You can also run test cases with the run button next to the case in the case list. You can run features and scenarios by right-clicking and pressing run. \ No newline at end of file diff --git a/doc/client/testing/System-Testing-with-Squish.md b/doc/client/testing/System-Testing-with-Squish.md new file mode 100644 index 000000000..6ec37c1cc --- /dev/null +++ b/doc/client/testing/System-Testing-with-Squish.md @@ -0,0 +1,220 @@ +# Squish + +## Squish Licensing Information & Contact Details + +we have one floating tester subscription and one floating execution subscription licence. The execution subscription is used by a build server to run tests, the tester subscription is used by us to develop tests. We can have at most one developer writing tests at a time, but we can all install the software on our machines. I don't yet know how the license system works e.g. if one developer forgets to close squish on their system, does it block everybody until they do? Or is there some timeout? We will have to experiment. See `license server` below for more details. + +## Set Up for local server + +1. Install SQUISH/JAVA WINDOWS from `\squish` +1. Read and accept the terms and conditions if still correct +1. Install download: + 1. Run download installer + 1. Floating license: Enter `control-svcs.isis.cclrc.ac.uk:49345` + 1. Licence Agreement: Read and accept the licence + 1. Uncheck `Test Center` from items to install, unless you really need it + 1. Select python version for script language (3.*) + 1. Installation folder: Choose a folder I chose `c:\tools` but `c:\squish` is used on the build server + 1. JRE or JDK: Select the JDK we are currently developing with + 1. Shortcuts: use defaults + 1. Start Menu: use defaults + 1. Accept default except for location which I suggest you change to c:\tools + 1. Install +1. Clone the current tests to your machine (dev is where I put mine) + ``` + git clone https://github.com/ISISComputingGroup/System_Tests_UI_E4.git + ``` +1. Open the test suites in the IDE + 1. Menu File -> Open Test Suite .. + 1. Open the root of the git clone you just made this will open all tests suites in the window +1. Set the Application under test (AUT) + 1. Ensure that the ibex_gui client has been built with build.bat + 1. Edit -> Server Settings -> Manage AUTs ... + 1. Select Mapped AUTs and click Add... + 1. Locate the executable from the built (built using the build.bat maven script) eclipse project (e.g. ibex-client in `ibex_gui\built_client\`) +1. To get access to global scripts right click in squish -> global scripts pane -> add -> global scripts directory (Found within System_Tests_UI_E4) and select the global scripts directory in the repository root. +1. If not running in Python 3 follow [these](https://kb.froglogic.com/squish/howto/changing-python-installation-used-squish-binary-packages/) instructions, and point to the default `python3` directory in the squish installation root. + +You may also need to install `psutil` and `mysql-connector-python==8.0.11` through the GUI (Edit -> Preferences -> PyDev -> Interpreters -> Python Interpreter then "Manage with pip") if running the experiment details tests. + +Once you have set up Squish via the steps above, you should be able to run a test suite to confirm everything is working. Note that you need the IBEX server running in the background, but not the client (which will be started by Squish when you run a test). + +## RDP to Server + +It is possible to remote desktop to the squish server but when you disconnect you must use the "Disconnect from RDP" shortcut on the desktop. To do this you must be an Admin on the desktop. + +## Setup For Build Server + +1. Install all the things needed for an instrument (Git, MySql, Java) +1. Install Jenkins build system but run it from a command line. +1. Add the script for running the command to the startup. +1. Install squish as above. +1. Add to `C:\Users\\AppData\Roaming\froglogic\Squish\ver1` the key `GlobalScriptDirs = "C:\\Jenkins\\workspace\\squish_ui_system_tests\\global_scripts"` +1. Add applications under test to the server setup as above using the IDE +1. Change the Application Behaviour to have a startup time of 120s. +1. Check that the global script directory has been set. +1. Switch off screen saver and power saving +1. Next remote desktop from another machine as the user stated in the password doc + - We have tried making this autologon but it get stuck at the policy screen + - We don't need VNC this seems to do the job without a problem +1. Then disconnect the session using the shortcut on the desktop +1. Leave the machine with an attached screen. I think this is needed to set the resolution when leaving remote desktop. + +## Creating a new Test Suite + +1. Click File -> New test Suite ... +1. New Test Suite: + 1. Create a sensible name, `suite__tests` + 1. Make the test suite path the same as the system testing folder +1. Language: Make sure it is python +1. Select the application as eclipse (the ibex client in E4 was called eclipse) +1. Finish +1. Edit the test suite settings (select test suite in test suites tab. Then click on icon with blue spanner) + 1. Edit Object Map to be `..\objects.map`. You may not be able to do this from the Squish client depending on your version, in which case you can directly edit `suite.conf` in `//suite__tests/` (it should say `OBJECTMAP=..\objects.map`) + +## System Testing The IBEX Script Generator with Squish BDD Tools + +The way we use Squish for testing the script generator is a bit different to the way we test the IBEX client. The method for testing is documented on the [System testing with Squish BDD](System-Testing-with-Squish-BDD) page. Some of the hints and tips from this page still apply e.g. using utilities such as `set_text_field`. + +## Creating a new Test + +A test contains one test case. + +1. Find the suite the test should be in +1. Click "Create new test case" (icon document with a plus in Test Suites tab) +1. Change name to `tst_` +1. A test suite should start: + ``` + # -*- coding: utf-8 -*- + import sys + import os + path = os.path.abspath(os.path.dirname(findFile("scripts", "test_running.py"))) + sys.path.append(path) + + from test_running import start_test + + + def main(): + + # Given application + start_test() + + + ``` + +## Writing tests + +Hints, tips and gotchas for writing tests: + +* Use `set_text_field` to set a text field because it deletes its contents before adding the new value. +* Use `menu` module to access menus because if a menu is interrupted then you want it to try again at the top level menu. +* Use `generate_config_name` to generate a config name so that it will be ignored by git and cleaned up by the system test +* If you open a dialogue capture it using a context manager. You could consider adding an option for OK and Cancel. +* If you need to select a perspective button the object picker will set it using the index (`occurrence`) however these buttons might change their positions it is better to select them based on their text. To do this: + 1. Open the object map. + 1. Click on the button object definition (you can use the search at the top to find it). + 1. In the properties tab change `occurrence` to `text` and the value to the text on the button. + 1. Save it and check it works by clicking highlight object; button should flash red. +* Often `test.compare` and `test.verify` in Squish provides logs that aren't very useful, please do add a `test.log` line to describe the error. + +## Other + +### Change Java that squish is using + +To change java that squish is using: + +``` +cd squish directory +"bin/squishconfig" --java="C:\Program Files\Java\jdk\jre\bin" +``` + +This fixes the issue: + +``` +"Internal Error: don't know where to log: Squish for Java has not been configured for the current user yet. Please configure the (Java Runtime Environment) used for executing the AUT (Application Under Test) in the Squish IDE via Edit > Preferences > Squish > .... (Or use `SQUISH_DIR/bin/squishconfig --java=path_to_jre`. Replace "path_to_jre" as required.) (Starting application)" +``` + +and also the error message: + +``` +java.lang.NoClassDefFoundError: com/froglogic/squish/swt/CompositeResolver + at com.froglogic.squish.swt.SWT.(SWT.java:922) + at org.eclipse.swt.widgets.Display.(Display.java:419) + at org.eclipse.swt.widgets.Display.(Display.java:481) + at org.eclipse.ui.internal.Workbench.createDisplay(Workbench.java:795) + at org.eclipse.ui.PlatformUI.createDisplay(PlatformUI.java:160) + at uk.ac.stfc.isis.ibex.e4.product.Application.start(Application.java:38) +``` +when starting the GUI. + +### Restart after a Power Cut + +After a power cut you will need to log into the machine via RDP and then disconnect using the shortcut on the desktop. + +### Diagnosing Error Screenshots + +Screen shooting on error should be turned on in `start_test` in `test_running.py`. The screen shots are placed on the squish server in `... Jenkins\workspace\System_Tests_Squish\suite_configuration_tests\\errorImages` the will only be from the last build. + +### Error in tests + +To tack the error we find in squish please add any errors you see to this chart. Remove the error when you think it is fixed: + +Frequency | Test | Error +---- | ----- | ------ +4 | `tst_can_add_edit_and_delete_block_to_current_config` | When getting blocks it failed to get all children of one of the components. `ValueError: need more than 0 values to unpack. ... tst_can_add_edit_and_delete_block_to_current_config\test.py: 73, instrument_blocks.py: 25` +2 | `tst_can_create_lots_of_blank_configs` | `RuntimeError: Error in activateItem() invocation: Menu not visible and/or enabled Called from: C:\Jenkins\workspace\System_Tests_Squish\suite_configuration_tests\tst_can_create_lots_of_blank_configs\test.py: 20` +2 | `tst_can_add_edit_and_delete_block_to_current_config` | `RuntimeError: Property read failed: exception: java.lang.reflect.InvocationTargetException () org.eclipse.swt.SWTException: Widget is disposed` `Called from: C:\Jenkins\workspace\System_Tests_Squish\suite_configuration_tests\tst_can_add_edit_and_delete_block_to_current_config\test.py: 74` +1 | `tst_user_names_can_be_set` | `LookupError: Object ':Experiment Details_Text' not found. Could not match properties: isvisible for object name: ':Experiment Details_Text' Called from: C:\Jenkins\workspace\System_Tests_Squish\suite_experiment_details_tests\tst_user_names_can_be_set\test.py: 19 C:\Jenkins\workspace\System_Tests_Squish\global_scripts\experiment_details.py: 19` + +## License server + +This was setup as per https://doc.qt.io/squish/setting-up-the-squish-floating-license-server.html on `control-svcs.isis.cclrc.ac.uk` in the directory `/usr/local/squish-licenceserver` the service is automatically started at boot time vis systemd, the file `squish-licenseserver.service` has the service details and is symbolically linked from the systemd `/etc/systemd/system` area. The log file is `/var/log/squish-licenseserver.log` and the service is running on the default port of 49345 + +To restart the licence server process use `sudo systemctl restart squish-licenseserver.service` on the licence server machine + +## Troubleshooting + +### Squish fails to run tests with an error in the C runtime library + +Error is: + +``` +Runtime Error! + +Program: C:\Squish\lib\_squishrunner.exe + +R6034 An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information. +``` + +This issue is due to us loading the `uuid` library in Python. This library loads a conflicted C runtime library and means tests aren't able to run completely correctly. + +Solution is to Rename `C:\Squish\python\msvcr90.dll` to `msvcr90_off.dll`, which removes the conflicting dependency version. See ticket [#4773](https://github.com/ISISComputingGroup/IBEX/issues/4773) for more details. + +### Squish fails to begin run + +Attempt to change your tcb file to a regular neutron tcb file and begin a run. You should see it beginning and then return to set up with the log message: `invalid tcb start - must be 0 not 5.00000 ns`. See the [DAE troubleshooting](/specific_iocs/dae/DAE-Trouble-Shooting) "invalid tcb start" section. + +### Squish Fails to Start the Application + +Look at the `Runner/Server` Log tab see if you can diagnose the problem. + +* `Unrecognized option: --add-reads=javafx.base=ALL-UNNAMED` probably running it through java8 it is on your path too high up. + - I fixed this by copying the java from the shares into the directory from which it runs so that it picks this up as the default when it runs. E.g. copy `...\Kits$\CompGroup\ICP\ibex_client_jre` to `C:\Instrument\Dev\ibex_gui\built_client\jre` (note the name change to jre). Running `build.bat` should now do that for you. + +### KeyError: `MYSQLPW` is missing + +To remedy this error set `MYSQLPW` to the root password in your environment variables when running the tests. If the Squish IDE has not been restarted before this you will need to close and re-open it before running the tests again. + +### No licence available + +If onsite/vpn you can access https://control-svcs.nd.rl.ac.uk/squish/squish_stats.txt - look for something like +``` +"clientAddress": "::ffff:a.b.c.d" +"licenseType": "tester" +``` +Then from a command windows do `nslookup a.b.c.d` to see machine name using licence. Currently licences should auto-expire after 12 hours anyway, so you may just need to wait. `licenceType` can be `tester` or `execution`, we have one of each type and `execution` is used by the jenkins squish test server (this licence type only allow running not editing of tests) + +### install new licence + +on `control-svcs` edit `/etc/squish-licence-server/licences/squish-licence.cfg` and paste in new licence details. Then + `service squish-license-server stop` and ``service squish-license-server start` \ No newline at end of file diff --git a/doc/client/testing/Test-naming.md b/doc/client/testing/Test-naming.md new file mode 100644 index 000000000..29c103a43 --- /dev/null +++ b/doc/client/testing/Test-naming.md @@ -0,0 +1,33 @@ +# Test naming + +When writing tests, it is important that the purpose and expected outcome of the test is clear from the name. This has several advantages: + +- It helps other developers debug failing tests +- It helps clarify your own thoughts as to the purpose of the test +- It defines the code itself by explicitly stating the expected interface. Ideally you should be able to reconstruct a functional class entirely from its tests +- And more + +It isn't expected that all tests will follow this guide, many tests pre-date this guide. Sometimes as well, these guidelines may prove overly restrictive or add unnecessary bulk to the test name. As ever, discretion is advised, but be clear why you're not following the guidelines if you choose not to. For example "'Initializes_ok' is fine as a test name because I'm just checking that I can initialize the class" would be a bad reason. + +There are many test naming formats out there, each with pros and cons, and each with people who get far too passionate about their personal favourite. We have opted to go with the GIVEN_WHEN_THEN format. + +Test names should take the following form: + +``` +GIVEN_[pre-conditions]_WHEN_[action]_THEN_[result] +``` + +GIVEN, WHEN, and THEN are in capitals with the rest of the test name in lower case, words are separated by underscores. + +In some cases, there are no preconditions, or the preconditions are truly trivial (be wary of jumping to that conclusion though). In those cases given may be omitted. + +Where possible don't include the method being tested name in the test name as that could change over time. + +Given that java methods usually use CamelCase, it is useful to tell CheckStyle to ignore the name format add a warning suppression to the top of the class: + +``` + @SuppressWarnings({ "checkstyle:magicnumber", "checkstyle:methodname" }) + public class GIVEN_my_test_isnt_written_in_camel_case_WHEN_test_is_viewed_in_eclipse_THEN_checkstyle_does_not_issue_warning { +``` + +It may be worth adding the magic-number suppression too depending on the type of tests. \ No newline at end of file diff --git a/doc/client/testing/Using-Mockito-for-Testing-in-the-GUI.md b/doc/client/testing/Using-Mockito-for-Testing-in-the-GUI.md new file mode 100644 index 000000000..a7b5ad110 --- /dev/null +++ b/doc/client/testing/Using-Mockito-for-Testing-in-the-GUI.md @@ -0,0 +1,220 @@ +# Mockito + +You should read the guide to testing in IBEX before reading this guide. + +This guide gives some basic advice on using Mockito for unit testing in IBEX. For more information on Mockito see http://mockito.org/. + +Please update this guide with tips or anything you find useful in Mockito. + +## Test Doubles + +Test doubles are objects that stand in for a real object, for the purposes of unit testing. Terminology varies but there are four usual types that are described: + +* Dummy - an object that is passed around but never used + +* Fake - a working implementation of a class with a cheat, for example an in memory database + +* Stub - an object that provides a canned answer to a method call + +* Mock - fake objects which know about which method calls they receive + +See [this article](http://martinfowler.com/articles/mocksArentStubs.html) for more information. Mockito mostly helps with Stub and Mock doubles. + +## Verifying Interactions + +``` + import static org.mockito.Mockito.*; + + // mock creation + List mockedList = mock(List.class); + + // using mock object - it does not throw any "unexpected interaction" exception + mockedList.add("one"); + mockedList.clear(); + + // selective, explicit, highly readable verification + verify(mockedList).add("one"); + verify(mockedList).clear(); +``` + +Here the List interface is mocked, and has some method calls made on it. The verify calls replace the usual assert calls in this unit test, and check the method calls were made. In this example it is trivial to see they are called. + +## Stubbing Method Calls + +``` + // you can mock concrete classes, not only interfaces + LinkedList mockedList = mock(LinkedList.class); + + // stubbing appears before the actual execution + when(mockedList.get(0)).thenReturn("first"); + + // the following prints "first“ + System.out.println(mockedList.get(0)); + + // the following prints "null" because get(999) was not stubbed + System.out.println(mockedList.get(999)); +``` + +This time the concrete class LinkedList is mocked instead of an interface. The mocked object returns what is asked of it when the method call is made with identical arguments. + +## IBEX Example + +Below is a less trivial example, showing how the verification and stubbing can be used to check behaviour of an observable. + +In this example InitialiseOnSubscribeObservable takes another observable as its argument, gets the current value of that observable, and listens for changes. Here we stub the class that InitialiseOnSubscribeObservable is observing, to simplify the test. The only method call we care about is `getValue()`. + +The InitialisableObserver is also mocked. As part of the test we want to check that it has its `update()` method called with a specific set of arguments. We use `times(1)` to specify we want the method called exactly once. + +``` + @Test + public void test_InitialiseOnSubscribeObservable_subscription() { + //Arrange + String value = "value"; + + // Mock observer, templated objects need cast + InitialisableObserver mockObserver = + (InitialisableObserver) mock(InitialisableObserver.class); + + // Mock observable with stub method + CachingObservable mockObservable = + (CachingObservable) mock(CachingObservable.class); + when(mockObservable.getValue()).thenReturn(value); + + // Object we are really testing + InitialiseOnSubscribeObservable initObservable = + new InitialiseOnSubscribeObservable<>(mockObservable); + + //Act + Object returned = initObservable.addObserver(mockObserver); + + //Assert + // The initialisable observer has its update method called once + verify(mockObserver, times(1)).update(value, null, false); + + // The InitialiseOnSubscribeObservable has the value returned from the mock observable + assertEquals(value, initObservable.getValue()); + + // A Unsubscriber is returned + assertEquals(Unsubscriber.class, returned.getClass()); + } +``` + +## Times Method is Called + +Options for checking how many times a particular method is called: + +* `atLeast(int minNumber)` at least this many times + +* `atLeastOnce()` at least once + +* `atMost(int maxNumber)` at most this many times + +* `never()` same as `times(0)` + +* `times(int number)` exactly this number of times + +## Any Methods + +When verifying method calls if the value of an argument is not important Mockito allows you to check that any object of a specific type was used as an argument instead. + +``` + // The initialisable observer has its update method called once + verify(mockObserver, times(1)).update(value, any(Exception.class), anyBoolean()); +``` + +For common types methods such as `anyString()` are available, otherwise `any(Object.class)` can be used. A null object will also be matched by using any. + +## Capturing Values on Method Calls + +If you want to capture the object called in a method, perhaps to check some value, then a captor can be used. See the code below for an example of how to do this. It is important to call `MockitoAnnotations.initMocks(this);` in the test set up method, otherwise the captor is never initialised. + +``` + @Captor ArgumentCaptor exceptionCaptor; + + @Before + public void setUp() { + // This is to initialise the exceptionCaptor + MockitoAnnotations.initMocks(this); + } + + @Test + public void test_ConvertingObservable_with_conversion_exception() throws ConversionException { + //Arrange + InitialisableObserver mockObserver = mock(InitialisableObserver.class); + + // initObservable is what our ConvertingObservable looks at, and testObservable we can call set methods on + TestableObservable testObservable = new TestableObservable<>(); + InitialiseOnSubscribeObservable initObservable = new InitialiseOnSubscribeObservable(testObservable); + + // Mock converter, with a stub conversion method + Converter mockConverter = mock(Converter.class); + when(mockConverter.convert(123)).thenThrow(new ConversionException("conversion exception!")); + + // Object we are really testing + ConvertingObservable convertObservable = new ConvertingObservable<>(initObservable, mockConverter); + + //Act + convertObservable.addObserver(mockObserver); + convertObservable.setSource(initObservable); + testObservable.setValue(123); + + //Assert + // The initialisable observer has its onError message called once, for the ConversionException + verify(mockObserver, times(0)).onValue(anyString()); + verify(mockObserver, times(1)).onError(exceptionCaptor.capture()); + assertEquals("conversion exception!", exceptionCaptor.getValue().getMessage()); + } +``` + +## Checking Order of Method Calls + +Mockito can be used to check the order methods were called in. + +``` + InOrder inOrder = inOrder(firstMock, secondMock); + + inOrder.verify(firstMock).add("was called first"); + inOrder.verify(secondMock).add("was called second"); +``` + +## Spies + +Spies can be used to stub a method or verify calls on a real class. Needing to use a partial mock like this might be a symptom of problems with code though! + +``` + // These are equivalent + @Spy Foo spyOnFoo = new Foo("argument"); + Foo spyOnFoo = Mockito.spy(new Foo("argument")); +``` + +## DB Tests using Answer Example + +In RdbWritterTests (C:\Instrument\Apps\EPICS\ISIS\IocLogServer\master\LogServer\src\test\java\org\isis\logserver\rdb) there is an example of using an answer to perform a more complicated return. The answer works like this: + +``` +when(mockPreparedStatement.executeQuery()).thenAnswer(resultAndStatement.new ResultsSetAnswer()); +``` +I have chosen to implement the answer class as an inner class of another class but you don't have to. The answer looks like: + +``` +public class ResultsSetAnswer implements Answer { + @Override + public ResultSet answer(InvocationOnMock invocation) throws Throwable { + openedResultsSet++; + return resultSet; + } + } +``` +The reason I am using answer here is to keep the number of times I opened a results set up to date so this answer stores that info in its parent class. + +## Tips and Advice + +* Use mocks to test interactions between a class and a particular interface + +* Use mocks to avoid unit tests touching complex or buggy dependencies + +* Do not mock type you don't own? Perhaps... + +* Do not mock simple classes or value objects - may as well use the real thing + +* Do not mock everything! diff --git a/doc/client/testing/coverage_result.png b/doc/client/testing/coverage_result.png new file mode 100644 index 000000000..8a6821cea Binary files /dev/null and b/doc/client/testing/coverage_result.png differ diff --git a/doc/client/testing/failed_test.png b/doc/client/testing/failed_test.png new file mode 100644 index 000000000..b159cc9e2 Binary files /dev/null and b/doc/client/testing/failed_test.png differ diff --git a/doc/client/testing/passed_test.png b/doc/client/testing/passed_test.png new file mode 100644 index 000000000..a3c14136c Binary files /dev/null and b/doc/client/testing/passed_test.png differ diff --git a/doc/client/troubleshooting/Debugging-memory-leaks-in-the-IBEX-GUI.md b/doc/client/troubleshooting/Debugging-memory-leaks-in-the-IBEX-GUI.md new file mode 100644 index 000000000..08b3bbfc4 --- /dev/null +++ b/doc/client/troubleshooting/Debugging-memory-leaks-in-the-IBEX-GUI.md @@ -0,0 +1,84 @@ +# Debugging memory leaks in the IBEX GUI + +This page is currently heavily based on https://github.com/ISISComputingGroup/IBEX/issues/2017. Please update this as we gain more experience! + +## Symptoms + +- Java memory leaks cause high CPU usage rather than the more intuitive high memory usage. The IBEX GUI is limited to a heap size of 1024MB by default. When the heap gets close to it's maximum size, the garbage collector goes crazy trying to recover memory wherever it can, and therefore uses a lot of CPU. + +- Because of the high CPU usage, the GUI will feel slow and sluggish. + +- When there is no more memory available, java will crash with an `OutOfMemoryException`. Note that this exception might not occur on the GUI thread, and therefore various components running in separate threads might fail before the GUI crashes. + +## Tools + +- Java visual VM (download the GPL-licensed binary from https://visualvm.github.io/download.html ) + - If you experience problems trying to run the standalone client where a pop up warning windows notifies you of a JRE problem, try first checking if you have the correct version of Java in your system environment variables and that no older/newer versions exist in addition to the correct version (removing if necessary). Secondly, try the following command next to the VisualVM executable you are having issues with: `visualvm.exe --jdkhome c:\Progra~1\Eclips~1\jdk-11.0.14.9-hotspot` changing the version of jdk where applicable to the version you have. +- Eclipse debugger +- A way to reproduce the issue + +## Setup + +- In the java visual VM, on the left, you should see an `Eclipse (pid xxxx)` process. This is either your Eclipse IDE or IBEX. If you're unsure which is which, look under "Threads" for anything to do with ActiveMQ: This will be IBEX. + +- Reproduce your issue, looking at the heap usage in the java visual vm. + +- A healthy GUI will show heap usage fluctuating quite a lot on a short-term basis, but with no long-term trend. + +- A GUI with a memory leak will also fluctuate in the short term, but will also have a long term trend upwards while the issue is being reproduced. + +## Setup to monitor a remote client + +- Look in the log file for the client that you want to remotely monitor + +- When the client first starts it will print `JMX url:` followed by a url that looks a bit like `service:jmx:rmi://machine/stub/rO0ABXNyAC5qYXZheC5tYW5hZ2V...` + +- Add a JMX connection in the java visual VM and copy in the whole url + +- **WARNING: using this interface you can do harm to the client and thus running experiments! Therefore you should only look at the monitoring graphs and use the sampler. For anything else you must inform the user that it may cause issues** + +## Diagnosis 1 + +- Under `Sampler -> Memory` and look at which objects are using up a lot of heap space. Note that this is only the object, **not** any of the objects that it owns! + +```java +public class MemoryLeak{ + RealMemoryLeak thisThingIsReallyBig = reallyBig; // Something enormous +} +``` + +will **not** make `MemoryLeak` show up as a large object! + +You may also find it useful to look at the instance count of suspicious classes. If some action in the GUI causes the instance count of a class to go up but not down again when closed, this is almost certainly a memory leak (but not necessarily a large memory leak - so it's possible that it's not the leak you're looking for). + +## Diagnosis 2 + +Once you have a few candidate objects from above which you think are suspicious, use the eclipse debugger to inspect them at runtime. If you find that they're very big / contain a lot of items, then your problem has been located. If not, then rinse and repeat Diagnosis 1. + +## Diagnosis 3 + +If you have objects which you think should be being garbage collected, but are not, then you can find out why they are being kept around using the Visual VM. +- Reproduce the issue (so that the GUI contains some objects which should have been released) +- Connect VisualVM +- Force a garbage collection by clicking the "GC now" button +- Under the "memory sampler" view, create a heap dump +- In the heap dump, search for the class that you're interested in +- Click the "GC root" button. This may take a few seconds, but will show you the reference chain which is keeping the object in memory +- You will now need to figure out where you need to "break" this chain, so that the object can be reclaimed. You may need to iterate this process several times if the object is referred to by multiple garbage collection roots. Eventually, once all reference chains from GC roots have been broken, the object should be eligible for GC. + +## Fixing + +Once you've diagnosed *where* the memory leak is, you need to fix it. This may not be obvious at first! There's not really any generic help that can be given here. + +## Notes + +- The java garbage collector is responsible for cleaning up dereferenced objects. If your objects aren't being freed when you click the `Perform GC` button in java visual VM, then you still have references to them somewhere. + * The java garbage collector uses reachability analysis, not reference counting, to test whether an object can be GC'd. The garbage collection "roots" are listed [here](https://www.ibm.com/support/knowledgecenter/en/SS3KLZ/com.ibm.java.diagnostics.memory.analyzer.doc/gcroots.html), but usually in IBEX these will be either statically reachable objects (e.g. singletons) or the current stack frame for each running thread. +- Some things need to be closed before/on garbage collection, for example sockets. Java's `finalize()` method is called once an object is eligible for GC, but before it is actually GC'd. Beware of using finalizers to perform "cleanup" - in some cases (e.g. closing observables), not running the cleanup will mean the object will still be reachable and so the finalizer will never be run! + +## Observables + +One common problem that we have repeatedly run into in IBEX is the use of observables. If an observer can be created multiple times, it is **very important** that it gets removed when no longer required. Otherwise, the observable will still have a reference back to the observer, which will prevent both the observer and anything which the observer references from being garbage collected. In other words, this will lead to a memory leak. The only case where is is acceptable for an observable to never be removed is if it is only created a small constant number of times and is necessary for the full lifetime of the client (for example, the instrument list). + +We have considered in the past making the reference from an observable to it's observers a `WeakReference`, however this does not work in many cases as the observable is often the only reference to the observer, meaning that the observer will be garbage collected and not work. + diff --git a/doc/client/troubleshooting/Java-Resource-Usage-Tools.md b/doc/client/troubleshooting/Java-Resource-Usage-Tools.md new file mode 100644 index 000000000..103e66af8 --- /dev/null +++ b/doc/client/troubleshooting/Java-Resource-Usage-Tools.md @@ -0,0 +1,46 @@ +# Java Memory and CPU Profiling + +## JVisualVM + +This is started from any java jdk in bin double click on jvisualvm.exe. You can then immediately look at the heap of your java application. + +You can then take a heap dump (button top right) and this can show where all the memory or instance have gone. (This can be extremely slow) + +You can also look at what functions are using what resources by using the sampler or the profiler and using CPU/memory sampling or profiling. + +### Setup JVisualVM to monitor a remote client + +- Look in the log file for the client (will be on the NDX at `C:\Instrument\Apps\Client_E4\workspace\logs`) that you want to remotely monitor + +- When the client first starts it will print `JMX url:` followed by a url that looks a bit like `service:jmx:rmi://machine/stub/rO0ABXNyAC5qYXZheC5tYW5hZ2V...` + +- Add a JMX connection in the java visual VM and copy in the whole url + +- **WARNING: using this interface you can do harm to the client and thus running experiments! Therefore you should only look at the monitoring graphs and use the sampler. For anything else you must inform the user that it may cause issues** + +## Memory Analyzer + +Installed in eclipse through the market place (search for this) + +Then edit your run configuration and include `-agentlib:hprof=heap=dump,format=b` in your VM arguments. When the program exists it produces a heap profile which the memory analyser can load and then find possible memory leaks in. + +## See also + +[Debugging memory leaks in the GUI](Debugging-memory-leaks-in-the-IBEX-GUI). + +## JStack + +JStack is a utility that comes bundled with a JDK. It can be used to perform post-mortem analysis of a crash dump generated by a java executable. It will give tracebacks of every thread at the time when the JVM crashed, which can help diagnose the root cause of a crash. + +Example of usage: + +``` +"C:\Program Files\Java\jdk1.8.0_192\bin\jstack.exe" -J-d64 ibex-client.exe hs_err_pidxxxxx.mdmp +``` + +To view "native" frames (frames inside DLLs), add the "-m" switch. + +Notes: +- The first argument should be the executable that generated the crash dump (i.e. it should be the released version of the client, not a dev version). The second argument is a minidump which the JVM will generate if there is a "hard" crash (e.g. a segmentation fault as opposed to a normal java exception). + +Full documentation available [here](https://docs.oracle.com/javase/7/docs/technotes/tools/share/jstack.html). diff --git a/doc/client/troubleshooting/PV-Manager-and-Observers-Logging.md b/doc/client/troubleshooting/PV-Manager-and-Observers-Logging.md new file mode 100644 index 000000000..dee6f1265 --- /dev/null +++ b/doc/client/troubleshooting/PV-Manager-and-Observers-Logging.md @@ -0,0 +1,116 @@ +# PVManager & Observers Logging + +To help debug the PVObserver and lower levels you can build a version of the GUI which includes logging and the code for PV manager. + +1. Checkout [csstudio repository](https://github.com/ISISComputingGroup/CSStudio_3_3) to somewhere accessible (probably c:\instrument\Dev) +1. Switch to the Logging_for_PV_connections branch + - NB the JCA and caj code has been copied from the caj and jca project to org.csstudio.platform.libs.epics and the jar deleted in this project. If you need to update this code create a new copy. +1. Switch the gui branch to Logging_for_PV_connections +1. Import all projects (including nested projects) from this directory into Eclipse +1. Clean and build - not all projects will build but all the ones needed should build. +1. Run with the following plugins from the workspace instead of the P2 repository. To do this select Run -> Run configuration. On the plug-ins tab type the plugin name and deselect target platform and select workspace: + - org.csstudio.logging + - org.csstudio.platform.libs.epics + - org.csstudio.utilities.pvmanager + - org.csstudio.utilities.pvmanager.epics +1. This will now print out debug statements all containing the text Ticket2162 +1. The logs can be filtered using a python program below (this program is very rough, don't forget to change the date) +1. This can be combined with Wireshark (see [Network traffic](/tools/Network-traffic)) to analyse traffic if needed + +Example log output from disconnection to reconnection + +``` +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: CAJChannel connect changegov.aps.jca.Channel$ConnectionState[DISCONNECTED=1] +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: MultiplexChannelHandler connect changeJCAConnection [connected: false writeConnected: false channel: CAJChannel = { name = TE:NDW1407:CS:SB:TEMP1, connectionState = DISCONNECTED }] +:NDW1407:CS:SB:TEMP1 = 13:52:47.265000: ConnectionCollector connect changefalse +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVDirector connected or exception event false +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVReaderImpl connection +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: PVManagerObservable connection false +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: DisplayBlock connection false +:NDW1407:CS:SB:TEMP1 = 13:52:47.266000: ChannelSearchManager Moved channel search to timer with delay 32 +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.276000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.309000: ChannelSearchManager Moved channel search to timer with delay 64 + all = 13:52:47.309000: ChannelSearchManager no work to do not restarting timer. period32 +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.319000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.366000: PVReaderImpl valueChange +:NDW1407:CS:SB:TEMP1 = 13:52:47.366000: PVManagerObservable valueChange false +:NDW1407:CS:SB:TEMP1 = 13:52:47.383000: ChannelSearchManager Moved channel search to timer with delay 128 + all = 13:52:47.383000: ChannelSearchManager no work to do not restarting timer. period64 +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.393000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.521000: ChannelSearchManager Moved channel search to timer with delay 256 + all = 13:52:47.521000: ChannelSearchManager no work to do not restarting timer. period128 +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.531000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:47.787000: ChannelSearchManager Moved channel search to timer with delay 512 + all = 13:52:47.787000: ChannelSearchManager no work to do not restarting timer. period256 +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:47.797000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:48.309000: ChannelSearchManager Moved channel search to timer with delay 1024 + all = 13:52:48.309000: ChannelSearchManager no work to do not restarting timer. period512 +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:48.319000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:49.343000: ChannelSearchManager Moved channel search to timer with delay 2048 + all = 13:52:49.343000: ChannelSearchManager no work to do not restarting timer. period1024 +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:49.353000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:51.401000: ChannelSearchManager Moved channel search to timer with delay 4096 + all = 13:52:51.401000: ChannelSearchManager no work to do not restarting timer. period2048 +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:51.411000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:52:55.508000: ChannelSearchManager Moved channel search to timer with delay 8192 + all = 13:52:55.508000: ChannelSearchManager no work to do not restarting timer. period4096 +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:52:55.518000: ChannelSearchManager request sent? true + all = 13:52:58.123000: ChannelSearchManager no work to do not restarting timer. period16384 +:NDW1407:CS:SB:TEMP1 = 13:53:03.711000: ChannelSearchManager Moved channel search to timer with delay 16384 + all = 13:53:03.711000: ChannelSearchManager no work to do not restarting timer. period8192 +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:03.721000: ChannelSearchManager request sent? true + all = 13:53:05.425000: ChannelSearchManager no work to do not restarting timer. period32 + all = 13:53:05.788000: ChannelSearchManager no work to do not restarting timer. period128 + all = 13:53:06.224000: ChannelSearchManager no work to do not restarting timer. period256 + all = 13:53:08.641000: ChannelSearchManager no work to do not restarting timer. period128 +:NDW1407:CS:SB:TEMP1 = 13:53:20.534000: ChannelSearchManager Moved channel search to timer with delay 32768 + all = 13:53:20.534000: ChannelSearchManager no work to do not restarting timer. period16384 +:NDW1407:CS:SB:TEMP1 = 13:53:20.830000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:20.830000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:53:20.831000: ChannelSearchManager request sent? true + all = 13:53:38.603000: ChannelSearchManager no work to do not restarting timer. period32 + all = 13:53:39.815000: ChannelSearchManager no work to do not restarting timer. period64 + all = 13:53:39.954000: ChannelSearchManager no work to do not restarting timer. period128 + all = 13:53:46.211000: ChannelSearchManager no work to do not restarting timer. period16384 + all = 13:53:50.802000: ChannelSearchManager no work to do not restarting timer. period256 +:NDW1407:CS:SB:TEMP1 = 13:54:10.818000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:10.818000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:10.819000: ChannelSearchManager request sent? false + all = 13:54:11.286000: ChannelSearchManager no work to do not restarting timer. period512 +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: CAJChannel generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: Search requests generate search request +:NDW1407:CS:SB:TEMP1 = 13:54:43.587000: ChannelSearchManager request sent? true +:NDW1407:CS:SB:TEMP1 = 13:54:43.589000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 22 +:NDW1407:CS:SB:TEMP1 = 13:54:43.589000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 18 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 22 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAResponseHandler id (18 is connection, 23 response, 1 reconnect) 18 +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAJChannel connect complete changegov.aps.jca.Channel$ConnectionState[DISCONNECTED=1] +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CATransport resubscribe subscriptions request +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CATransport resubscribe subscriptions request +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: CAJChannel connect changegov.aps.jca.Channel$ConnectionState[CONNECTED=2] +:NDW1407:CS:SB:TEMP1 = 13:54:43.591000: MultiplexChannelHandler connect changeJCAConnection [connected: true writeConnected: false channel: CAJChannel = { name = TE:NDW1407:CS:SB:TEMP1, connectionState = CONNECTED }] +``` + +## Python to analysis the logs: + +See `...ibex_utils\ibex_log_parser\parse.py` diff --git a/doc/client/troubleshooting/Pydev-autocomplete-issue.md b/doc/client/troubleshooting/Pydev-autocomplete-issue.md new file mode 100644 index 000000000..cd9280a8a --- /dev/null +++ b/doc/client/troubleshooting/Pydev-autocomplete-issue.md @@ -0,0 +1,37 @@ +# PyDEV Autocompletion + +## Summary + +We have re-enabled the pydev autocomplete, which is the autocomplete in the ibex GUI scripting window. This had been disabled for a while after we discovered a problem, but it is not an easy fix and the problem may actually have existed for a much longer time than we realised. + +The problem only occurs with a "chained autocomplete". For example if you type `g.` or `b.` to autocomplete a command or block name then that works fine. The problem occurs if you try and autocomplete further on an already completed command. For example if you type `g.c` you will be able to complete a `g.cset` command with no problems. However if after having completed this as `g.cset("ablock", 1)` you then typed a further `.` (i.e. `g.cset("ablock", 1).`) you would end up running the `cset` command continuously. This is because the command line needs to run the function to find out what type of object it returns so as to be able to display the list of new options to complete on. So the bottom line is that simple single autocompletes will not give any problems, but be aware that autocompleting on the result of a command/function will cause the command/function to run continuously which may have undesirable side effects. This does not apply only to genie python functions, it applies to any python function. For a simple example if you type `print("hello")` in the scripting window and then press `.` rather than the enter key, it will print hello continuously as it needs to run the print function to get the autocomplete choice. + +## Further details and history + +We found an issue in the `pydev` plugin, used for the embedded scripting console in the GUI, in https://github.com/ISISComputingGroup/IBEX/issues/7604 which basically ran a function on autocomplete continuously. An example of this is like so: + +```python +def x(): + print("hello") +``` + +if you then used `x().` in a scripting console this would run in a never-ending loop and print "hello" continuously. + +This was alarming as we have several functions in our genie_python library which could be called this way by accident and this could have done bad things ie. move motors or set temperatures without the user realising. + +It was decided after investigation that we should [disable this for now](https://github.com/ISISComputingGroup/Pydev/pull/4), [triangulate](https://github.com/ISISComputingGroup/IBEX/issues/7850) when it started happening, then look into a solution. This led to [some investigation](https://github.com/ISISComputingGroup/IBEX/issues/7898) and we discovered that: +a) this had been happening since the introduction of pydev in the GUI, though no one other than us had actually noticed - more on this later +b) this problem only happens in pydev's console view, which has no form of dynamic analysis of the code put into it + +After talking to the maintainer and doing several days work into looking into the code, we decided that the best course of action was to re-enable for now on instruments and let instrument scientists know that it may cause issues. The fix isn't trivial due to the lack of dynamic analysis, but the continuously doing the action is a definite bug. + +## How to enable or disable auto completion +Pydev on a built GUI client is stored as several jars in the `plugins\` directory. The one we care about is called `org.python.pydev.ast_XXX` where `XXX` is the version suffix. The jar contains lots of bytecode-compiled java classes which are not directly editable. + +In order to disable autocompletion on an instrument, you will need to (in `\instrument\apps\client_e4\plugins\org.python.pydev.ast_xxx`) + +- rename `ast.jar` to `ast.jar.ACENABLED` +- copy `ast.jar` from `\\shadow\ICP_P2$\Pydevup_042023\plugins\org.python.pydev.ast` + +_Note: this will get reverted when we deploy a new GUI, so we may have to keep doing it until the underlying issue is solved._ + diff --git a/doc/client/troubleshooting/Slow-reconnection-in-the-GUI.md b/doc/client/troubleshooting/Slow-reconnection-in-the-GUI.md new file mode 100644 index 000000000..ac3f3f21c --- /dev/null +++ b/doc/client/troubleshooting/Slow-reconnection-in-the-GUI.md @@ -0,0 +1,39 @@ +# Slow PV Reconnections + +This document is based on the findings of [#1971](https://github.com/ISISComputingGroup/IBEX/issues/1971). + +## Running on the local machine (NDXDEMO) + +IOC off for between 12 and 24 hours typically gives the following results: + +Camonitor connects after <5 seconds +Vanilla CS-Studio 3 and 4 OPI < 30 seconds (only one PV on OPI) +IBEX OPI ~1 minute + +The different fields on the IBEX OPI reconnect at different times. + +## Running on the local machine (my PC) + +Similar except the IBEX OPI can take significantly longer sometimes (> 40 minutes!) + +Could this be related to excessive IOC log production on my machine? + +## Client running on a different machine to NDXDEMO + +The client and camonitor can take a long time to reconnect, anything from a few seconds to 5 minutes. +Disabling the gateway on NDXDEMO and running a client/camonitor on the same subnet (i.e. no gateways involved) connects much quicker. +Note: this was for a spectrum data PV. + +## Conclusions + +ISSUE 1: Something about the gateways means that remote clients don't respond well to IOCs turning off then back on. Though starting a new client usually connects straightaway. + +ISSUE 2: IBEX’s OPIs take longer to reconnect than vanilla CS-Studio does. The longer the IOC is off the worse it tends to be. + * Is it because IBEX is doing a lot generally, e.g. logs, blocks, etc? + * Could it be that the PVManager has connected fine, but the GUI is not being updated immediately because the GUI thread is busy? Once the PV has been connected to the value updates come through right away though. + * Or is it something else? + +Anecdotal evidence suggests that the "busier" IBEX is the longer it takes to reconnect. +Busier means more logs, blocks etc. - this could relate to the amount of memory being used by JVM. + +BTW: Blocks and OPI values are inconsistent on which comes back first. \ No newline at end of file diff --git a/doc/conf.py b/doc/conf.py new file mode 100644 index 000000000..d2c168070 --- /dev/null +++ b/doc/conf.py @@ -0,0 +1,83 @@ +# Configuration file for the Sphinx documentation builder. +# +# For the full list of built-in configuration values, see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Project information ----------------------------------------------------- +# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information + + +project = "IBEX Developer's Manual" +copyright = "" +author = "ISIS Experiment Controls" +release = "0.1" + +# -- General configuration --------------------------------------------------- +# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration + +nitpicky = True + +myst_enable_extensions = [ + "dollarmath", + "strikethrough", + "colon_fence", + "linkify", + "html_image", + "attrs_block", +] +suppress_warnings = ["myst.strikethrough", "myst.header"] + +extensions = [ + "myst_parser", + "sphinx.ext.autodoc", + # and making summary tables at the top of API docs + "sphinx.ext.autosummary", + # This can parse google style docstrings + "sphinx.ext.napoleon", + # For linking to external sphinx documentation + "sphinx.ext.intersphinx", + # Add links to source code in API docs + "sphinx.ext.viewcode", +] +napoleon_google_docstring = True +napoleon_numpy_docstring = False + +templates_path = ["_templates"] +exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] + +# -- Options for HTML output ------------------------------------------------- +# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output + +html_context = { + "display_github": True, # Integrate GitHub + "github_user": "ISISComputingGroup", # Username + "github_repo": "ibex_developers_manual", # Repo name + "github_version": "master", # Version + "conf_py_path": "/doc/", # Path in the checkout to the docs root +} + +html_title = "IBEX Developer's Manual" +html_short_title = "Dev Manual" +html_theme = "sphinx_rtd_theme" +html_logo = "logo.svg" +html_theme_options = { + "logo_only": False, + "style_nav_header_background": "#343131", +} +html_favicon = "favicon.ico" +html_static_path = ["_static"] +html_css_files = [ + "css/custom.css", +] + +autoclass_content = "both" +myst_heading_anchors = 7 +myst_linkify_fuzzy_links = False +html_last_updated_fmt = "" +html_show_copyright = False + +spelling_lang = "en_GB" +spelling_filters = ["enchant.tokenize.MentionFilter"] +spelling_warning = True +spelling_show_suggestions = True +spelling_suggestion_limit = 3 diff --git a/doc/deployment/Creating-a-release.md b/doc/deployment/Creating-a-release.md new file mode 100644 index 000000000..8e319ce23 --- /dev/null +++ b/doc/deployment/Creating-a-release.md @@ -0,0 +1,161 @@ +# Creating a release + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +release/* +``` + +## Understanding Java Licensing +Make sure you [understand how Java is licensed](/processes/dev_processes/Understanding-Java-Licensing), so that we do not inadvertently make ISIS liable for licensing fees. + +## Release tickets + +Text for release ticket: + +> As a developer I want an IBEX release so I can install it on the instrument machines for the start of machine physics on 26th. +> +> Acceptance Criteria: +> +> - [ ] [Pre Testing Steps](#creating_release_pre_testing_steps) +> - [ ] [Testing steps](#creating_release_testing_steps) +> - [ ] [Post Testing](#creating_release_post_testing_steps) +> - [ ] [Create a ticket to update dependencies to latest versions and test](/processes/dev_processes/Dependency-Updates) +> - [ ] [Create a ticket to give instrument demos](https://github.com/ISISComputingGroup/IBEX/wiki/Timetable-for-Instrument-Demos) +> - [ ] Distribute release notes via email [Instrument scientist release email](#updating_instrument_new_release_email_template) + +## Standard Release + +Project is ready to be released not for a specific event, e.g. at the end of a sprint. Script Generator version number always differs to the client's. + +{#creating_release_pre_testing_steps} +### Pre Testing + +1. Contact computing group to let us know of the upgrade. Find out anything that needs to be in the release that isn't and mark with `for release` label. (This does not prevent a release) +1. Look at the released features in this branch [Upcoming Changes](https://github.com/ISISComputingGroup/IBEX/blob/master/release_notes/ReleaseNotes_Upcoming.md) and find the most significant level of change (i.e. is this cumulatively a major change, a minor change, or a patch?). +1. Update the [upgrade script](https://github.com/ISISComputingGroup/EPICS-upgrade/blob/master/upgrade.py) to include the latest version (this is done on master). Steps to do this are in [Config Upgrader in section *creating a production upgrade script*](/tools/Config-Upgrader) + 1. After committing these changes to `master` on the `EPICS-upgrade` submodule, don't forget to push the new submodule version to `master` on the top `EPICS` branch. This is needed to make sure you changes appear on the release branch created in the next step. +1. For packages which are published on `PyPI`, in particular `genie_python` and `ibex_bluesky_core`, create PyPI releases if needed. + 1. These packages are released by performing a `git tag x.y.z` on a checkout of `main`, where `x.y.z` is the new version you want to release. Push the tag using `git push origin tag x.y.z`. They will then build in github actions, and prompt to "approve" the release pipeline to PyPI. + 1. `genie` should be released with a version number matching the main IBEX version number. The release takes around 15 minutes in GHA. + 1. `ibex_bluesky_core` should generally have a minor/patch version incremented for now until we reach v1. The release takes a couple of minutes in GHA. + 1. **Ensure these releases were successful** by checking on [pypi](https://pypi.org/) for the new release number before proceeding. +1. Start the Jenkins pipeline [Release Branches](https://epics-jenkins.isis.rl.ac.uk/job/Release_branches/). + - Click on 'Build with Parameters'. + - Set `VERSION` to the new release version (e.g. `X.x.m`). + - Set `TAG` if you wish to branch off a commit other than the latest top level `HEAD`. If you do branch off an earlier commit, also set `REMOTE` to `false` as it now does not make sense to verify if you are on the latest submodule versions. + - Check `REMOTE` if the `EPICS` submodules should be checked for later versions on their remote - the script will fail if there are submodule commits unpushed to top level. For a normal release you will be expecting all submodules to be on the latest version. If you want the currently pinned (not necessarily latest) versions, do not check REMOTE. If you should/expect to be using the latest versions of all dependent submodules, check REMOTE box to verify this. If there are unpushed submodules the `EPICS repo checks` Jenkins build will likely be in error already. + - The script will then (as selected): + - Create the release branches (named `Release_X.x.m` except `Release_Script_Gen_X.x.m` for script generator) for: + - `EPICS` and submodules. + - `IBEX GUI`. + - `Script Generator`. + - `Uktena`. + - `JSON_bourne`. (not none by default as it does not often change and is also not directly deployed to instruments) + - Update or add version numbers: + - In `ioc/master/INSTETC/INSTETC-IOC-01App/Db/svn-revision.db.tmpl` for `EPICS`. + - In `/uk.ac.stfc.isis.ibex.e4.client/pom.xml` and `/uk.ac.stfc.isis.ibex.e4.client/META-INF/MANIFEST.mf` for `IBEX GUI`. + - In `/uk.ac.stfc.isis.scriptgenerator.client/pom.xml` and `/uk.ac.stfc.isis.scriptgenerator.client/META-INF/MANIFEST.mf` for `Script Generator`. + - Note: `genie_python` library version numbers are set automatically from the git tag, and no longer need manually updating. + - Push these changes to remote release branch. + - Start the Jenkins builds (click `Scan Repository` on the ones that are multibranch pipelines): + - `EPICS_release`. (Build will be in `Kits$\CompGroup\ICP\Releases\X.x.m\EPICS`) + - `EPICS_release32`. (Build will be in `Kits$\CompGroup\ICP\Releases\X.x.m\EPICS32`) + - `uktena_release_pipeline`. (Build will be in `Kits$\CompGroup\ICP\Releases\X.x.m\genie_python_3`) + - `ibex_gui_releases_pipeline`. (Build will be in `Kits$\CompGroup\ICP\Releases\X.x.m\Client`) + - `scriptgenerator_release`. (Build will be in `Kits$\CompGroup\ICP\Releases\script_generator_release\X.x.m`) +1. Move the `Upcoming Release Notes` page (copy & paste) into a new blank `Release Notes "X.x.m"` file: Check that all of the merged tickets have also had their [release notes merged](https://github.com/ISISComputingGroup/IBEX/pulls) then move the changes which have been merged into the new release from the [upcoming page](https://github.com/ISISComputingGroup/IBEX/blob/master/release_notes/ReleaseNotes_Upcoming.md) to a new release notes page for the version, and commit this change to the master branch. Check that the release notes are as understandable as possible. +1. Create a released version entry in the [releases table](https://github.com/ISISComputingGroup/IBEX/blob/master/docs/all-releases.md) (including link to release notes) and commit to master. +1. Update the "Latest Stable Release" link on the [IBEX wiki homepage](https://github.com/ISISComputingGroup/IBEX) to be the new `"Release X.x.m"` and commit to master. +1. Remove all entries from `Upcoming Release Notes`, leaving a blank file with only the headers, e.g. "Instrument Specific Changes", etc. and commit to master. +1. If applicable, update the dependencies since the last release and add them to the bottom of the release notes. To find the python dependencies list, run a `pip freeze` on a cleanly released `uktena`. Note that you will need to specify the scripts directory to run pip commands. i.e. `C:\Instrument\Apps\Python3\Scripts\Pip.exe freeze` +1. Update the [user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki) with any relevant changes +1. copy the release to `control-svcs`, this is so we can set git remotes for hotfixes etc. as part of the deploy. So if release is number 1.2.3 + - `robocopy "Kits$\CompGroup\ICP\Releases\1.2.3\EPICS\.git" "\\control-svcs.nd.rl.ac.uk\git$\releases\1.2.3\EPICS.git" /mir /nfl /ndl` + - edit `EPICS.git\config` on the control-svcs version (you can browse straight to the `\\control-svcs` share above) + - change `bare = false` to `bare = true` + - Add an extra section at end of file +``` +[http] + receivepack = true +``` + +1. Do the same for the `EPICS32` 32-bit build, replacing `EPICS` with `EPICS32` everywhere in the commands above. +1. Check release is now listed in [`https://control-svcs.isis.cclrc.ac.uk/git/?a=project_list;pf=releases`](https://control-svcs.isis.cclrc.ac.uk/git/?a=project_list;pf=releases) + +{#creating_release_testing_steps} +### Testing + +Using GitBash, update the `experiment controls public share` has the most recent version of `ibex_utils` from Git (i.e. do git pull) - this is so the most recent install script will be used for testing and install. In git bash you can do a `cd` with the windows path but just change `\` to `/` e.g. `cd //isis/somewhere`. When you do a git operation it may warn about directory ownership, just follow the command it suggests to add an exception and then git try again. You should: +* do a `git branch` and check it is on `master` and not e.g. a test or ticket branch +* do a `git status` and check for any changes, if there are post on Teams +* do a `git pull` so you are on the latest `master` branch with no local changes + +One or more people should do [manual system tests, using this page](Manual-System-Tests). + +{#creating_release_post_testing_steps} +### Post Testing + +These steps should only be done once all changes to a release have been made and we are ready to deploy. Here we are creating a tag +which will become inconsistent if further changes are made to the release branch. Hence it is important to delete +the relevant release branch after it has been tagged. + +1. Create a release tag in the EPICS, ibex_gui, uktena and JSON_bourne repositories. For each repo + 1. Go to `[REPO_URL]/releases`, e.g. `https://github.com/ISISComputingGroup/ibex_gui/releases` + 1. Click `Draft a new release` + 1. Enter the tag version in the format `vX.x.p` and target the release branch + 1. Enter the title `Release version X.x.p` + 1. Add a link to the release notes in the description + 1. **Delete the release branch** once the release and tag has been created. +1. Create release tag from the release branch for each submodules in EPICS, then delete the release branch. To do this, run the following two git commands in top level EPICS (replace `X.x.x` with the release number): + 1. `git submodule foreach --recursive "git fetch && git tag Release_ibex_X.x.x origin/Release_X.x.x || exit 0"` // Create tag + 1. `git submodule foreach --recursive "git push origin tag Release_ibex_X.x.x && git push --delete origin Release_X.x.x || exit 0"` // Push tags and delete release branch + + _Note: you may need to run `git config fsck.badEmail ignore` for the above step_ +1. Make sure any changes on the release branch are merged back onto master for EPICS, ibex_gui, genie_python, and JSON_bourne (except version numbering) +1. Consider which instruments need this release: + * Breaking release: upgrade everyone + * Big improvement: upgrade everyone if there is a big improvement that everyone will benefit from + * Standard release: upgrade instruments that need updates, i.e. they need a newly released feature, and all those that are in the current release group, see [column in instruments table](https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information). Note on the release ticket which instruments need to be released to using checkboxes (one for start and one for finish). +1. Deploy a new JSON_bourne if required see [here](/webdashboard/Web-Dashboard) +1. [Create repository for recording instrument changes post release](release/Release-based-repository) the deploy script automatically puts the instrument onto a branch of this repository +## Partial Release +For any release in which GUI version increments but server version does not, ensure the previous server version is added to the release folder via symbolic links or junctions, [see this ticket](https://github.com/ISISComputingGroup/IBEX/issues/7250). + +## Patch Release + +The is a release when a change needs to be made between standard releases; i.e. a standard release is inappropriate because it includes lots of code which has not been through a code freeze and test. + +1. Follow all Pre Testing steps above except: + * For step 4 (creating the release branch) instead of creating from master create it from the latest release tag + * Update GUI Java JRE + * Finding the most significant change + * Updating the MySQL and Java being deployed +1. Is the code for the bug/enhancement already written? + * No: Make the changes on the branch. + * Yes: Cherry pick commits needed from the branch with the written code +1. Test +1. Release as per a standard release but + 1. Use this branch instead of master + 1. Only include the change you have made in the release notes + 1. Copy the release notes issues into the development page +1. Merge the new code back into master + + +## Adding late commits to the release Branch for EPICS + +1. Test and merge the change into master +1. Create a release branch in the submodule (if it doesn't exist) + 1. Navigate to EPICS in github + 1. navigate to the submodule + 1. Create new branch (this means it will be branched from the release point) +1. Switch the branch of the submodule: `git checkout Release_X.x.m` +1. Pull the latest release branch: `git pull` +1. Merge in the ticket `git merge XXX` +1. push that branch +1. cd to EPICS +1. checkout release branch and pull +1. update submodule reference, commit and push. diff --git a/doc/deployment/Deploy.md b/doc/deployment/Deploy.md new file mode 100644 index 000000000..cca92228a --- /dev/null +++ b/doc/deployment/Deploy.md @@ -0,0 +1,22 @@ +# Deployment + +```{toctree} +:glob: +:maxdepth: 1 + +deploy/Updating-Instrument-Machines +deploy/Deployment-on-an-Instrument-Control-PC +deploy/Configure-Mini-Inst +deploy/Client-Release-Tests +deploy/Server-Release-Tests +deploy/IBEX-Deployment-on-Training-machine +deploy/Deploy-script-workflow +deploy/Autostart-IBEX-on-Computer-Restart +deploy/Making-an-Instrument-Available-from-the-GUI +deploy/Creating-a-new-branch-for-IBEX-configurations +deploy/Reviewing-Deploy-Tickets +deploy/Upgrade-ISISICP +deploy/Upgrade-Java +deploy/Installing-and-Upgrading-MySQL +deploy/NPort-install +``` \ No newline at end of file diff --git a/doc/deployment/Future.md b/doc/deployment/Future.md new file mode 100644 index 000000000..18239e0f9 --- /dev/null +++ b/doc/deployment/Future.md @@ -0,0 +1,13 @@ +# Future Workflows + +```{warning} +The workflows in this tree are **not currently in use**. They document steps towards future deployment workflows, which have +not yet been fully implemented. It is not currently possible to deploy IBEX using these workflows directly. +``` + +```{toctree} +:glob: +:maxdepth: 1 + +future_workflows/* +``` \ No newline at end of file diff --git a/doc/deployment/Manual-System-Tests.md b/doc/deployment/Manual-System-Tests.md new file mode 100644 index 000000000..0134d7383 --- /dev/null +++ b/doc/deployment/Manual-System-Tests.md @@ -0,0 +1,26 @@ +# Manual system tests + +Manual system tests should be performed as part of the release cycle and should be performed once a branch has been created. See [Creating-a-release](Creating-a-release) + +1. Create a Github project to track the testing + 1. Go to [the group projects](https://github.com/orgs/ISISComputingGroup/projects?query=is%3Aopen) + 1. On the `...` menu alongside `Manual System Tests Template` select `Make a copy`. + 1. Check the `Draft Issues will be copied` checkbox + 1. Name the file as per `VXX.xx.xx Manual System Tests` +1. Choose an instrument to test on (local or remote) and install the test version + 1. Navigate to an `ibex_utils` folder (either locally or in the `ISIS_Experiment_Controls_Public` share) and pull the latest changes from git + 1. Run `installation_and_upgrade\instrument_deploy.bat` + 1. Follow the instructions on the command line interface. If you are unsure whether a step needs to performed, ask someone from the team. + 1. [DEMO only] run `create_icp_binaries.bat` in the EPICS directory. If the instrument never makes it out of "Processing" with the ISISDAE IOC throwing errors that read "CoCreateInstanceEx (ISISICP) : The system cannot find the path specified", this is how you fix it. +1. Run through the tests in the project. Set the status to `In Progress` as you start it, then `Pass`, `Fail`, `Change` as appropriate. +1. After the testing is complete review those shown on the Failures and Change tabs. + 1. If the issue can be fixed now (e.g. a documentation error) do so. + 1. If the issue doesn't need fixing, then it can be ignored. + 1. If the issue should be fixed, but requires a new release being built: + 1. Edit the title to read `FAILURE: VXX.x.x: [test title]` or `CHANGE: VXX.x.x: [test title]` as appropriate. + 1. Use the `Convert to Issue` option on the ticket and add it to the issues in IBEX. + 1. If it's a failure and needs to be fixed immediately add it to the appropriate project boards (the PI board with the current sprint as the selected one, and the IBEX project board in the IBEX repo) with a `for release` label. + 1. If the issue should be fixed, but can wait until the next release + 1. Edit the title to read `FAILURE: VXX.x.x: [test title]` or `CHANGE: VXX.x.x: [test title]` as appropriate. + 1. Use the `Convert to Issue` option on the ticket and add it to the issues in IBEX. + 1. Add it to the PI board with the next sprint selected. \ No newline at end of file diff --git a/doc/deployment/Patch.md b/doc/deployment/Patch.md new file mode 100644 index 000000000..39f7af572 --- /dev/null +++ b/doc/deployment/Patch.md @@ -0,0 +1,14 @@ +# Patching + +How to patch or hotfix a new change onto an instrument which already has IBEX. + +```{toctree} +:glob: +:maxdepth: 1 + +patch/* +``` + +:::{seealso} +See also [powershell remoting](/systems/inst_control/PS-Remote) for a script to apply patches to many instruments simultaneously. +::: diff --git a/doc/deployment/deploy/Autostart-IBEX-on-Computer-Restart.md b/doc/deployment/deploy/Autostart-IBEX-on-Computer-Restart.md new file mode 100644 index 000000000..da6ee93f4 --- /dev/null +++ b/doc/deployment/deploy/Autostart-IBEX-on-Computer-Restart.md @@ -0,0 +1,27 @@ +# Auto-start IBEX on Computer Restart + +## Windows Server 2012 R2 + +**This appears to work but doesn't because the user does not have interactive mode permissions. There is a ticket to sort this out** + +To make the ibex server restart when the computer restarts, do the following: + +1. Ensure that the user who will run IBEX server has "log on as batch job" rights + 1. Open "Local Security Policy" as an admin + 1. Find "Local Policies" -> "User Right Assignments" click + 1. Find "Log on as a batch Job" + 1. Open and add the user above if they are not in this list +1. Create a scheduled task to start on reboot + 1. Open "Task Scheduler" as an admin + 1. Create a basic task + 1. Call in "Start IBEX on restart" + 1. Trigger: Trigger on "When the computer starts" + 1. Action: Start a program + 1. Start a program: + 1. Program script is `cmd` + 1. Add arguments is `/c "C:\Instrument\Apps\EPICS\start_ibex_server.bat 2>&1 > C:\Instrument\var\logs\startup.log` + 1. StartIn `C:\Instrument\Apps\EPICS` + 1. Finish: Click "Open the Properties dialog for this task when I click Finish" + 1. Enter credentials for user running ibex server + 1. General Tab: Make sure Run whether user is logged on or not + 1. Check other tabs diff --git a/doc/deployment/deploy/Client-Release-Tests.md b/doc/deployment/deploy/Client-Release-Tests.md new file mode 100644 index 000000000..2eab702bb --- /dev/null +++ b/doc/deployment/deploy/Client-Release-Tests.md @@ -0,0 +1,14 @@ +# Client release tests + +These are tests/checks that should be performed on the IBEX client after a release. They are split into those things that *must* be done and those that should be done (the only reason for not doing these is if the instrument scientists say it is ok not to perform these tests). These tests are things which are important for the running of the instrument and things which we have missed in previous releases which should not be missed again. + +## Tests that must be done + +### All Instruments + +1. (After installing both the server and the client) Start the IBEX client. Connect to an instrument. Check Menu -> Help -> About. The client number should be the version you just released and should match that on the page (https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information) +1. Make sure genie python works both from the GUI and when launched from the `genie_python.bat` shortcut under `C:/Instrument/Apps/Python` + 1. Verify that no issues arise on starting a scripting window (e.g. `init_[INST]` is loaded correctly) + 1. Genie is available via the `g` prefix: try `g.get_blocks()` + 1. Instrument scripts are available via the `inst` prefix. **DO NOT** use one of the existing instrument scripts as it could trigger unwanted changes on the instrument. Create a test script in `C:\Instrument\Settings\config\NDX[INST]\Python\inst` and run that. +1. View the current config make sure it looks sensible. diff --git a/doc/deployment/deploy/Configure-Mini-Inst.md b/doc/deployment/deploy/Configure-Mini-Inst.md new file mode 100644 index 000000000..9e6126d00 --- /dev/null +++ b/doc/deployment/deploy/Configure-Mini-Inst.md @@ -0,0 +1,39 @@ +# Configure Mini Inst + +To install a mini inst, you need to install the EPICS backend and add the following in the `\NDXxxxxx\configurations\` area. + +- `startup.txt`: contains a list of the IOCs to start (use the IOC name as in the IOCs list) +- `globals.txt`: add macro values you want the IOCs in `startup.txt` to start up with. [To do this, add a line per macro of the format `__=`](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Create-and-Manage-Configurations#editing-a-global-setting) + +### Startup and shutdown + +You can start up the mini inst using the normal `start_ibex_server` and stop it with `stop_ibex_server`. It will look for the `startup.txt` and start up the mini inst instead of the full IBEX server if it exists. + +### Updating config + +If you update a configuration in [globals.txt](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Create-and-Manage-Configurations#editing-a-global-setting) then don't forget to restart the IOC. This can be done either by running `start_ibex_server` but it is more efficient to console to the IOC and restart it using `ctrl+x`. + +### Adding Devices + +If you want to run an IOC in a mini-inst then: + +1. Add the device name to the "startup.txt" (`C:\Instrument\Settings\config\NDX\configurations\startup.txt`) + - e.g. JSCO4180_01 +1. Then set macros in "globals.txt" (`C:\Instrument\Settings\config\NDX\configurations\globals.txt`) + - e.g. JSCO4180_01__PORT=COM13 +1. Run `make iocstartups` in the epics terminal. This adds the device to the allowed running devices. +1. Start IBEX server as normal `start_ibex_server.bat` +1. If you are running the IOC to allow a Vi in SECI to communicate then configure that vi (usually the pv prefix) + - e.g. Run `C:\LabVIEW Modules\Drivers\Jasco PU-4180 HPLC Pump\Jasco HPLC Pump - System.llb` - Initialisation File change IOC Root to `IN:CRISP:JSCO4180_01:` +1. Run the Vi + + +### Running `ARINST` and/or `ARACCESS` in a mini-inst + +ENGIN-X will run a mini-inst for the Instron stress rig that needs to use both `ARINST` and `ARACCESS`. These need to be added to `startup.txt` and MySQL needs to be installed and running on the machine. + + + + + + diff --git a/doc/deployment/deploy/Creating-a-new-branch-for-IBEX-configurations.md b/doc/deployment/deploy/Creating-a-new-branch-for-IBEX-configurations.md new file mode 100644 index 000000000..8955cc09b --- /dev/null +++ b/doc/deployment/deploy/Creating-a-new-branch-for-IBEX-configurations.md @@ -0,0 +1,14 @@ +# Creating a new branch for IBEX configurations + +This creates the repository to save all the IBEX configurations for a new instrument. + +1. Navigate to `C:\Instrument\Settings\NDXxxxx` in a git bash terminal on an IBEX developer's machine (i.e. not an instrument) +1. Note the current branch you are on, usually this is will be the name of the machine you are working on, e.g. `NDXxxxx` +1. Review and commit any changes you have on your current branch +1. Run `git checkout master` and `git pull` to get the latest copy of the base configuration +1. Remove any uncommitted changes you have on your branch + * NOTE: This may delete some uncommitted changes that you may wish to keep. Ensure these are all committed before running `git clean -fdx` +1. Create and checkout a new branch for the instrument using `git checkout -b NDXyyyy` where `yyyy` is the name of the instrument in capital letters +1. Run the [config upgrader](#config_upgrade_script) to get this into an up to date state +1. Run `git push` to push your changes to the main git repository +1. `git checkout NDXxxxx` will get you back to your local configuration \ No newline at end of file diff --git a/doc/deployment/deploy/Deploy-script-workflow.md b/doc/deployment/deploy/Deploy-script-workflow.md new file mode 100644 index 000000000..8f9ebfbda --- /dev/null +++ b/doc/deployment/deploy/Deploy-script-workflow.md @@ -0,0 +1,90 @@ +# Deploy script workflow + +Scripts needed are in: `\\isis\shares\ISIS_Experiment_Controls_Public\ibex_utils\installation_and_upgrade` +some instruments it might want instead if not default on cclrc.ac.uk +`\\isis.cclrc.ac.uk\shares\ISIS_Experiment_Controls_Public\ibex_utils\installation_and_upgrade` + +## Pre-script +1. Use RealVNC Viewer to ensure that the instrument is not currently in use. +1. Remote desktop into the instrument +1. Open git-bash + 1. cd into `C:\Instrument\Apps\EPICS` + 2. Run `git status` + 3. Run `git diff` + 4. Ask senior team member if any of the deleted/new/modified files are of note and record the ones that are, to deal with later. +1. Run deploy script in the tools' directory, you can double click on `instrument_deploy.bat` but it is safer to create a command window and then run the command from there, if the process were to crash you see the last message in the window. The install does now write a log file so this may not be necessary, but i usually still do it this way in case. + +## During script + +| Config step (in order of script) | Instruction (y/n) | Comment | +| ----------- | ----------------- | ------- | +| Confirm new version | y if correct | N/A | +| Record LabView VIs | y | Take screenshots of blocks and relevant information such as motors in some cases. (for future reference) | +| Save motor parameters | y | Saves into csv file at `C:\Instrument\var\deployment_pv_backups\motors\` not strictly necessary on a test deploy machine, but useful to do there as well to test the process is still working correctly | +| Save block params | y | Saves into csv file at `C:\Instrument\var\deployment_pv_backups\` | +| Save block server to file | y | It'll now save existing installation | +| Update Git | y | It will need an admin account password for the instrument | +| Update Java | y | This will be manual: go to `\\isis\shares\ISIS_Experiment_Controls_Public\third_party_installers\` and install the latest version of Java. You can find some more info [here](Upgrade-Java) | +| Backup directories | y | This might take a long time. Close any window that have it open. The dir. Find size of epics i apps dir and then find size of backup in data and then old to find out its stage | +| Manually check Python installation | y | `Python` will not exist but `Python3` will, which the script will realise and back up| +| Verify backup | y | N/A | +| Backup database | y | Find password in Keeper | +| Truncate database | y | N/A | +| Start ibex server install | y | N/A | +| Keep old Galil driver | y/n | Should automatically locate the version and proceed but if it requires a decision of which version to use then ask a team member | +| Update ICP | y | N/A | +| Upgrade ICP found in LabView | y | Needs admin password | +| Install genie_python | y | N/A | +| Install MySQL | y | N/A | +| Install ibex client with built-in python | y | N/A | +| Update instrument config | y | N/A | +| Automatic config merge | y | N/A | +| Update calibration repo | y | N/A | +| Apply release notes | y | N/A | +| Update release notes | y | N/A | +| Reapply hotfixes | yes if any to reapply | [See "Reapply Hotfixes" section](#reapply-hotfixes) | +| Update script generator script definitions | y | N/A | +| Remove Instrument Scripts `githooks` | y | N/A | +| Start ibex GUI | y | N/A | +| Restart vis | yes | N/A | +| Client release test | y | N/A | +| Check version | y | Navigate to Help - > About | +| Confirm genie python works | y | In scripting tab run `g.cshow()` correctly and run as well in `C:\Instrument\Apps\Python 3\genie_python.bat` | +| Confirm config is consistent | y | N/A | +| Check web links work | y | N/A | +| Switch instrument to one correct one without NDX prefix | y | N/A | +| Verify the server is up | y | May need to refresh the PVs | +| Client release tests | n | N/A | +| Server release tests | y | N/A | +| Confirm blocks logging as expected | y | Navigate to `C:\Data\[RUN NUMBER]` | +| Confirm correct branch | y | Open git bash and cd to EPICS | +| Check web dashboard | y | N/A | +| Run config checker | y | N/A | +| Save motor params if applicable | y/n| N/A | +| Save block params | y | N/A | +| Set username & password | n | N/A | +| Set autostart script | n | N/A | +| Inform scientists | n | You can if you want but generally done in release messages | + +## Reapply Hotfixes + +1. Open git bash. +2. For each file you noted in the pre-script hotfixes instruction + 1. Run `git diff` between the file in the NEW EPICS dir and the `C:\Data\Old\` dir + 2. Use the outcome and response table to evaluate the file + +| Outcome for each file of note | Response | +| ----------- | ----------------- | +| Returns nothing | Must be part of release so nothing to do | +| Different but you have checked the repo and found a forgotten pull request that matches (you `curl [URL]` the url of the raw file in the PR and run git diff between old and this PR file) | Merge the pull request. Update submodule (pull latest with merge completed), go to EPICS dir and git add .and push this version of EPICS with the updated submodule version | +| Different but no open PR | Ask team member whether okay to leave or if it needs to be copied to new install and a PR (and potentially a ticket) to be made | + + +## Backup and truncation can be done separately from deploy, for example if instrument in cycle: +| Config step | Instruction (y/n) | Comment | +| ----------- | ----------------- | ------- | +| Stop ibex server | y | N/A | +| Run `truncate_database.db` | y | In `\\isis\shares\ISIS_Experiment_Controls_Public\ibex_utils\installation_and_upgrade\` | +| Truncate db | y | The previous step was just the backup | + + diff --git a/doc/deployment/deploy/Deployment-on-an-Instrument-Control-PC.md b/doc/deployment/deploy/Deployment-on-an-Instrument-Control-PC.md new file mode 100644 index 000000000..b99dd1005 --- /dev/null +++ b/doc/deployment/deploy/Deployment-on-an-Instrument-Control-PC.md @@ -0,0 +1,45 @@ +# Deploy to an instrument machine + +This document describes the steps necessary to install/upgrade IBEX on an Instrument control PC. Be sure to login as `spudulike` when deploying on instruments instead of `gamekeeper`. + +## Preparatory Steps for Client and Server + +- Inform the instrument scientist that you are going to upgrade the instrument in 5 minutes so that they are not surprised when you remote desktop to the instrument, include a link to the release notes of the latest release in this email. Wait 5 minutes. +- Visit each instrument on which code will be released and check for changes which have been made which are not summarised on the [IBEX wiki](https://github.com/ISISComputingGroup/IBEX/wiki#tocInstrumentInfo). Do this by running `git status` in the EPICs directory and looking at the diff and comparing with those changes. +- Make sure that the public share has the most recent version of `ibex_utils` from Git (i.e. do git pull). + +## Notes on network share access +The install will need to access both `\ibex_utils` (where the install script is located) and `kits$` (where the ibex release and a genie python instance is kept). As we will be updating python, we cannot use the locally installed python for the deploy. + +On an instrument NDX computer the D: and O: drives will be mapped to the instrument archive and have sufficient access rights, however they can sometimes become disconnected and then things don't work. So before starting look in windows explorer and if D: and/or O: are showing as Disconnected click on them to reconnect. + +If you do not have either a D: or O: network drive, then you will need to specify network credentials to map the drive via the `net use` command. + +## Upgrading IBEX to the latest version +1. If an instrument, check D: network drive status as per above _Notes on network share access_ +1. Ensure the instrument is running and in a setup state +1. Take screenshots of blocks, motors, running VIs, etc. to allow later comparison +1. Ensure all command lines to EPICS or windows accessing the EPICS path are closed (though there is no need to stop the IBEX Server) +1. Run `\ibex_utils\installation_and_upgrade\instrument_deploy.bat` (or for 32-bit releases run with the parameter `x86`) + - you can double click on this, but usually better to drag the path into a new empty cmd window so you don't miss any errors on abort + - It will look for the highest version number in the release folder as a source. + - If you want to install a non-default release you need to set the `SUFFIX` variable in the batch file. For example with `x.y.z` being the current release and `hotfix` being the suffix, it will look for the folder `Releases/x.y.z-hotfix` + - Apart from the below points, just follow instructions + - Be warned the upgrade runs in 3 steps and so will claim to have finished the upgrade 3 times + - Do not remove any SECI icons from the task list if this is not the first time install +1. Compare screenshots taken earlier to current state + +## Install IBEX for the first time +_Note this is unlikely to happen now we've migrated most instruments, and SECI instruments should have a copy of IBEX at the least._ +
+ Click to expand + +- If an instrument, check D: network drive status as per above _Notes on network share access_ +- Run `\ibex_utils\installation_and_upgrade\instrument_install.bat` (if you are on a test machine you may have to enter the full DNS path rather than the shorthand) + - you can double click on this, but usually better to drag the path into a new empty cmd window so you don't miss any errors on abort + - It will look for the highest version number in the release folder as a source. + - If you want to install a non-default release you need to set the `SUFFIX` variable in the batch file. For example with `x.y.z` being the current release and `hotfix` being the suffix, it will look for the folder `Releases/x.y.z-hotfix` + - Follow the instructions on the command line. +- If needed, shutdown IBEX and [upgrade the ISISICP](Upgrade-ISISICP). Remember to run the journal parser installation (Step 8 of upgrading ICP) - this can take some time. +- After the script has successfully finished and the IBEX server has been started, run `instrument_test.bat` in the same folder and follow the instructions. +
diff --git a/doc/deployment/deploy/IBEX-Deployment-on-Training-machine.md b/doc/deployment/deploy/IBEX-Deployment-on-Training-machine.md new file mode 100644 index 000000000..3ff5595de --- /dev/null +++ b/doc/deployment/deploy/IBEX-Deployment-on-Training-machine.md @@ -0,0 +1,15 @@ +# Deploy to a training machine + +Training machines have there own deployment script. It is similar but not the same as the usual instrument upgrade/deployment. It removes all previous traces of IBEX on the system and then installs it as if it was a clean machine. It also installs some training material and a training config. + +Steps: + +1. Make sure that the install script has been pulled it is in `...isis experiment controls public share\ibex_utils` +1. Open a command line inside the `installation_and_upgrade` folder +1. Run `training_update.bat` +1. Follow the steps until the end + +On register step +Copy contents of C:\Instrument\Apps\EPICS\ICP_Binaries\isisdae\x64\visa to C:\Instrument\Apps\EPICS\ICP_Binaries\isisdae\x64\Release + + diff --git a/doc/deployment/deploy/Installing-and-Upgrading-MySQL.md b/doc/deployment/deploy/Installing-and-Upgrading-MySQL.md new file mode 100644 index 000000000..dbf49d0e0 --- /dev/null +++ b/doc/deployment/deploy/Installing-and-Upgrading-MySQL.md @@ -0,0 +1,17 @@ +# MySQL + +## Installation + +There is a script to automatically install MySQL 8. You can find it in `\\isis\shares\ISIS_Experiment_Controls_Public\ibex_utils\installation_and_upgrade\upgrade_mysql.bat` + +## Troubleshooting + +### The installer is complaining that it needs a new version of the .NET framework + +This may just be an issue with the installer rather than with MySQL itself. There is a workaround that lets you avoid having to install a new .NET framework: +1. Use an installer for an older version of MySQL +1. Once complete, run the MySQL Community Installer from the start menu +1. Click on the "Catalog..." button. Follow the prompts to download the latest MySQL product catalog. +1. Once that is done, click on the "Upgrade" button on the MySQL installer home page. Here you can manually select the version you would like to install. + +``` diff --git a/doc/deployment/deploy/Making-an-Instrument-Available-from-the-GUI.md b/doc/deployment/deploy/Making-an-Instrument-Available-from-the-GUI.md new file mode 100644 index 000000000..bed61cde9 --- /dev/null +++ b/doc/deployment/deploy/Making-an-Instrument-Available-from-the-GUI.md @@ -0,0 +1,38 @@ +# Adding an instrument to the instrument list + +The new instrument must be added to the list of available instruments so that users can control it from the Ibex GUI. The list of available instruments is stored in a PV called `CS:INSTLIST`, which runs on the beam status IOC, and is therefore independent of any specific instrument. + +The content of the PV is a string in json format, compressed and hexed, for example: + +``` +[ + {"name": "LARMOR", "hostName": "NDXLARMOR", "pvPrefix": "IN:LARMOR:", "isScheduled": true}, + {"name": "ALF", "hostName": "NDXALF", "pvPrefix": "IN:ALF:", "isScheduled": true}, + {"name": "DEMO", "hostName": "NDXDEMO", "pvPrefix": "IN:DEMO:", "isScheduled": false}, + {"name": "IMAT", "hostName": "NDXIMAT", "pvPrefix": "IN:IMAT:", "isScheduled": true}, + {"name": "MUONFE", "hostName": "NDEMUONFE", "pvPrefix": "IN:MUONFE:", "isScheduled": false}, + {"name": "ZOOM", "hostName": "NDXZOOM", "pvPrefix": "IN:ZOOM:", "isScheduled": true}, + {"name": "IRIS", "hostName": "NDXIRIS", "pvPrefix": "IN:IRIS:", "isScheduled": true}, +] +``` + +To add a new instrument to this list proceed as follows: + +1. Create a new git issue for adding the instrument to the list (for traceability) +1. Check the current PV value from an EPICS terminal: `caget -t -S CS:INSTLIST|uzhex` +1. Edit the instrument list update script to include the new instrument `...EPICS\ISIS\inst_servers\master\scripts\set_instrument_list.py` +1. Create a pull request and get it flash reviewed. +1. Run the script. +1. Verify that all the instruments are picked up by the GUI (e.g. there are no parsing errors): `IBEX -> Switch Instrument` and that they have the correct alarm server and configuration loaded. +1. Verify that the web dashboard for the instruments still works + +## Restricting the GUI to switching to a reduced set of instruments +On the IDaaS machines we need to be able to limit the instruments that a user can switch to in the GUI, to protect the data of other users. This is performed by adding a whitelist to the instrument preferences supplier. This needs to be applied for each VM group that needs to be restricted. + +This shouldn't get overwritten during the RPM deploy as the preferences folder is created by the GUI when it first starts, and isn't in the build. + +1. Locate the preferences supplier in the VM group (`Client_E4\plugins\uk.ac.stfc.isis.ibex.instrument_1.0.0.SNAPSHOT\resources\allowed_groups.conf`). +1. Add the following line to the preferences file, replacing the instrument names for those which the users of this group are allowed to access: +`MUONS,SANS` + - This would allow the GUI to only switch to MUON or SANS instruments from the switch instrument dialogue + - The definitive list of group names is in the instrument list, set by `inst_servers\scripts\set_instrument_list.py` \ No newline at end of file diff --git a/doc/deployment/deploy/NPort-install.md b/doc/deployment/deploy/NPort-install.md new file mode 100644 index 000000000..4b7c9eb28 --- /dev/null +++ b/doc/deployment/deploy/NPort-install.md @@ -0,0 +1,23 @@ +# Moxa NPort + +To install/upgrade NPort driver: + +1. Use remote desktop because you need admin privileges +1. Open NPort admin and note the IP address, Com ports and moxa ports for the current config (NB there maybe be multiple moxas) +1. Uninstall NPort using windows add/remove programs +1. Install the latest NPort software from `...\installs\Installs\Applications\LabVIEW\Other bits\LabVIEW related\MOXA\MOXA NPort` +1. Open NPort Admin +1. Click Add +1. Search for the NPort (click stop when it times out) +1. Clear All, then select the moxa you are using +1. Click OK, do not activate ports +1. Select all ports +1. Click Settings +1. Under basic settings Click "Auto Enumerating COM Number for Selected Ports" +1. Set Com to correct port (probably port 5) +1. Click OK. +1. Click Apply. +1. Ignore the message about setting moxa to real port. +1. Close NPort administrator + +If the upon clicking apply you are told the port is in use, you may need to move to using NPort Driver Manager instead, it can be found here `\\isis\inst$\Kits$\CompGroup\Utilities\MOXA Nport Software` diff --git a/doc/deployment/deploy/Reviewing-Deploy-Tickets.md b/doc/deployment/deploy/Reviewing-Deploy-Tickets.md new file mode 100644 index 000000000..434aae2ab --- /dev/null +++ b/doc/deployment/deploy/Reviewing-Deploy-Tickets.md @@ -0,0 +1,8 @@ +# Reviewing deploy tickets + +Once a deploy has been finished and the ticket requires a review the following tasks should be done to review the ticket: + +- Connect to the machines deployed to and check the current versions of the GUI and server. (In the gui select about, the GUI and Server version numbers should be listed). +- Check that any hotfixes have appropriate tickets. +- Ensure hotfixes are linked at https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information--hotfixes, and versions are linked at http://beamlog.nd.rl.ac.uk/inst_summary.xml. +- Ensure there is a ticket for demonstrating changes between the old version and the newly deployed version. \ No newline at end of file diff --git a/doc/deployment/deploy/Server-Release-Tests.md b/doc/deployment/deploy/Server-Release-Tests.md new file mode 100644 index 000000000..3a2d700a7 --- /dev/null +++ b/doc/deployment/deploy/Server-Release-Tests.md @@ -0,0 +1,20 @@ +# Server Release Tests + +These are tests/checks that should be performed on the IBEX server after a release. They are split into those things that *must* be done and those that should be done (the only reason for not doing these is if the instrument scientists say it is ok not to perform these tests). These are things which are important for the running of the instrument and things which we have missed in previous releases which should not be missed again. + +## Tests that must be done + +### All Instruments + +1. Start an IBEX client. Connect to the instrument. Check Menu -> Help -> About. The Server number should be the version you just released and should match that on the page (https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information) + +1. Ensure that the configuration is being pushed. + 1. go to settings directory in git bash (/c/Instrument/Settings/config/\) + 1. `git fetch` + 1. `git status` + 1. Ensure that the message says `up-to-date with 'origin/'`. +1. Ensure that a block PV is being logged. Do this by right-clicking a PV (one which would be expected to change a little) and select `Display block history -> create a new plot`. There should be data on the plot from after the instrument was upgraded. +1. Confirm that the web dashboard is working for the specific instrument (http://dataweb.isis.rl.ac.uk/) + +### Instruments which are having their first IBEX installation +1. Setup or find a block which is logging itself. Ensure that a log file with this block value in appears in `c:\data\*.log`. If this doesn't happen see [DAE troubleshooting](/specific_iocs/dae/DAE-Trouble-Shooting) diff --git a/doc/deployment/deploy/Updating-Instrument-Machines.md b/doc/deployment/deploy/Updating-Instrument-Machines.md new file mode 100644 index 000000000..02f1ebb29 --- /dev/null +++ b/doc/deployment/deploy/Updating-Instrument-Machines.md @@ -0,0 +1,55 @@ +{#updating_instrument_email_template} +# Updating an instrument machine (email templates) + +It is important to remember when making any changes to instruments (e.g. installing new IBEX versions, cleaning disk space etc) that they are in (potentially) constant use by the instrument scientists and so they must be properly informed. Below is a template of what to send to scientists, this should be sent from the Experiment Controls email account. + +If only a subset of instruments are being modified lookup the instrument scientists for these instruments [here](https://www.isis.stfc.ac.uk/Pages/Instrument-Scientists.aspx) and email only those affected. If most instruments are going to be modified then email the ISIS Instrument Scientists + Others. + +{#updating_instrument_new_release_email_template} +## New Release + +For a new release of IBEX the following should be sent as soon as possible, at least a week (ideally 2 weeks) before the deployment date: + +``` +Dear All, + +We have recently created version **X.x.x** of IBEX and will be looking to install it to the (summer | winter) group, between **date** and **date** which includes the following instruments: + +The list of new features and bug fixes in this release can be found at **link to release notes**. + +To perform the release we will have to stop the instrument from taking data for a couple of hours. If there is a specific day you would like us to perform the release, you would rather not have this release for your instrument or you have any other concerns please get in contact with us ASAP. If we do not hear back from you we will assume that you are happy for us to do this any day in the above range. + +Please note that this is a separate task to any operating system updates that may be going on, we will co-ordinate between those two tasks as appropriate. + +Thanks, +ISIS Experiment Controls +``` + +{#updating_instrument_hotfix_email_template} +## Hot-fix + +In this context a hot fix refers to any modification of an instrument machine that needs to be done at somewhat short notice e.g.: +* Patching a critical bug in some code +* Clearing up disk space + +Hot-fixes may cause disruption to experiments (but less than the issue you are trying to fix) and so to minimise this you should do as much of the following as possible: +* Try to organise for maintenance days +* Give as much notice to scientists as you can +* Minimise the number of instruments to modify + +The following is a template and will require some modification based on what the issue is you are trying to resolve: + +``` +Dear All, + +Unfortunately we have recently seen an issue on your instruments where **describe issue here**. + +To resolve this issue we will need to **detail fix here**. To do this we will require **stopping taking data? use of remote desktop?** for **time it will take to fix**. + +If we do not do this then **how will it affect the user's experiments**. + +Ideally we would like to do this **on maintenance day? ASAP**. Please let us know when would be a suitable time to make this change. + +Apologies for any inconvenience, +ISIS Experiment Controls +``` \ No newline at end of file diff --git a/doc/deployment/deploy/Upgrade-ISISICP.md b/doc/deployment/deploy/Upgrade-ISISICP.md new file mode 100644 index 000000000..e75bf0dc7 --- /dev/null +++ b/doc/deployment/deploy/Upgrade-ISISICP.md @@ -0,0 +1,86 @@ +# ISISICP + +## DAE type remaining the same + +1. Check `isisicp` and `isisdatasvr` processes are not running. You cannot kill them if ISISDAE-IOC-01 is running so you need to stop this separately (and stop procserv restarting them) or run stop_ibex_server +1. Backup existing installation: copy the following to `c:\data\old\isisdae_backup_YYY_MM_DD`: + - `c:\LabVIEW Modules\dae` + - `c:\data\recovery.run` + - `c:\data\selog.*` (`.sq3` `.sq3-shm` and `.sh3-wal` files) + +1. Confirm the type of DAE hardware you have - look in `c:\labview modules\dae\icp_config.xml`, if DAEType in this file is 1 or 2 you have DAE2 hardware (1 means it runs in neutron mode, 2 in muon mode), if it is 3 you have a DAE3 hardware. So if DAEType is 1 or 2 use the **DAE2** folder on the share below to install from, if DAEType is 3 use the **DAE3** folder to install from. If you install the wrong type, it will just fail to start and complain in the ICP log file about wrong dae type; you would just need to reinstall the right one. +1. Go to `\\isis\inst$\Kits$\CompGroup\ICP\ISISICP` and into the directory for the DAE hardware you determined was installed (DAE2 or DAE3) +1. Go into the latest build number directory (also in `LATEST_BUILD.txt`) and double click on `update_inst.cmd` + This can take a while (several minutes). You will get various messages. The following are OK: + * If it can't replace the file `isisicp_extMC.dll` as it is open +1. As printed to screen at the end of script, open a command terminal as administrator (gamekeeper) and run + ``` + cd "c:\labview modules\dae" + register_programs.cmd + ``` +1. Now delete `c:\data\selog.*` (`.sq3` `.sq3-shm` and `.sh3-wal` files), `c:\data\current.run*` and `c:\data\data.run*` +``` +del /q c:\data\selog.* +del /q c:\data\current.run* +del /q c:\data\data.run* +``` + +This completes the ISISICP installation. + +*** + +At this point is it useful to install/update the JournalParser. This utility is not part of the ISISICP +but parses the XML journal files generated by the ISISICP in `c:\data` at run end and updates entries in the MqSQL database + used by the IBEX journal viewer. The journal parser executable program is ran by the isisicp at each run end +via end_of_run.bat to add a new entry + +1. To install/update journal parser, navigate to `\journalparser_static_new` + - *Either* run `install.bat` to install it an migrate all old journal files (new ibex instrument) + - *or* run `install_no_migrate.bat` to install it without copying old journal files (existing ibex instrument) + +If this install fails for some reason, it will not affect an instrument operating - it will just mean that new run entries will not +be visible in the ibex journal viewer (which looks at MySQL). It will not affect the actual creation of the XML reference files by the ISISICP, +so a full re=import can be done at a later date when any issues are resolved. + + +### A log from register_programs.cmd +``` + +C:\Windows\system32>cd "c:\labview modules\dae" +c:\LabVIEW Modules\dae>register_programs.cmd +ERROR: The process "LabVIEW.exe" not found. +ERROR: The process "SeciUserInterface.exe" not found. +ERROR: The process "SECIStartup.exe" not found. +ERROR: The process "mkscript3.exe" not found. +ERROR: The process "muonscript.exe" not found. +ERROR: The process "makescript.exe" not found. +ERROR: The process "PlotScan.exe" not found. +ERROR: The process "vs7jit.exe" not found. +ERROR: The process "mari_script.exe" not found. +ERROR: The process "tkgenie32.exe" not found. + +INFO: No tasks running with the specified criteria. +ERROR: The process "isisdatasvr.exe" not found. +ERROR: The process "isisicp.exe" not found. +ERROR: The process "cwdss.exe" not found. +ERROR: The process "runapp.exe" not found. +The process cannot access the file because it is being used by another process. +Registering Release images +Registering in x64\Release +Press any key to continue . . . + +c:\LabVIEW Modules\dae> +``` + +## Changing DAE type (DAE2 -> DAE3) + +1. If DAE3 is new to this computer, it may not have the "Microsoft Visual C++ 2015 redistributable" installed. Check in the installed program list, if it is missing run `vc_redist.x64.exe` in `\\isis\inst$\Kits$\CompGroup\ICP\ISISICP\VS2015` +1. install the new DAE software as above +1. edit icp_config.xml and change DAEType from 1 to 3 +1. Before you run DAE3 for first time you will need to run `set_dae3_arp.bat` in `labview modules\dae` as an administrator +1. You will also need to add `set_dae3_arp.bat` to be run as administrator at system startup. To do this run the task scheduler as administrator and then import the `DAE3_arp_boot_task.xml` boot task file. +1. Also check firewall settings + +## Developer installation + +The ISISICP on a developer machine is ran from the `ICP_Binaries` directory, this will get automatically updated when `create_icp_binaries.bat` is run. If you get simulated DAE issues, you may need to re-register the ISISICP by running `EPICS\ICP_Binaries\isisdae\register_programs.cmd` as an admin account diff --git a/doc/deployment/deploy/Upgrade-Java.md b/doc/deployment/deploy/Upgrade-Java.md new file mode 100644 index 000000000..76d488f1c --- /dev/null +++ b/doc/deployment/deploy/Upgrade-Java.md @@ -0,0 +1,28 @@ +# Java + +## Instrument + +```{note} +If you are following this link from the first-time installation guide, then you do not need to worry about this section. +``` + +1. Before you start, ensure you understand [Java's licensing](/processes/dev_processes/Understanding-Java-Licensing). +1. If any previous java version is installed on the system, uninstall it. Note that previous java versions may be under "Oracle", "AdoptOpenJDK", or "Eclipse Temurin JDK". +1. Install an appropriate version of java: + - For developer installs, use the latest java in `...\ISIS_Experiment_Controls_Public\third_party_installers\latest_versions\OpenJDK...` + - For IBEX version v11 and before, use JDK 11 in `...\ISIS_Experiment_Controls_Public\third_party_installers\old_versions` + - For IBEX version v12 onwards, use JDK 17 in `...\ISIS_Experiment_Controls_Public\third_party_installers\latest_versions\OpenJDK...` + - Do not use a copy downloaded from the internet as it may not have been tested with IBEX +1. Run the installer and install all components. The install menu should look like this: + +![open_jdk_install_prompt](open_jdk_install_prompt.PNG) +1. Make sure you run the instrument, either now or before you finish + - Answer any firewall questions "yes, allow through the firewall". + +{#developer_upgrade_java} +## Developer + +1. To build CS-Studio, Phoebus or versions of the IBEX GUI which include Phoebus components from source, you will need JavaFX binaries. These can be patched onto the AdoptOpenJDK/Eclipse Temurin installation. Download the Windows SDK from `\\isis\inst$\Kits$\CompGroup\ICP\Java_utils\openjfx-19_windows-x64_bin-sdk\javafx-sdk-19` (originally from [gluon](https://gluonhq.com/products/javafx/)) and copy the `bin`, `lib`, and `legal` directories over the corresponding directories in your java installation. Note that the JavaFX version does not necessarily need to match your java installation, as long as the versions are compatible. For example we can use JavaFX 19 on a Java 11 installation. Please check that the license is still appropriate before you install. +1. If you will be debugging lots of java applications you may wish to install the java Visual VM. This used to be bundled with oracle java but is no longer present in OpenJDK. You can download a GPL-licensed version of the visual VM from https://visualvm.github.io/ + +The binaries listed above are also copied in `\\isis\inst$\Kits$\CompGroup\ICP\Java_utils`. diff --git a/doc/deployment/deploy/open_jdk_install_prompt.PNG b/doc/deployment/deploy/open_jdk_install_prompt.PNG new file mode 100644 index 000000000..03a018473 Binary files /dev/null and b/doc/deployment/deploy/open_jdk_install_prompt.PNG differ diff --git a/doc/deployment/future_workflows/Containerising-an-IOC.md b/doc/deployment/future_workflows/Containerising-an-IOC.md new file mode 100644 index 000000000..c42343c39 --- /dev/null +++ b/doc/deployment/future_workflows/Containerising-an-IOC.md @@ -0,0 +1,149 @@ +# Containerising an IOC + +## Setting Up a Development Environment on Windows + +WSL, or Windows Subsystem for Linux, allows the use of a Linux/GNU environment from a Windows machine. + +### Check Your Windows Version + +WLS2 is only available in Windows 11 or Windows 10, Version 1903, Build 18362 or later. + +The Windows version of your machine can be checked by pressing `Win + R`, typing `winver` and pressing Enter. + +### Installing WSL + +To install WSL, open a PowerShell or CMD window in Administrator mode, and enter the following: + +`wsl --install` + +Note: This method only works if WSL is not installed. If you run `wsl --install` and see the WSL help text, please try running `wsl --list --online` to see a list of available distribution and run `wsl --install -d ` to install a distribution. + +### Uninstalling Legacy WSL Versions + +If you wish to uninstall the legacy version of WSL, instructions of how can be found [here](https://learn.microsoft.com/en-us/windows/wsl/troubleshooting#uninstall-legacy-version-of-wsl) + +### Additional Resource + +Further information on the WSL installation process can be found [here](https://learn.microsoft.com/en-us/windows/wsl/install#install-wsl-command) + +## Rancher Desktop Installation + +Rancher Desktop is an application provides container management and `Kubernetes` on the desktop. Rancher Desktop provides the ability to build, push, and pull container images along with the ability to run containers. + +Rancher Desktop can be installed [here](https://docs.rancherdesktop.io/getting-started/installation/). + +Once the .exe file has been downloaded, follow the steps to complete the installation process. + +Ensure the `dockerd` is selected as the runtime during the installation or within the Rancher Desktop settings. + +## Creating a Container + +1. Create a folder in which all container file will be stored i.e. `C:\Instrument\App\Epics\Container`. + +2. Within the folder, create a `test.db` file: + + `record(ao, "TESTTEST:FOO") {}` + +3. A `Dockerfile` should be added as this is the set of 'instructions' from which Docker builds images. Below is an example which creates a container with a simple IOC: +``` + FROM python:3.11-slim + RUN apt-get update && \ + apt-get install -y \ + build-essential \ + wget \ + git \ + && rm -rf /var/lib/apt/list/* + + RUN pip install --upgrade pip + RUN pip install epicscorelibs + + ENV EPICS_BASE=/epics + ENV EPICS_HOST_ARCH=linux-x86_64 + ENV EPICS_CA_SERVER_PORT 5066 + + WORKDIR /app + COPY . /app + + ENTRYPOINT ["python", "-m", "epicscorelibs.ioc", "-d", "test.db"] + +``` + +4. To build the container from the `Dockerfile`, cd into the folder within a CMD window with admin privilege, ensure that Rancher Desktop is running and enter: + + `docker build -t .` + + docker build: Tells Docker to create a new image. + + -t: Specifies a name for the image you're building. + + : The name you're giving to the new image. + + .: Tells Docker to use the current folder for the build. + +5. To run the container, enter the following (replace the path with your own folder location): + + `docker run --rm -it --net=host -v :/usr/src/app ` + + `EXAMPLE: docker run --rm -it --net=host -v /c/Instrument/Apps/EPICS/isis-ioc-container:/usr/src/app ioc_container` + + docker run: Starts a container. + + --rm: Deletes the container when done. + + -it: Lets you interact with the container. + + --net=host: Uses your computer’s network. + + -v :/usr/src/app: Shares a folder from your computer with the container. + + : The image to use for the container. + + After this, the container should be running within the Python Environment within the CMD window. + +6. To test if the PV is readable within the container, a EPICS terminal window is needed. For this, open another CMD window and enter the following: + + `C:\Instrument\Apps\Epics\config_env.bat` + + `set EPICS_CA_ADDR_LIST=127.0.0.1:5066` + + +7. Now that both terminals are running, the value from the PV can be retrieve by entering the following command into the EPICS terminal: + + `caget TESTTEST:FOO` + + To which the response should be `TESTTEST:FOO 0` + + `caput ` can also be used to set the value of the PV. + + +### Network Connections + +The following command entered within the EPICS terminal allows users to check for network connections that are being used by a specified port, in this example the port 5066 is being checked: + +`netstat -a -n -o |findstr 5066` + + +## Observations and Present Limitations + +It would appear that both IBEX and the network=host wish to use port 5064, and the container is being blocked whilst the IBEX server is running. + +A solution as been found, which is already included within the `Dockerfile`, where another port has been bound which enables the container's IOC to be communicated with whilst the IBEX server is running. However, it has been found that each IOC will need it's own port, which can lead to issues as the number IOC increases. + +## Further Exploration + +In the above example, an approach similar to [epics-containers](https://epics-containers.github.io/main/index.html) was attempted, however there are a couple of alternative methods that could be explored. + +[SnowSignal](https://github.com/ISISNeutronMuon/SnowSignal/tree/main) is a possible alternative to UDP communication between localhost and the container. SnowSignal is designed to create a mesh network between instances of the program that will listen for UDP broadcasts received on one node of the network and rebroadcast on all other nodes. + +WLS on Windows 11 22H2 and higher introduced ['Mirror mode networking'](https://learn.microsoft.com/en-us/windows/wsl/networking#mirrored-mode-networking). This changes WSL to an entirely new networking architecture which has the goal of 'mirroring' the network interfaces that you have on Windows into Linux, to add new networking features and improve compatibility. + +## WSL2 on a Windows 11 Virtual Machine + +There can be an issue with running a VM on a Windows 11 VM, which may result in the following error: + +["Error: 0x80370102 The virtual machine could not be started because a required feature is not installed."](https://learn.microsoft.com/en-us/windows/wsl/troubleshooting#error-0x80370102-the-virtual-machine-could-not-be-started-because-a-required-feature-is-not-installed) + + +The command is as below, if you run it directly on the right server you can skip the `-Computername HOSTSERVERNAME` bit. + +`PS C:\> Set-VMProcessor -ComputerName HOSTSERVERNAME -VMName GUESTMACHINENAME -ExposeVirtualizationExtensions $true` \ No newline at end of file diff --git a/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance-windows-options.png b/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance-windows-options.png new file mode 100644 index 000000000..d243d4369 Binary files /dev/null and b/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance-windows-options.png differ diff --git a/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance.md b/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance.md new file mode 100644 index 000000000..a4621c2ad --- /dev/null +++ b/doc/deployment/future_workflows/Containerising-the-Archiver-Appliance.md @@ -0,0 +1,116 @@ +# Containerising the Archive Appliance + +The EPICS Archiver Appliance is principally targeted to run on a Linux host and there are some issues when attempting to run it under Windows. +[Ticket #8438](https://github.com/ISISComputingGroup/IBEX/issues/8438) was to investigate and document whether the Archiver Appliance can be run in a container on a Windows host. Particular interest is its ability to run on a NDX Windows virtual machine. + +A new EPICS gateway has been configured to make localhost PVs available to the container - In C:\Instrument\Apps\EPICS\gateway (https://github.com/ISISComputingGroup/EPICS.git), new files have been added: + +| File | Description | +| ---- | ----------- | +| `gwcontainer.acf` | Container gateway security | +| `gwcontainer.pvlist` | Container gateway PV list | +| `start_gwcontainer.bat` | Container gateway start script, called from start_gateways.bat | +| `stop_gwcontainer.bat` | Container gateway stop script, called from stop_gateways.bat | + +A new git repository has been created (`https://github.com/ISISComputingGroup/isis-aa-config.git`). This contains all the code required to build a Archiver Appliance image and run a container instance. + +| File | Description | +| ---- | ----------- | +| `Containerfile` | Defines the content to build into the image | +| `aa-compose.yaml` | Used by `nerdctl compose` to control building the image and running the container | +| `docker-compose.yaml` | This was used as an experimental compose file to test various networking options to try to circumvent Windows lack of 'host' container networking. Retained as there is some useful info and techniques | +| `containerdata` | This directory is mounted by the container and facilitates data persistence | + +Due to complications and uncertainties of licencing of Docker Desktop, it has been decided to adopt an open source alternative, a number of which are freely available, such as Rancher Desktop, `Podman`, etc. For the purpose of this exercise, [Rancher](https://rancherdesktop.io/) was chosen. + +## Rancher Desktop Installation +Non-windows Containers on Windows hosts need Windows System for Linux (WSL2) to be installed on the host machine. If not already present, this is installed as part of the Rancher Desktop installation process. Rancher Desktop will create its own necessary distributions (rancher-desktop-data and rancher-desktop) on WSL and it there is no need to manually install anything else. + +**Note:** Rancher (and other container managers) have a minimum Windows version that it will run on, which is 10-1909 or newer. + +Download the Installation MSI file for Windows (x64). At the time of writing, it is: `Rancher.Desktop.Setup.1.15.1.msi` + +Run the msi file, you will need admin access at some point. Under testing, an error message was presented: "Rancher Desktop Setup Wizard ended prematurely. Your system has not been modified...", but on the second attempt, the installation completed successfully. + +## Creating the Archiver Appliance image +The image is composed of the EPICS Archiver Appliance, hosted on a minimal Linux platform. The definition for this is specified in the `Containerfile` file. +The repository for the containerised Archiver Appliance development is: [isis-aa-config.git](https://github.com/ISISComputingGroup/isis-aa-config.git) + +The image can be built either using the `Containerfile` directly: + +`nerdctl build . --tag isis-aa` + +Or it can be built and run using `compose`: + +`nerdctl compose -f aa-compose.yaml up` + +The `compose` route is preferred, as it specifies all the port mappings and host mount points within the YAML file. + +if `nerdctl build` is used, then the container will need to be spun up via the following: +`nerdctl run -it --rm -v "containerdata:/storage" -p 17665:17665,5064:5064,5065:5065 isis-aa /bin/bash` +Where: +* 17665 is the Archiver Appliance web interface port. +* 5065,5064 are the standard EPICS channel access ports (both UDP and TCP). + +**Note** that the local `containerdata` directory is specified as a relative path to the current working directory. It is possible to define an absolute path, but this needs further reading and testing to discover how it works. + + + +## Observations and present limitations +### Minimum Windows version +Rancher Desktop (and likely any other container management platform under Windows) requires WSL2. The minimum Windows version that is supported is 10-1909. Unfortunately, the NDX VMs are older, probably running 10-1809, so it will not be possible to run any form of containerisation on those hosts until they are upgraded. Upgrading the NDX hosts requires a carefully designed test plan to ensure minimal risk to loss of beam time. + +### Rancher WSL vs Windows WSL +Although Rancher does install a version of WSL automatically, I ran into an issue which prompted me to update it using the `cmd` command: `wsl --update` +after which there were no further issues. + +### Virtualisation support on the NDX and NDH hosts +On the container host system (NDX) it is necessary to switch on WSL (Windows Subsystem for Linux): +![image](Containerising-the-Archiver-Appliance-windows-options.png) + +On the main VM host machine (NDH) it is necessary to switch on 'Nested Virtualisation' to allow the NDX VM to run its own VMs. If this is not done, then errors will be presented when trying to run Rancher Desktop: 'Requires WSL with kernel 5.15 or newer (have 0.0.0.0) + +## EPICS Container Gateway +### The problem +The Archiver Appliance running inside a container requires visibility of Channel Access broadcasts to enable it to see PVs. By default a container is isolated from the host network, unless networking is specified or specific ports mapped at runtime to the host network. A Linux host, with a standard network stack can permit a container to full network access by specifying `--network 'host'` parameter. Unfortunately, Windows does not implement a standard stack and has some weird network interface construction that prevents the 'host' option being used. +Instead a gateway has been configured which the container 'sees' via port mapping 5064 and 5065. +The new container targeted gateway is started with the `start_gwcontainer.bat` script, called from within `start_gateways.bat`. + +### Access security +A new ACF file (`gwcontainer.acf`) has been created specifically for the container gateway running on the localhost (NDX). Only localhost is in the machine list (`HAG`) and PVs are made read/write by anyone. + +## Performance +Reduced performance is to be expected when running containers on a virtual machine. Some metrics have been obtained by running simple containerised stress tests and comparing with those observed on a development machine. +A simple python script to find 1000 prime numbers: +| Host type | Elapsed time | +| --------- | ------------ | +| NDX native | 0.004 - 0.010 s | +| NDX container | 0.006 - 0.009 s | + +At first sight, this seems to indicate that there should be very little concern regarding container performance on NDX machines compared with native execution. + +### Storing images in a different location +On installing Rancher Desktop, two WSL distributions are created: `rancher-desktop` and `rancher-desktop-data`. The latter is where images are stored and it is feasible to modify the physical storage volume via the following procedure: +``` +wsl --shutdown +wsl --export rancher-desktop-data rancher-desktop-data.tar +wsl --unregister rancher-desktop-data +wsl --import rancher-desktop-data c:\instrument\var\containers rancher-desktop-data.tar --version 2 +``` +Once verified that it has worked, ```rancher-desktop-data.tar``` can be deleted. + +## Testing +In order to verify correct operation of the Archiver Appliance container, follow the procedures from the sections above: +1. Rancher Desktop Installation (above) +2. It is important to ensure that Rancher Desktop is running. +3. Start a local EPICS gateway, using Instrument/Apps/EPICS/gateway/start_gateways.bat +4. Build and run the container by following: Creating the Archiver Appliance image (above) +5. Wait for Tomcat to complete initialisation in the container. +6. Use a browser to view: http://localhost:17665/mgmt/ui/index.html +7. Select a small number of PVs and type them into the web page (Home page) where it says: "please type in some PV names here" +8. Click the 'Archive' button. +9. After a delay of around a minute or longer, the PV status should change from 'Initial sampling' to 'Being archived'. +10. Check that archive files are being created in `./Containerdata/{sts,mts,lts}` directory trees. + + + diff --git a/doc/deployment/future_workflows/Mount-or-create-IBEX-VHDs.md b/doc/deployment/future_workflows/Mount-or-create-IBEX-VHDs.md new file mode 100644 index 000000000..48444c0af --- /dev/null +++ b/doc/deployment/future_workflows/Mount-or-create-IBEX-VHDs.md @@ -0,0 +1,100 @@ +# Mount or create IBEX VHDs + +The new architecture for deployment is to deploy onto VHDs which will be copied to the desired places and linked in. + +## Parts of the system + +### System VHD + +There needs to be a system VHD, this has been created by Chris and will be detailed *CHRIS TODO*. This contains windows and user level programs (probably including LabVIEW). + +### App VHD + +IBEX applications need to be placed on a disk and these should be generic for all instruments. In practice hot fixes will make them different, we record hotfixes and these are reapplied on upgrade of needed. + +### Configuration Settings VHD + +IBEX settings are placed on a setting VHD, these will be created once when the instrument is converted to use VHDs and after that this will be persisted through IBEX and Windows upgrades. On Upgrade, the configuration will need to be upgraded and the common calibration files need to be updated. + +### Var VHD + +These contain part of the system which change often. We are undecided what to do on upgrade: + +- *Either:* Create the VHD when migrated and on upgrade truncate the database, move old log files and old autosave files. Then upgrade the database. +- *Or:* Every upgrade use a new common VHD copy across autosave files (maybe these should live in settings), dump the data schema for interesting tables and reimport it. + +For the test system I have gone with the first approach because it was easy. I like the second approach better. + +## VHD Creation + +### Create empty VHDs + +1. Copy the three empty vhdx files (`empty_apps.vhdx`, `empty_var.vhdx`,`empty_settings.vhdx`) to a local disk (from CompGroup\chris) and rename to +`apps.vhdx`, `var.vhdx` and `settings.vhdx`. +2. These are pre-configured to be 30GB each. + +### Mount a VHD + +1. Open a command prompt as administrator +1. `diskmgmt.msc` +1. Click Action -> Attach VHD + - I had to click help first +1. Select location of VHD + 1. Mount `Apps.vhdx` to `C:\Instrument\Apps` by creating a directory junction from `c:\instrument\apps` to the drive letter of the VHD you mounted + 1. Mount `Settings.vhdx` to `C:\Instrument\Settings\config` as above + 1. Mount `Var.vhdx` to `C:\Instrument\var` as above + +### Dismounting VHD + +1. Close the VHD and copy back + 1. Right click on drive on left at the bottom + 1. Detach VHD + 1. Copy file back to the source + +## VHD Creation Jenkins Job + +There is an [automated job in jenkins](http://epics-jenkins.isis.rl.ac.uk/job/Create_VHD/) which builds VHDs from the latest IBEX server/client/python versions. + +### Build server setup + +To set up a computer to be able to run the [automated VHD creation script](https://github.com/ISISComputingGroup/ibex_utils/tree/master/installation_and_upgrade): +- Hyper-V must be enabled on the computer which will be running the script. It can be turned on by searching for "turn windows features on or off" from the start menu and then selecting the entire Hyper-V tree. If hyper-v wasn't already turned on this will require a restart. If you are on Windows 2012, enable it from the Server Manager (by the start button on the desktop) then select Manage -> Add Roles and Features and check the entire Hyper-V tree. +- Powershell must be upgraded to at least version 5 to support the commands we are using. +- Set up an environment variable called `MYSQL_PASSWORD` containing the MySQL root user password. +- In the admin documents area, create an (empty) folder at `C:\Users\Administrator\Documents\fake_release_dir\1.0.0` +- A local python must be installed in `C:\Users\Administrator\Documents`. Note: we do not use the python from the network share here as we are running as Admin. +- The machine you're installing this on must be listed in genie_logger.py in the vhd_builder list to avoid writing logs to `C:\Instrument\Var\logs` before Var is mounted + +These items cannot be automated by jenkins as they need to be the admin user. + +### Mounting and dismounting VHDs automatically + +Because VHD mounting and dismounting requires admin rights, this is done by a scheduled task running as the admin user. The code run by these scheduled tasks is checked out to `C:\Users\Administrator\Documents\ibex_utils\installation_and_upgrade`, and the bat file which is run is `vhd_scheduled_task.bat`. + +The scheduled tasks run every minute and look for a file which is created by the install script. If this file exists, the tasks will mount/dismount the vhds and then delete the file. Otherwise the tasks do nothing. + +### Build artefacts + +Empty VHDs are currently taken from: +``` +\\isis\inst$\Kits$\CompGroup\Chris +``` + +Once filled with IBEX files, the VHDs are copied to: +``` +\\isis\inst$\Kits$\CompGroup\ICP\VHDS +``` + +## Upgrade IBEX Version on a VM By Copying VHD + + - The VM set up, running an older version of IBEX, has been located within Hyper-V manager. + - If you wish, a 'Checkpoint' (Right Click -> Checkpoint) can be capture of the current system to allow a way to restore the VM to a functioning state, if required. + - Ensure that the VM is switched OFF (Right Click -> Turn Off...). + - Once the State has changed to 'Off', Right Click the VM and select 'Settings...'. + - Under Hardware -> SCSI Controller, Select the Hard Drive for '/apps'. + - Select 'Browse...' and locate the '/apps' drive containing the latest IBEX build. + - Select 'Open', and double check this is the correct drive. + - Select 'Apply' and then 'OK'. + - To test that the IBEX has been updated, double click the VM or right click and press 'Start'. + - Once the IBEX server has been start, you can check the version within 'Help' -> 'About' on the toolbar within the IBEX GUI. + diff --git a/doc/deployment/patch/Creating-a-patch-hotfix-branch.md b/doc/deployment/patch/Creating-a-patch-hotfix-branch.md new file mode 100644 index 000000000..9b222eacc --- /dev/null +++ b/doc/deployment/patch/Creating-a-patch-hotfix-branch.md @@ -0,0 +1,18 @@ +# Creating a patch/hotfix branch + +If there has been an EPICS dependency update, then taking DLLs/EXE from the latest build server or your desktop for an instrument will not work as there will be library mismatches. If you need to build a new exe/dll for an existing release, this is the procedure to setup a jenkins job for this + +If EPICS has a git tag v14.0.0 then + +if first time you need to create branch +``` +git checkout -b Release_14.0.0_hotfix v14.0.0 +git push -u origin Release_14.0.0_hotfix +git submodule update --recursive +``` +subsequently just change to it - either locally or if somebody else has already created it and pushed it +``` +git checkout Release_14.0.0_hotfix +git submodule update --recursive +``` +If you just want to use master of a new module, you can just `git add` that. In more complicated setups you may need to create a `Release_14.0.0_hotfix` branch in the submodule based on its original 14.0.0, cherry pick across relevant changes, and then `git add` the submodule at the top diff --git a/doc/deployment/patch/Deploying-a-DEBUG-build-IOC.md b/doc/deployment/patch/Deploying-a-DEBUG-build-IOC.md new file mode 100644 index 000000000..1e493de10 --- /dev/null +++ b/doc/deployment/patch/Deploying-a-DEBUG-build-IOC.md @@ -0,0 +1,14 @@ +# Deploying a Debug IOC + +**NOTE**: if you are a developer wishing to develop on your desktop with a debug build, the debug version of the [Developer server build](/iocs/compiling/Developer-Server-Build) may be easier to work with. The builds below do not ship `.lib` or intermediate `O.*` object directories so if you wish to rebuild with changes after having run the debugger it is a little more time consuming. + +DEBUG DLL builds are kept in `EPICS_DEBUG_CLEAN_win7_x64` in the usual kits deployment area. If you wish to deploy only a single IOC in debug mode rather than replace the whole installation with the above debug build then: +1. You need to build the `dllCopy.bat` by running ```make dllCopy.bat``` in the ioc boot area. +1. Double-click on the **`dllCopy.bat`** file in the `iocBoot` area of the IOC you want to deploy on kits, this will copy the dependent DLL and `.pdb` files to the {ioc}/bin/windows-x64-debug directory +1. Backup the `{ioc}/bin/windows-x64` directory on the target computer to be tested. +1. Copy the `.EXE` and all `.DLL`/`.pdb` files from the `kits/{ioc}/bin/windows-x64-debug` directory to the `bin/windows-x64` of the target IOC. +1. Also copy the files in `kits/EPICS/crtl/windows-x64-debug` to the target `{ioc}/bin/windows-x64` directory. + +When you start the IOC it should now start the debug build you have copied over. + +Note: there is also a EPICS_STATIC_DEBUG_CLEAN_win7_x64 build - this would only require copying the single `.EXE` and `.pdb` file as all DLLs are bundled into the `.EXE` in a static build; however static builds have sometimes behaved differently to DLL builds in the past and so the above DLL option is a better test of the eventual mechanism, though trying a static EXE may be enough for diagnostics. The files here will be in a `windows-x64-static-debug` directory. diff --git a/doc/deployment/patch/Modifying-Code-on-an-instrument.md b/doc/deployment/patch/Modifying-Code-on-an-instrument.md new file mode 100644 index 000000000..ee5d5d923 --- /dev/null +++ b/doc/deployment/patch/Modifying-Code-on-an-instrument.md @@ -0,0 +1,14 @@ +# Modifying code on an instrument + +What to do when you need to fix something on an Instrument or modify code on the instrument. + +1. Fix the bug +1. Record the bug as a ticket; with solution done +1. Note the bug next to the instrument on the main IBEX page +1. If the bug is likely to effect other instruments inform the team using the group email + 1. If it will seriously effect instruments an email can be sent round offering/organising time to fix it to all scientists + 1. If it is minor it can be sent to those who are interested + 1. If it is unlikely to effect anyone then it can be noted in known bugs section of the release notes +1. If the bug will seriously effect instrument scientists consult the PM + +The reasoning behind this procedure is so we always know what is on any instrument but don't have to create a new release to fix something. Part of the release process is to delete any changes to an instrument and then put back any noted changed. So if the changes aren't recorded then they will be lost. diff --git a/doc/deployment/patch/Modifying-Device-on-an-Instrument.md b/doc/deployment/patch/Modifying-Device-on-an-Instrument.md new file mode 100644 index 000000000..b01145961 --- /dev/null +++ b/doc/deployment/patch/Modifying-Device-on-an-Instrument.md @@ -0,0 +1,6 @@ +# Modifying devices on an instrument + +When visiting an instrument to modify or setup a device we should: + +1. Consult the [specific IOC details page](/Specific-IOCs) to see what connection setting are likely to be needed. +1. Update the [specific IOC details page](/Specific-IOCs) diff --git a/doc/deployment/patch/Modifying-Plugins-on-a-Deployed-Client.md b/doc/deployment/patch/Modifying-Plugins-on-a-Deployed-Client.md new file mode 100644 index 000000000..94d29c493 --- /dev/null +++ b/doc/deployment/patch/Modifying-Plugins-on-a-Deployed-Client.md @@ -0,0 +1,18 @@ +# Patching a deployed client + +If you need to modify a plugin on an IBEX client running on an instrument, such as OPIs, you need to go to + +```/plugins/```. + +## Live editing opis / non-compiled code + +These can be edited on the fly. If you edit a `.opi` file, the change will be seen the next time the opi is opened in IBEX. However, if you edit `opi_info.xml` (for example, to add an entirely new opi) you will need to relaunch the client as this file is only read once at startup. + +## Live editing "compiled" code + +Plugins can be built as either folders containing `.class` files, or `.jar` files. If you need to edit the contents of a `.jar`: +- Rename the plugin from `_.jar` to `_.zip` +- You can now extract the files and make any necessary changes +- When done, put it all back into a `.zip` archive, ensuring that the directory structure is the same as before +- Rename the `.zip` back to a `.jar` +- Relaunch the client and test your changes \ No newline at end of file diff --git a/doc/deployment/patch/Patch-or-hotfix-an-IOC.md b/doc/deployment/patch/Patch-or-hotfix-an-IOC.md new file mode 100644 index 000000000..33eb27c13 --- /dev/null +++ b/doc/deployment/patch/Patch-or-hotfix-an-IOC.md @@ -0,0 +1,31 @@ +# Patch or hotfix an IOC + +To deploy a new IOC to an instrument, or patch an existing one, different levels of care need to be taken depending on what has changed + +## OPI for new IOC, or changes to existing OPI + +You do not usually need to deploy a new GUI, in fact it is best to avoid this as you may update other OPIs unintentionally, introducing changes that need updates to IOC that you are not deploying. So go for the minimal change approach if possible: +- keep existing GUI build on instrument +- the directory you will be updating on the instrument will be of the form `C:\Instrument\Apps\Client_E4\plugins\uk.ac.stfc.isis.ibex.opis_1.0.0.4670\resources` and if from your local machine then from somewhere like `C:\Instrument\Dev\ibex_gui\base\uk.ac.stfc.isis.ibex.opis\resources` +- copy across just the new/changed OPIs and leave everything else as is +- If adding a new IOC, then you may need to manually edit `opi_info.xml` to add new OPI to ibex list. Do not copy across a new `opi_info.xml` as it could refer to things that are not present. + +## Updating an existing IOC (no EXE changed) + +again, change the minimum possible, don't copy a whole tree from the jenkins build server unless you need to. It is usually safer to use a clean jenkins build from `kits$` as a basis unless you are really sure your local build is consistent. If you do not need to update the IOC `EXE` file in `ioc/master` you can probably just copy across the new db files or st.cmd changes etc. Note that we often edit files in e.g. a `YYYApp\Db` and `YYYApp\protocol` and they get deployed to a top level `db` and `data` when make is run, IOC normally load these top level ones but copying both sets is safe just in case. + +## Deploying new IOC (or updating an existing ioc when EXE has changed) + +An IOC EXE from `ioc\master\MYIOC\bin\windows-x64` depends on DLL files etc in `support` and these may change after a developer dependency update to our EPICS repositories, so just copying an EXE from usual EPICS_CLEAN jenkins will not work (c++ libraries expose functions from DLLs, these may change with a new version of e.g. EPICS base or asyn). If you are sure that the dependencies on the build server and instrument were the same, you can just copy a new EXE, but it may be safer to copy a static build of an EXE across as that includes all these DLL dependencies inside the EXE and so is self contained (if a lot larger). If you copy an EXE you should also copy the ioc's `dbd` file from the same build e.g. for INSTETC ioc this is copying `INSTETC/dbd/INSTETC-IOC-01.dbd` file as well as the `INSTETC/bin` EXE files. + +So for a new IOC do the following: + +* copy the new `support` directory from the `EPICS-CLEAN` jenkins build to the instrument e.g. `support\stanford_sr400_photon_counter` +* copy the `ioc` directory from `EPICS-CLEAN` jenkins build to the instrument e.g. copy all of `ioc\master\SR400` +* copy the just the ioc EXE files from the `bin` of the `EPICS-STATIC-CLEAN` build to replace those you copied from `EPICS-CLEAN` e.g. replace files on instrument in `ioc\master\SR400\bin\windows-x64` with the files from `EPICS-STATIC-CLEAN` jenkins directory `ioc\master\SR400\bin\windows-x64-static` (this will just be replacing exe, if a DLL is present in the original instrument directory don't worry about it) +* copy the dbd files i.e. copy all dbd files in `ioc\master\SR400\dbd` from `EPICS-STATIC-CLEAN` jenkins to equivalent location on instrument computer e.g. to `c:\instrument\apps\EPICS\ioc\master\SR400\dbd`. We need to copy dbd files as they relate to EPICS records in DLLs that are embedded. We can't copy all of the `EPICS-STATIC-CLEAN` tree as some bits have the wrong `EPICS_HOST_ARCH` embedded (as in windows-x64-static). So we need to do this hybrid approach. + +## Notes + +* on windows you cannot replace an EXE file if it is in use +* The list of IOCs to start is produced by a `start_ibex_server` and it also builds the list of ioc macros that the GUI reads. So you will need to run `start_ibex_server` for a new IOC or changed macros, and may have to restart the GUI after this too. \ No newline at end of file diff --git a/doc/deployment/patch/Release-Single-IOC.md b/doc/deployment/patch/Release-Single-IOC.md new file mode 100644 index 000000000..f02d94061 --- /dev/null +++ b/doc/deployment/patch/Release-Single-IOC.md @@ -0,0 +1,26 @@ +# Release a single IOC + +We may want to release a single IOC to an instrument but not release a whole build. This is the procedure to do this: + +1. Add all new code to master. +1. Wait for the overnight build. +1. Ensure that nothing has happened in the code between the release and this release that will cause it not to work + * the easiest way to do this is to look at the merge of master on to the release branch/tag. +1. Copy the needed IOC and support module (and anything else) from the relevant build server directory on the ICP share to the equivalent directories in a release build directory. e.g. + - from `...Kits$\CompGroup\ICP\EPICS\EPICS_CLEAN_win7_x64\BUILD-275\EPICS\ioc\master` + - to `...Kits$\CompGroup\ICP\Releases\Build275_Knaur_pump\ioc\master` +1. Copy these file on to the instrument. +1. Run in a epics terminal in EPICS `utils\build_ioc_startups.py` + - If this fails it can be because there is a duplicate IOC. The error message is rubbish! +1. Restart the IBEX server. +1. Test the IOC boots and connects + +## Troubleshooting + +### IOC does not Start + +Look at the IOC log this should give you a clue. If there is no log message in the IOC log, i.e. it just appears to be restarting itself. Start the IOC from the command line. If you then get a pop-up message box with: + + "xxx.dll" not found + +This means that a function has been added to a module that this IOC relies on to work. Locate the function and the module it belongs to then copy over this module and try again. This can now affect not just your IOC but others that rely on this new module; Make sure that this is what you want. diff --git a/doc/deployment/release/Creating-a-script-generator-release.md b/doc/deployment/release/Creating-a-script-generator-release.md new file mode 100644 index 000000000..ab19c74ac --- /dev/null +++ b/doc/deployment/release/Creating-a-script-generator-release.md @@ -0,0 +1,14 @@ +# Creating a script generator release + +The IBEX Script Generator is released as a zip file on GitHub (https://github.com/ISISComputingGroup/ScriptGeneratorReleases/releases). + +First, to create the release on the share [follow the instructions for the script generator](../Creating-a-release). + +To create a release run through create_release.py script from the [ScriptGeneratorReleases](https://github.com/ISISComputingGroup/ScriptGeneratorReleases) repo. For this script, you will need a GitHub personal access token with push access to the repo. + +You should run it with ` create_release.py -v -t `. Replacing `` with the version number e.g. `7.2.0` (you should leave out any `v`, this will be prepended by the script) and with your generated personal access token. + +There is also an optional argument for the script `-d ` where you can specify which drive you would like to mount the share to (you must add the colon) e.g. `U:`. This defaults to `Z:`. + +## Testing +After the release has been created, the standalone script generator must be tested in a non-developer environment. This emulates a user computer, where there is no pre-existing installation of python, git or the JVM. The instructions to do this testing are [here](/script_generator/Script-Generator-Testing-Environment) \ No newline at end of file diff --git a/doc/deployment/release/Release-based-repository.md b/doc/deployment/release/Release-based-repository.md new file mode 100644 index 000000000..2f4746b1f --- /dev/null +++ b/doc/deployment/release/Release-based-repository.md @@ -0,0 +1,33 @@ +# Release Repository on `control-svcs` + +## Creating repository + +If a release repo of the correct version is not shown on https://control-svcs.isis.cclrc.ac.uk/git/ then it can be added as follows + +e.g. for 12.0.1 +``` +robocopy "\CompGroup\ICP\Releases\12.0.1\EPICS\.git" "\\control-svcs.nd.rl.ac.uk\git$\releases\12.0.1\EPICS.git" /mir /nfl /ndl +``` +then edit EPICS.git\config and change `bare=false` to `bare=true` + +Also Add an extra section at end of the config file +``` +[http] + receivepack = true +``` +## Associating an instrument with release + +Newer deployment scripts do this automatically, if you are not sure if it has already been done run +``` +git remote -v +git branch +``` +and check output + +In instrument computer NDXEMMA-A is running release 12.0.1 for example then open git bash in c:\instrument\apps\epics and run +``` +git remote add origin http://control-svcs.isis.cclrc.ac.uk/gitroot/releases/12.0.1/EPICS.git +git checkout -b NDXEMMA-A +git push -u origin NDXEMMA-A +``` +`12.0.1` should agree with the local `VERSION.txt` file, we use computer name (i.e. NDX prefix, as per `COMPUTERNAME` environment variable) to be consistent with configurations \ No newline at end of file diff --git a/doc/favicon.ico b/doc/favicon.ico new file mode 100644 index 000000000..8ec971a02 Binary files /dev/null and b/doc/favicon.ico differ diff --git a/doc/index.md b/doc/index.md new file mode 100644 index 000000000..5808d76e9 --- /dev/null +++ b/doc/index.md @@ -0,0 +1,57 @@ +# IBEX Developer's Manual + +Welcome to the structured edition of the IBEX developer's manual. + +New here? See [the project overview](Overview) to get started. + +--- + +```{toctree} +:caption: Overview +:maxdepth: 1 +:titlesonly: + +Overview +Editing-the-Wiki +Glossary +``` + +```{toctree} +:caption: User Interfaces & Scripting +:maxdepth: 1 +:titlesonly: + +Client +Script-Generator +Scripting +Web +``` + +```{toctree} +:caption: IBEX Server & Systems +:maxdepth: 1 +:titlesonly: + +System-components +IOCs +Specific-IOCs +Systems +``` + +```{toctree} +:caption: Project Processes & Tools +:maxdepth: 1 +:titlesonly: + +Providing-Support +Deployment +Tools +Processes +``` + +## External Documentation + +- [IBEX project wiki](https://github.com/isiscomputinggroup/ibex/wiki) +- [IBEX user manual](https://github.com/isiscomputinggroup/ibex_user_manual/wiki) +- [Python shared scripts (also known as shared instrument scripts)](https://github.com/ISISNeutronMuon/InstrumentScripts/wiki) +- [Scientific Advisory Group (sharepoint)](https://stfc365.sharepoint.com/sites/IBEXSAG) diff --git a/doc/iocs/Compiling.md b/doc/iocs/Compiling.md new file mode 100644 index 000000000..2058e81d2 --- /dev/null +++ b/doc/iocs/Compiling.md @@ -0,0 +1,9 @@ +# Compiling + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +compiling/* +``` diff --git a/doc/iocs/Conventions.md b/doc/iocs/Conventions.md new file mode 100644 index 000000000..8a96207e8 --- /dev/null +++ b/doc/iocs/Conventions.md @@ -0,0 +1,9 @@ +# Conventions + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +conventions/* +``` diff --git a/doc/iocs/Creation.md b/doc/iocs/Creation.md new file mode 100644 index 000000000..08e10d756 --- /dev/null +++ b/doc/iocs/Creation.md @@ -0,0 +1,9 @@ +# IOC Creation + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +creation/* +``` diff --git a/doc/iocs/Testing.md b/doc/iocs/Testing.md new file mode 100644 index 000000000..4a34cd02c --- /dev/null +++ b/doc/iocs/Testing.md @@ -0,0 +1,9 @@ +# Testing + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +testing/* +``` diff --git a/doc/iocs/Tips-and-Tricks.md b/doc/iocs/Tips-and-Tricks.md new file mode 100644 index 000000000..b0b4862b6 --- /dev/null +++ b/doc/iocs/Tips-and-Tricks.md @@ -0,0 +1,9 @@ +# Tips, Tricks & Techniques + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +tips_tricks/* +``` diff --git a/doc/iocs/Tools.md b/doc/iocs/Tools.md new file mode 100644 index 000000000..de3534814 --- /dev/null +++ b/doc/iocs/Tools.md @@ -0,0 +1,9 @@ +# Tools & Support Modules + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +tools/* +``` diff --git a/doc/iocs/Troubleshooting.md b/doc/iocs/Troubleshooting.md new file mode 100644 index 000000000..411cecf4a --- /dev/null +++ b/doc/iocs/Troubleshooting.md @@ -0,0 +1,15 @@ +# Troubleshooting + +```{note} +This is **generic** troubleshooting information for IOCs & EPICS. + +For troubleshooting notes specific to a particular device, see [specific IOCs](../Specific-IOCs). +``` + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +troubleshooting/* +``` diff --git a/doc/iocs/compiling/Build-Arch.md b/doc/iocs/compiling/Build-Arch.md new file mode 100644 index 000000000..a66ca6a49 --- /dev/null +++ b/doc/iocs/compiling/Build-Arch.md @@ -0,0 +1,19 @@ +# Build Arch + +```{note} +This page documents an advanced compilation workflow. + +These instructions are unnecessary for a "standard" build +``` + +Build a directory under a given architecture instead of the default. This will switch the architecture temporarily to a given state for building. You can build multiple architectures within the same tree of EPICS (it will be built into `bin/`). Run: + +``` + build_arch +``` + +e.g. build_arch windows-x64-static + +Architectures allowed are given my the files in `EPICS/base/master/configure/os/CONFIG..Common` + +NB You must build `EPICS/base` in the given architecture first. diff --git a/doc/iocs/compiling/Building-multiple-architectures-(e.g.-32-&-64-bit-builds).md b/doc/iocs/compiling/Building-multiple-architectures-(e.g.-32-&-64-bit-builds).md new file mode 100644 index 000000000..dbae44e9e --- /dev/null +++ b/doc/iocs/compiling/Building-multiple-architectures-(e.g.-32-&-64-bit-builds).md @@ -0,0 +1,20 @@ +# Building multiple architectures (e.g. 32/64-bit builds) + +```{note} +This page documents an advanced compilation workflow. + +These instructions are unnecessary for a "standard" build +``` + +The EPICS build system allows you to have multiple architectures present in the same source tree, architecture independent files are built in O.Common and architecture dependent ones built in O.windows-x64 and O.win32-x86 etc. + +Running `config_env` selects a default `EPICS_HOST_ARCH` for builds, and typing "make" or running "build.bat" will use this. Any IOCs and start_ibex_server will also use this. + +The safe way to build another architecture in the same (or different) tree is to: +- Open a new command plain command windows (don't use an `epicsterm` link) and run `config_env` with the new architecture e.g. `config_env.bat win32-x86` +- This build tree will now default to your new architecture until you change it with `config_env` as above +- you can now `make` or `build.bat` + +**Note:** it used to be possible to just pass a new architecture to `build.bat` e.g. `build.bat win32-x86` but it looks like recent software packages don't support this and it will need looking at if it is to work again. So use above instead. + +If you need a 32 bit build in a hurry check out using a [Developer Server Build](Developer-Server-Build) diff --git a/doc/iocs/compiling/Compile-Errors.md b/doc/iocs/compiling/Compile-Errors.md new file mode 100644 index 000000000..7784dec7c --- /dev/null +++ b/doc/iocs/compiling/Compile-Errors.md @@ -0,0 +1,31 @@ +# Compile Errors + +## Using `callback.h` +If you get +``` +c:\instrument\apps\epics2\support\hvcaenx527\master\libhvcaenx527app\src\hvcaenx527.h(79): error C2059: syntax error: '__cdecl' +c:\instrument\apps\epics2\support\hvcaenx527\master\libhvcaenx527app\src\hvcaenx527.h(80): error C2059: syntax error: '}' +c:\instrument\apps\epics2\support\hvcaenx527\master\libhvcaenx527app\src\hvcaenx527chaio.c(42): error C2223: left of '->evntno' must point to struct/union +``` +when including `callback.h` it is because EPICS defines CALLBACK as a structure, windows defines it as `__cdecl`. If you need to include `windows.h` you must include it before epics redefines `CALLBACK` in `callback.h` +Including `osiSock.h` is a workaround as that pulls in `windows.h` but it may be better to be explicit with +``` +#ifdef _WIN32 +#include /* we need to make sure EPICS callback.h is loaded after windows.h */ +#endif +``` + +## `cmake error: could not load cache` seen during build + +Try deleting any `CMakeCache.txt` files in the appropriate directory + +## Can't build any IOCS + +When trying to use Make to build IOCs you might encounter an Error 2. The failing Make will look something like this: +``` +process_begin: CreateProcess(NULL, echo Generating runIOC.bat, ...) failed. +make (e=2): The system cannot find the file specified. +``` +This can be a result of having an environment path for git that points to `git/bin`. If it is, then make will think you are on linux and then the build will fail. You must change this to be `git/cmd` to point at the Windows binaries. + +See also [Ticket 4201](https://github.com/ISISComputingGroup/IBEX/issues/4201) for a potential fix. diff --git a/doc/iocs/compiling/Developer-Server-Build.md b/doc/iocs/compiling/Developer-Server-Build.md new file mode 100644 index 000000000..935904b94 --- /dev/null +++ b/doc/iocs/compiling/Developer-Server-Build.md @@ -0,0 +1,63 @@ +# Developer Server Build + +It is possible to use one of the intermediate outputs from a build server as the basis of your local development environment. This can provide you with either a normal `Release` build (EPICS_HOST_ARCH=windows-x64), or a `Debug` build (EPICS_HOST_ARCH=windows-x64-debug) or a different architecture such as 32bit (EPICS_HOST_ARCH=win32-x86) + +## EPICS +You need to be using a Visual Studio compatible with the build server, currently `Visual Studio 2022` + +> [!TIP] +>### Quick Instructions +> Run `install_developer_build.bat` in either `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\EPICS` (for 64bit builds, what you usually need) or `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\EPICS32` (for 32 bit builds, only if you have to build/test a specific driver needing 32bit versions). It will prompt you to choose from normal/debug/static - in most cases `normal` +is what you want and is what is deployed to an instrument, `static` and `debug` are for special cases/testing + + +> [!NOTE] +> if you just want a recent EPICS build to run somewhere and do not need to be able to develop using it, you just need to run the `install_to_inst.bat` in the appropriate build on `kits$` when connected from the machine you want to install it on + +
Details (not normally needed) + +If you wish to use a `debug` build, replace `x64` in paths below with `x64-debug` + +The EPICS clean Windows 10 Jenkins build workspace is copied to `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\EPICS` - this contains +much more than an instrument installation e.g. it has all the local `.git` submodules and `*.lib` link libraries as well as temporary build (`O.*`) object file directories. It can thus be used as the basis for rapid further development work on your local computer. A standard instrument debug install for example would allow you to run an ioc in debug mode, but any further development would require re-generating the relevant `*.lib` files need to link the IOC and so take longer than using a developer debug build. + +CAUTION: updating an existing `C:\Instrument\Apps\EPICS` by the mechanism described here is the equivalent of deleting the directory, a full new `git clone --recursive` and then a build. You will lose all current changes to files and also any local git branches you created in that directory. Any changes there you want to keep you must push to Github and then re-checkout the branch after the update. Stashing changes to files is not enough as the local git repos are replaced - you need to push remotely. + +you would need to find an appropriate build number in `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\EPICS\windows-x64` and then run e.g. +``` +robocopy \\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\EPICS\windows-x64\BUILD-1 c:\Instrument\Apps\EPICS -MIR -NFL -NDL -NP -R:1 -MT -LOG:NUL +``` +As an extra check you can re-run the robocopy without the LOG option. This should not take too long and should not print any errors or have any entries in the Mismatch/FAILED/Extras columns displayed at the end. + +you should have a compiled and ready to use distribution, with git pointing at current commit heads. +This may take a while to complete - at least 10 minutes, but longer if your disk is not an SSD for example. + +**Alternative for slow network connection** + +You can instead copy `EPICS-windows-x64.7z` from `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\zips` and unpack this locally using +the [7-zip](https://www.7-zip.org/) program. Either drag the file using windows explorer, or if you have a very slow connection or one that might get interrupted you can try using `robocopy` in _restartable mode_. Open a cmd prompt, change to the relevant directory and type: +``` +robocopy "\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\developer\zips" "." EPICS-windows-x64.7z /J /Z +``` +Note that this pre-allocates the full file space before starting the copy. I found that if I interrupt the copy with Ctrl-C and then type the command again, it picks up where it left off, so it looks hopeful it will handle network connection breaks. +
+ +### Using Files + +After copying note that all submodules will be on a detached HEAD. + +If you wanted to temporarily use an updated distribution for e.g. a review then you can: +- rename current `c:\Instrument\Apps\EPICS` to `c:\Instrument\Apps\EPICS-keep` +- use one of the above procedures to create a new `c:\Instrument\Apps\EPICS` +- checkout branches required for review, run make in just these directories if required +- when done delete `c:\Instrument\Apps\EPICS` and rename `EPICS-keep` back to `EPICS` + +This scheme works as Visual Studio is binary compatible (even at object file level) from version 2015 onwards. Linking must be done with the most recent visual studio version used, the build server is currently running 2019, so any developer must also be using Visual Studio 2019. + + +
NOTES + +We now copy `CMakeCache.txt` across, if we later again have different developer and build server versions of visual studio then we will again need to not copy it as it is invalid if the visual studio version number is different (2017 v 2019). + +CMake is used in a few third party modules e.g. MySQL, gsl, OpenCV. Compatible binaries for these will always be copied across, so IOCs can be compiled and linked, but if `CMakeCache.txt` is missing a `make` in the top level will rebuild these modules, some of which do take a while. +
\ No newline at end of file diff --git a/doc/iocs/compiling/Developer-Updating.md b/doc/iocs/compiling/Developer-Updating.md new file mode 100644 index 000000000..2e02e4262 --- /dev/null +++ b/doc/iocs/compiling/Developer-Updating.md @@ -0,0 +1,73 @@ +# Updating your local IBEX EPICS directory + +## using a pre-built distribution + +If you just want a clean start then you can use a pre-built build server checkout [[Developer-Server-Build]] as a new starting point. This is by far the easiest - it is the equivalent of deleting your current EPICS directory, doing a clean checkout and then building, just a bit quicker as it has already been built/compiled for you on the build server. If you have no important local changes (i.e. everything is pushed to github) then you will lose nothing and can just swap back to these branches after the new install. + +## updating in-place + +Here we will update all modules from git to the latest version and then rebuild. This however does not always proceed smoothly, mainly due to issues with the directory tree that is being updated. Some things that can cause problems are: +- a local directory has changed files and git will not want to overwrite these +- upstream has renamed a directory, git likes to do a delete and re-create, but it cannot do the delete if the directory is not empty (i.e. has some build products like .obj files in it or a subdirectory) + +So you need to make sure that your directory tree is in a good state. You can probably do this iteratively after each failure as you update, or you can do it all first. +- make sure all work is pushed. You don't usually need to reset branches back to master/main, but may be a good point to do this if work is finished +- particularly after a dependency update where the likelihood of directory name changes is greatest, remove all build products: either `make clean uninstall` or `git submodule foreach --recursive "git clean -fdx"` from top directory. The git command is probably faster, but removes all files not under version control so you need to be sure you haven't forgotten to `git add` and push something to a remote branch. + +To check what branches all your submodules are on: + +`git submodule foreach --recursive "git branch" | egrep "^Entering|^\*"` + +If you don't see either "master" or "detached HEAD" then you need to decide if you really want to be on that branch, or just forgot to swap back after reviewing a ticket. If you are leaving a module on a non main/master branch, it may be worth merging the latest upstream main/master into this branch. + +To update, from a git shell at top do: + + git pull + git submodule init + git submodule update --init --recursive + +You can use + + git submodule update --init --recursive --merge + +if you wish to leave items on their branch and merge in upstream, if you leave off `--merge` you will get switched to a detached HEAD at the upstream version. + +If you want to remove local changes as they are causing issues, you can use `git reset --hard` in a module or `git submodule foreach --recursive "git reset --hard"` to do all from the top level. + +For info, Jenkins does the following to update a git tree on a clean build: +``` +git reset --hard +git clean -fdx +git submodule foreach --recursive "git reset --hard" +git submodule foreach --recursive "git clean -fdx" +git fetch +git checkout master +git submodule init +git submodule sync +git submodule update --init --recursive +git reset --hard +git clean -fdx +git submodule foreach --recursive "git reset --hard" +git submodule foreach --recursive "git clean -fdx" +``` + +Now build IBEX in the usual way. See [[Things-to-know-as-a-developer]] for speeding up a build. + +### Troubleshooting + +typing `git status` should not show unexpected changes being flagged. + +If you get errors, the most likely causes are: +1. A file was renamed, but EPICS has not updated its Makefile dependency rules and is still looking for the old one +2. A directory was renamed/deleted, but the original directory is still on your computer because it was not empty + +For 1. a "make clean uninstall" in the directory concerned should be enough +For 2. You need to go to the directory and do a "git status" and remove any "untracked files" that shouldn't be there. The "git clean" command can be used for this, "git clean -fd" will remove all untracked files and directories. "git clean -fdx" will additionally remove files ignored by `.gitignore` (such as compiler build output). Some of our submodules now contain submodules of their own, if a `git status` keeps showing something as modified you may need to `git submodule update` in this module to update that directory properly. + +In many cases problems will be isolated to a particular module and you do not need to build everything again. If you are about to go home and want to set off a complete rebuild then you can do the following (assuming you have no untracked files you are working on and have forgotten to add to git) +``` + git clean -fdx + git submodule foreach --recursive "git clean -fdx" +``` +this simulates a clean checkout (its actually what Jenkins does too) + \ No newline at end of file diff --git a/doc/iocs/compiling/IOC-Libraries-to-include-with-order.md b/doc/iocs/compiling/IOC-Libraries-to-include-with-order.md new file mode 100644 index 000000000..11ab068ad --- /dev/null +++ b/doc/iocs/compiling/IOC-Libraries-to-include-with-order.md @@ -0,0 +1,16 @@ +# IOC Libraries to include with order + +The following is a list of libraries to include and their order for different record types. These are added to the build.mak in the src directory of the ioc app. The order is important in the static build. + +## In general +1. seq and pv should be listed last as any module may add sequencer support at some point +1. calc now depends on sscan so must be listed before sscan (it also depends on seq and pv too) + +## specific rules + +### Lib section + +1. `scalc`: $(APPNAME)_LIBS += calc sscan + +### DBD section +1. `scalc`: $(APPNAME)_DBD += calcSupport.dbd diff --git a/doc/iocs/compiling/Install-Visual-Studio.md b/doc/iocs/compiling/Install-Visual-Studio.md new file mode 100644 index 000000000..1a99e7572 --- /dev/null +++ b/doc/iocs/compiling/Install-Visual-Studio.md @@ -0,0 +1,58 @@ +# Install Visual Studio + +Most developers should install VS2022 - we have 2010 on a build server for building the old galil driver (until we remove it), if you need to install 2010 locally then install it before you install 2022 + +## Instructions for Visual Studio 2010 (not needed in most cases) + +Install SDK 7.1 +-The SDK may fail if you have these installed: +* Microsoft Visual C++ 2010 x86 Redistributable +* Microsoft Visual C++ 2010 x64 Redistributable + +If these do exist on your computer you need to uninstall them before installing the SDK. + +Install: +* [Visual Studio 2010 SP1](https://my.visualstudio.com/Downloads?q=visual%20studio%202010&pgroup=) +* Visual C++ 2010 SP1 Compiler Update for the Windows SDK 7.1 + +Note: The free version of Visual Studio 2010, Visual Studio 2010 Express, isn't supported by the EPICS build process. + +## Instructions for Visual Studio 2022 +Download installer (Visual Studio **Community** 2022 exe) from installer page https://my.visualstudio.com using your stfc email address. +During install, select "Desktop development with C++" and check/enable these individual features if they are not already: +- The most recent Windows 11 SDK +- C++ MFC support +- C++ ATL support + +Also select ".NET desktop development" (needed for mk3chopper and Astrium chopper): +- make sure .NET 4.7.2 development is included as one of the options + +## Instructions for Visual Studio 2019 (old) +Download installer (Visual Studio **Community** 2019 exe) from installer page https://my.visualstudio.com using your stfc email address. +During install, select "Desktop development with C++" and check/enable these individual features: +- The most recent Windows 10 SDK +- C++ MFC support +- C++ ATL support + +Also select .NET development and "Universal windows CRT/universal platform development" from the features + +NOTE: If you are having issues running the installer, moving the .exe from 'Downloads' to 'Documents'. + +## Installing a new Version of Visual Studio +If you are on a newer version of Visual Studio then you will need to upgrade various files to take this into account. Below is a list of things we needed to do when upgrading to 2017 your list may be different. + +See https://github.com/ISISComputingGroup/IBEX/issues/5173 for the changes that were necessary to add support for Visual Studio 2019. + +### Setup the environment (Only if you are going for a later version) + +The visual studio compiler environment variables are set up from `...\EPICS\base\master\startup\windows.bat` this calls into the visual studio variable set up. Add your version to this. + +### Converting Tabs to Spaces + +To convert tabs to spaces inside of Visual Studio go to + +`Tools->Options->Text Editor->All Languages->Tabs` + +Change Tab to use "Insert Spaces" instead of "Keep Tabs". + +Note you can also specify this per language if you wish to have different behaviour in a specific language. \ No newline at end of file diff --git a/doc/iocs/compiling/Reducing-Build-Dependencies.md b/doc/iocs/compiling/Reducing-Build-Dependencies.md new file mode 100644 index 000000000..ac9e3920b --- /dev/null +++ b/doc/iocs/compiling/Reducing-Build-Dependencies.md @@ -0,0 +1,32 @@ +# Reducing build dependencies + +This is a page describing how to reduce dependencies in our EPICS build. Currently we have a MASTER_RELEASE file at the top level that, though convenient, means the build system does a lot of cross-checking for things it doesn't need, disabling checkRelease can speed this up. + +A simple way to reduce dependencies is to remove the +``` +include $(TOP)/../../../configure/MASTER_RELEASE +-include $(TOP)/../../../configure/MASTER_RELEASE.$(EPICS_HOST_ARCH) +-include $(TOP)/../../../configure/MASTER_RELEASE.private +-include $(TOP)/../../../configure/MASTER_RELEASE.private.$(EPICS_HOST_ARCH) +``` +lines and replace them by the lines that were in MASTER_RELEASE that are needed for the build and runtime (e.g. ioc loading db files) e.g. +``` +ASYN=$(SUPPORT)/asyn/master +``` +To determine what you need to add: +* look at the Makefile to see what DBD and library files are used when building an IOC and then add the relevant macro definitions from MASTER_RELEASE to the new RELEASE to allow these to be located. See the _LIB and _DBD macros +* Look at DB template *.substitution files in the App/Db directory to see where DB files are included from, you'll need any macros from here too +* Check st.cmd for use of such macros e.g. `$(ACCESSSECURITY)` and other DB files it may load and add these macros to the new RELEASE too + +For the first step you can use the script in `ibex_utils/developer_support_script` which will add the lines to RELEASE based on libs and dbd files in you build.mak which it knows about. This script can be run multiple times. To run use: + + python build_dependencies.py + +If you get it wrong / miss something, it will usually just break the build due to a missing file; however if you miss a macro used in st.cmd it will not be noticed until the IOC is run. So you will need to run the IOC up in simulation mode and check for any warnings about undefined macros or commands. + +If the module you are changing is using AREA_DETECTOR or EPISC_V4 then things a little more complicated, I'll add extended notes at a later point. + +after any change to RELEASE do a "make clean uninstall" followed by "make" to check + +Something I mean to look more closely at for ideas is http://epics-sumo.sourceforge.net/ + diff --git a/doc/iocs/compiling/Removing-or-Renaming-IOC-module.md b/doc/iocs/compiling/Removing-or-Renaming-IOC-module.md new file mode 100644 index 000000000..47a9b5669 --- /dev/null +++ b/doc/iocs/compiling/Removing-or-Renaming-IOC-module.md @@ -0,0 +1,6 @@ +# Removing or renaming an IOC module + +If you rename or remove a support module, you will need to edit the Makefile in e.g. support/Makefile or ISIS/Makefile +The Makefile contains a default list of modules for all architectures, then additional modules based on build architecture can be added e.g. modules only for Windows + +If you rename or remove an IOC, you will need to edit the Makefile in ioc/master/Makefile This Makefile contains a variable IOCDIRS that lists all potentially valid IOC directories, then other _NOTBUILD variables are used to remove certain builds on certain architectures. \ No newline at end of file diff --git a/doc/iocs/compiling/SpeedUpBuilding.md b/doc/iocs/compiling/SpeedUpBuilding.md new file mode 100644 index 000000000..09fcbd01e --- /dev/null +++ b/doc/iocs/compiling/SpeedUpBuilding.md @@ -0,0 +1,13 @@ +# Speed Up Building + +```{note} +This page documents an advanced compilation workflow. + +These instructions are unnecessary for a "standard" build +``` + +On windows it takes some time to do the checkRelease part of the build, which in our case is duplicated work as we use a MASTER_RELEASE file. + +To disable this turn off the check release flag (DO NOT CHECK THIS CHANGE IN). Edit line 181 of `...EPICS\base\master\configure\CONFIG_site` and uncomment the line to read: + + CHECK_RELEASE_YES = noCheckRelease diff --git a/doc/iocs/compiling/Static-builds.md b/doc/iocs/compiling/Static-builds.md new file mode 100644 index 000000000..160e70cab --- /dev/null +++ b/doc/iocs/compiling/Static-builds.md @@ -0,0 +1,13 @@ +# Static Builds + +```{note} +This page documents an advanced compilation workflow. + +These instructions are unnecessary for a "standard" build +``` + +Static builds are where all the code is built into the executable and not sometimes loaded at runtime from a shared DLL. Thus statically linked executables will be much bigger than ones using DLLs, but will have less dependencies. + +Though we don't use them directly in IBEX, it is a future option and it is useful to keep the ability to use them. It also makes our dependencies much clearer in the following way. If IOC A depends on library B and library B depends on library C then creating a DLL for library B will hide the library C dependency from IOC A, you only need to link against B as C will be activated at runtime automatically. For a static build as everything must be put into A you must add library C to the IOC Makefile. + +In IBEX we will often see this when adding the "calc" module - this has a dependency on the "sscan" module, so you need to add both "calc" and "sscan" to _LIBS for windows-x64-static builds to work, the windows-x64 builds will work fine with just "calc" added to _LIBS diff --git a/doc/iocs/compiling/Using-Visual-Studio-with-the-EPICS-build-system.md b/doc/iocs/compiling/Using-Visual-Studio-with-the-EPICS-build-system.md new file mode 100644 index 000000000..e961d63cd --- /dev/null +++ b/doc/iocs/compiling/Using-Visual-Studio-with-the-EPICS-build-system.md @@ -0,0 +1,21 @@ +# Using VS with EPICS + +Some of the EPICS modules are written in C++ and so it makes sense to use Visual Studio to edit these. To correctly set up Visual Studio to edit a project you should do the following: + +1. Start an EPICS terminal e.g. `C:\Instrument\Apps\epics\EPICSTerm.bat` +1. Start Visual Studio in this EPICS terminal by running `devenv` +1. Go to File -> New -> Project from Existing Code +1. Select C++ and press Next +1. Select the top directory for the project file location e.g. `C:\Instrument\Apps\epics\support\sampleChanger\master` +1. Give the project a sensible name and press Next +1. Select `Use external build system` and press Next +1. As the build command line type `make` and as the clean command type `make clean uninstall` and press Finish + +The code will now be in Visual Studio and you should be able to Build it from the Build menu at the top. To point Visual Studio at the dependencies of the code do the following: + +1. Open the RELEASE file for the project e.g. `C:\Instrument\Apps\epics\support\sampleChanger\master\configure\RELEASE` +1. In VS right click on the project and select properties -> VC++ Directories +1. In the include directories add an entry for each line in the RELEASE file (apart from the optional extras ones) that points to the include directory of that submodule (remembering that `$(SUPPORT)` is `C:\Instrument\apps\epics\support`) e.g. you will add `C:\Instrument\Apps\epics\support\asyn\master\include` (note that in reality a lot of these includes will not be needed for the C++ code, the following will definitely not: `AUTOSAVE`, `CAPUTLOG`, `DEVIOCSTATS`, `ICPCONFIG`, `MYSQL`, `SQLITE`, `PVDUMP`) +1. Add `C:\Instrument\Apps\epics\base\include` to the include directories + +Visual Studio should now pick up all the dependencies and so give you `intellisense`/autocomplete etc. \ No newline at end of file diff --git a/doc/iocs/compiling/building-on-linux.md b/doc/iocs/compiling/building-on-linux.md new file mode 100644 index 000000000..cf77d3fc5 --- /dev/null +++ b/doc/iocs/compiling/building-on-linux.md @@ -0,0 +1,187 @@ +# Building on Linux + +```{note} +This page documents an advanced compilation workflow. + +These instructions are unnecessary for a "standard" build +``` + +## Creating a linux box on your desktop + +1. Install virtual box +1. Install vagrant (https://www.vagrantup.com/downloads.html) +1. Clone repo (https://github.com/ISISComputingGroup/ibex_utils.git) +1. cd .../ibexutils/linux_env +1. vagrant up (this may not work from the Windows command line, but it should work from the Git Bash) + +Should start a fresh vagrant machine (will take a while but is quicker second time round). With a gui you can log into. Login details are: `vagrant` (for username) and `vagrant` (for password). + +If you get the following error: +``` +VT-x is disabled in the BIOS for all CPU modes (VERR_VMX_MSR_ALL_VMX_DISABLED). + +Result Code: +E_FAIL (0x80004005) +Component: +ConsoleWrap +Interface: +IConsole {872da645-4a9b-1727-bee2-5585105b9eed} +``` +you should try entering your BIOS (power off your machine; restart and immediately press the `F2` key) and verify that virtualisation options are enabled. See also [this related post] (https://forums.virtualbox.org/viewtopic.php?f=6&t=77139). + +If you get the following error: +``` +==> default: Errors were encountered while processing: +==> default: dictionaries-common +==> default: aspell +==> default: aspell-en +==> default: hunspell-en-us +==> default: libenchant1c2a:amd64 +==> default: libwebkitgtk-3.0-0:amd64 +==> default: empathy +==> default: mcp-account-manager-uoa +==> default: account-plugin-aim +==> default: account-plugin-jabber +==> default: account-plugin-salut +==> default: account-plugin-yahoo +==> default: gir1.2-webkit-3.0 +==> default: apturl +==> default: enchant +==> default: gedit +==> default: librhythmbox-core8 +==> default: gir1.2-rb-3.0 +==> default: libyelp0 +==> default: yelp +==> default: gnome-user-guide +==> default: zenity +==> default: nautilus-sendto-empathy +==> default: nautilus-share +==> default: rhythmbox +==> default: rhythmbox-mozilla +==> default: rhythmbox-plugin-cdrecorder +==> default: rhythmbox-plugin-magnatune +==> default: rhythmbox-plugin-zeitgeist +==> default: rhythmbox-plugins +==> default: software-center +==> default: ubuntu-release-upgrader-gtk +==> default: unity-control-center +==> default: ubuntu-desktop +==> default: ubuntu-docs +==> default: unity-control-center-signon +==> default: webaccounts-extension-common +==> default: xul-ext-webaccounts +==> default: shotwell +==> default: update-notifier +==> default: update-manager +==> default: indicator-bluetooth +==> default: E: Sub-process /usr/bin/dpkg returned an error code (1) +The SSH command responded with a non-zero exit status. Vagrant +assumes that this means the command failed. The output for this command +should be in the log above. Please read the output to determine what +went wrong. +``` +it may be that some packages failed to install within the virtual machine, but you still should be able to operate without them. In the same console where you started vagrant, type: + +1. `vagrant halt` +1. `vagrant up` + +and the VM should start normally. + + +## Installing the client + +I have not done this from scratch you will need to experiment the following is what I did before I ran out of time on the ticket. Ideally I think the plan is to place the compiled version of the client and an install script into the linux_env directory then on the Ubuntu machine this appears in `/vagrant` (not home directory). You should be run install.sh and it should install and possibly start the client. To do this you will need at least java installed but I am unsure what else. see the instructions below of how I built and started my client. + +## Create virtual machine + +I am currently on a windows machine so it is useful to have a virtual machine on which to install. Using Michaels vagrant script seems like a good choice. So: + +1. Install virtual box +2. Install vagrant (https://www.vagrantup.com/downloads.html) +3. Clone repo (https://github.com/DMSC-Instrument-Data/ecp-dev-envs) +4. cd directory +5. change virtual machine to use + +``` +config.vm.box = "ubuntu/trusty64" +config.vm.box_url = "ubuntu/trusty64" #"../../vagrant_ubuntu_minimal/package.box" +``` + +6. cd `.../ecp-dev-envs\nicos-dev-env` +7. `vagrant up` +8. `vagrant ssh` from a git bash terminal + +## Building it + +Install prerequisites (this for the backend and front end so not all of these are needed for the client) +``` +sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + build-essential \ + cmake \ + conserver-client \ + conserver-server \ + default-jdk \ + doxygen \ + git \ + ipython \ + libcurl4-openssl-dev \ + libjpeg8-dev \ + libnet1-dev \ + libpcap-dev \ + libreadline-dev \ + libxml2-dev \ + libxslt1-dev \ + maven \ + mysql-server-5.6 \ + perl \ + php5 \ + php5-dev \ + procServ \ + python \ + python-dev \ + python-matplotlib \ + python-pip \ + python-tk \ + re2c \ + swig \ + python-docutils \ + rst2pdf +``` + +clone epics repo + + git clone https://github.com/ISISComputingGroup/ibex_gui.git + +alter preference supplier to be linux aware (there is an extra path in here to which needs to be updates for epics utils) +``` +cd "$INSTALL_DIR/ibex_gui" +PREFERENCE_SUPPLIER="./base/uk.ac.stfc.isis.ibex.preferences/src/uk/ac/stfc/isis/ibex/preferences/PreferenceSupplier.java" + + sed -i -- "/DEFAULT_EPICS_BASE_DIRECTORY/s|\".*\"|\"$INSTALL_DIR/EPICS/base/master/bin/linux-x86_64\"|" $PREFERENCE_SUPPLIER + sed -i -- '/DEFAULT_PYTHON_INTERPRETER_PATH/s|".*"|"/usr/bin/python"|' $PREFERENCE_SUPPLIER + sed -i -- "/DEFAULT_GENIE_PYTHON_DIRECTORY/s|\".*\"|\"$PACKAGE_DIR/genie_python\"|" $PREFERENCE_SUPPLIER + sed -i -- "/DEFAULT_PYEPICS_DIRECTORY/s|\".*\"|\"$PACKAGE_DIR\"|" $PREFERENCE_SUPPLIER +``` + +Build the client (this must be done in a gui environment for all the tests to pass) +``` +cd build +./build.sh +``` + +Install python modules: + + ./install_python_modules.sh (not all of these install but this seemed not to be a problem install those that work) + +Add genie python to default python build: +``` +GENIE_DIR="$INSTALL_DIR/genie_python/source" +PACKAGE_DIR=`python -c "import site; print site.getsitepackages()[0]"` +sudo cp -r "$GENIE_DIR" "$PACKAGE_DIR/genie_python" +``` + +Run product: + +`ibex_gui\base\uk.ac.stfc.isis.ibex.client.product\target\products\ibex.product\linux\gtk\x86_64\ibex-client` + + diff --git a/doc/iocs/conventions/IOC-Descriptions.md b/doc/iocs/conventions/IOC-Descriptions.md new file mode 100644 index 000000000..ec18cdebc --- /dev/null +++ b/doc/iocs/conventions/IOC-Descriptions.md @@ -0,0 +1,12 @@ +# IOC Descriptions + +In order to make sure that IOC descriptions, and how they appear in the IOC selection list is understandable the IOC Description in the config.xml should consist of the following items, in this order: + +1. Manufacturer (for example Agilent) or originating facility (typically this will be ISIS) +2. Model/Series for devices (for example E3613A), IBEX or instrument/group - this part can be omitted if it would create more confusion that it causes for a particular IOC +3. What it is/does (e.g. waveform generator, jaws manager) + +### Examples of IOC Descriptions +* Agilent 33220A 20MHz waveform generator +* ISIS IBEX Reflectometry Server +* ISIS DAE \ No newline at end of file diff --git a/doc/iocs/conventions/IOC-Naming.md b/doc/iocs/conventions/IOC-Naming.md new file mode 100644 index 000000000..b95bf832d --- /dev/null +++ b/doc/iocs/conventions/IOC-Naming.md @@ -0,0 +1,10 @@ +# IOC Naming + +An example IOC name is `HMEG8123`, to which a suffix like `_01`, `_02` is automatically appended. New IOC names should be max 8 characters long (excluding the `_01` suffix) and be in uppercase letters. The same name (with the same casing) should be used also for the top-level IOC directory under `EPICS\ioc\master`. + +You don't have to use the full 8 characters, use the smallest number that form a sensible truncated representation of the name that captures its overall form. Often this can be obtained by removing vowels (in general remove all the vowels not just some), generally just truncating the name as is doesn't always look good. You may also be able to shorten a model name or use the general model series if that is applicable e.g. `Eurotherm2000` -> `eurotherm2k`, `TPG3xx` etc or just TPG if it is model series independent. + +Support modules may have a slightly longer name if need be, but if truncated please follow guidelines above. + +If an IOC is *truly* instrument specific (i.e. there is absolutely no chance that this equipment will ever be used on any other instrument), the first three letters of the IOC name can indicate the instrument. E.g. EGX for Engin-X or RKN for Riken. + diff --git a/doc/iocs/conventions/IOC-Startup.md b/doc/iocs/conventions/IOC-Startup.md new file mode 100644 index 000000000..55ab62290 --- /dev/null +++ b/doc/iocs/conventions/IOC-Startup.md @@ -0,0 +1,6 @@ +# IOC Startup + +In the vast majority of cases an IOC should start up and not set values on the device. Values should only be set when the set point PV is processed. Exceptions to this are: + +- Format: if multiple formats can be specified the format returned can be set on startup + - This may alternatively be done in a mismatch handler diff --git a/doc/iocs/conventions/ISIS-PV-Guide.md b/doc/iocs/conventions/ISIS-PV-Guide.md new file mode 100644 index 000000000..bf99f3bfa --- /dev/null +++ b/doc/iocs/conventions/ISIS-PV-Guide.md @@ -0,0 +1,114 @@ +# ISIS PV Guide + +**This does not cover the PV naming convention, rather it covers the PVs you need to implement.** + +The PV naming conventions are described on the [PV naming](PV-Naming) page. + +1. For a slow changing value (e.g. a temperature), there needs to be a current value readback, a setpoint and a setpoint readback: + + ``` + record(ai, "$(P)TEMP") + { + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@devEuro.proto getCurrentTemp $(PORT)") + field(PREC, "3") + field(EGU, "K") + } + + record(ao, "$(P)TEMP:SP") + { + field(DTYP, "stream") + field(OUT, "@devEuro.proto setTempSetpoint $(PORT)") + field(PREC, "3") + field(EGU, "K") + } + + record(ai, "$(P)TEMP:SP:RBV") + { + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@devEuro.proto getTempSetpoint $(PORT)") + field(PREC, "3") + field(EGU, "K") + } + ``` + +1. For a value that changes instantly (e.g. a trigger level), there needs to be a current value readback, a setpoint and a setpoint readback. However the setpoint readback should be an alias for the readback: + + ``` + record(ai, "$(P)TRIG_LVL") + { + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@devHameg_8123.proto getTriggerLevel $(PORT)") + field(PREC, "3") + field(EGU, "V") + } + + record(ao, "$(P:)TRIG_LVL:SP") + { + field(DTYP, "stream") + field(OUT, "@devHameg_8123.proto setTriggerLevel $(PORT)") + field(PREC, "3") + field(EGU, "V") + } + + alias("$(P)TRIG_LVL", "$(P)TRIG_LVL:SP:RBV") + ``` + +1. For a toggle-style button (e.g. voltage on/off) where the current setting can be read, use the same format as 2) but with binary records: + ``` + record(bi, "$(P)VOLTAGE_ON") + { + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@devPSU.proto getVoltageOn $(PORT)") + field(ZNAM, "OFF") + field(ONAM, "ON") + } + + record(bo, "$(P)VOLTAGE_ON:SP") + { + field(DTYP, "stream") + field(OUT, "@devPSU.proto setVoltageOn $(PORT)") + field(ZNAM, "OFF") + field(ONAM, "ON") + } + + alias("$(P)VOLTAGE_ON", "$(P)VOLTAGE_ON:SP:RBV") + ``` + +1. For a toggle-style button where the current setting ```cannot``` be read or a push-style button (e.g. a reset button), there needs to be a setpoint and, for convenience, an alias for the current value. This allows the value to be set using either PV. + ``` + record(bo, "$(P)RESET:SP") + { + field(DTYP, "stream") + field(OUT, "@devMyDevice.proto reset $(PORT)") + field(ZNAM, "YES") + field(ONAM, "TRUE") + } + + alias("$(P)RESET:SP", "$(P)RESET") + ``` + +1. A read-only value (e.g. a status string). + ``` + record(stringin, "$(P)STATUS") + { + field(DTYP, "stream") + field(INP, "@devMyDevice.proto getStatus $(PORT)") + field(SCAN, "1 second") + } + ``` + +1. For a value(s) which the user wants to review before changing then press a set button (for motor records this can be done via [SPMG](https://epics.anl.gov/bcda/synApps/motor/R6-5/motorRecord.html#Fields_command)): + - `XXX` - read back + - `XXX:SP` - set the value immediately + - `XXX:SP:RBV` - the value that the device has as a setpoint + - `XXX:SP_NO_ACTION` - don't set the value but wait (this is a temporary value if you restart the IOC it is lost, consider putting autosave on it) + - `XXX:ACTION` - when written to with any value take the SP_NO_MOVE and send it to the device + - `XXX:ACTION:SP` - when written to with any value take the SP_NO_MOVE and send it to the device + - `XXX:CHANGED` - are the setpoint on device and current setpoint different? It is an enum: + - `NO` there is no change to apply + - `YES` there is a change to apply diff --git a/doc/iocs/conventions/Macro-Naming.md b/doc/iocs/conventions/Macro-Naming.md new file mode 100644 index 000000000..a4e01e1d6 --- /dev/null +++ b/doc/iocs/conventions/Macro-Naming.md @@ -0,0 +1,58 @@ +# Macro Naming + +As well as the [macros used in PVs](PV-Naming) there are other Macros that we should set as standard. When an IOC is acquired from another source they may use the same term for different items, we should probably do some fancy substitutions so that our macro setting locations (via the configurations editor in the GUI, or via globals.txt) remain consistent. These eventually take a format, which is easily seen in globals.txt, of IOC_0n__MACRO=value, where n is the indicator of which macro is under consideration. (Note, existing IOCs may not conform, and may need to be updated at some point.) + +Once a macro has been set consider whether it should be added into the configuration so the user can set it. To do this see the instructions in [IOC Finishing Touches](../creation/IOC-Finishing-Touches). Some macros follow a common enough format across multiple devices that they can simply be included from `...\EPICS\ioc\master\COMMON\`, the syntax for this can also be found in IOC Finishing Touches. + +Where there are multiple instances of a device within an IOC (e.g. SDTEST, GENESYS) then a number should be added to the end of the macro to indicate which macro it is. + +| Macro | Description | Default Value (for st.cmd) | Can be included from COMMON +| --- | --- | --- | --- | +| `DEV_TYPE` | The Device Type where one IOC handles multiple devices (e.g. the Danfysik PSU IOC) | Varies by device | No (Not in config.) | +| `PORT` | The COM port to use | Do not set this to a default | Yes (`PORT.xml`) | +| `HOST` | The host name or IP address of the device, can include the port separated by a colon | Do not set this to a default | Yes (`HOST.xml`) | +| `ADDR` | Secondary address information, e.g. on RS485 | Varies by device | No | +| `BAUD` | The Baud rate of a serial device | Varies by device | Yes (if default of 9600, `BAUD9600.xml`) | +| `BITS` | The number of bits in the serial interface | Varies by device | Yes (If default 8, `BITS8.xml` | +| `PARITY` | The parity of the serial interface | Varies by device | Yes (If default of None, `PARITYNONE.xml`) | +| `STOP` | The number of stop bits used by the serial interface | Varies by device | Yes (If default of 1, `STOP1.xml` | +| `IEOS` | The end of line signal for input within asyn | Varies by device | No | +| `OEOS` | The end of line signal for output within asyn | Varies by device | No | +| `SOFTFLOWCNTL` | Software (xon/xoff) flow control within asyn | Varies by device | No | +| `HWFLOWCNTL` | Hardware (crtscts) flow control within asyn | Varies by device | No | +| `OEOS` | The end of line signal for output within asyn | Varies by device | No | +| `CALIB_PATH` | The path to the folder containing calibration files for that IOC/function (e.g. resistance/temperature or current/magnetic field) | Varies by type of calibration | No | +| `CALIB_FILE` | The name of the specific file in that folder to use | | No | +| `DEVSIM` | Device should be simulated 1 - yes, 0 - no. (Provided by IBEX backend and can be set in the GUI) | 0 | No (Not in config.)| +| `IFDEVSIM` | Set to ' ' if device is being simulated; otherwise '#' (Provided by IBEX backend) | | No (Not in config.)| +| `IFNOTDEVSIM` | Set to '#' if device is being simulated; otherwise ' ' (Provided by IBEX backend) | | No (Not in config.)| +| `RECSIM` | IOC should simulate at the record level 1 - yes, 0 - no. (Provided by IBEX backend and can be set in the GUI) | 0 | No (Not in config.)| +| `IFRECSIM` | Set to ' ' if device is being record simulated; otherwise '#' (Provided by IBEX backend) | | No (Not in config.)| +| `IFNOTRECSIM` | Set to '#' if device is being record simulated; otherwise ' ' (Provided by IBEX backend) | | No (Not in config.)| +| `DISABLE` | Should communications be disabled 1 - yes, 0 - no. (Provided by IBEX backend and can be set in the GUI) | 0 | No (Not in config.)| +| `LVDCOM_HOST` | The host of the LVDcom vi (see [Creating IOC wrapper VI](../creation/Creating-IOC-wrapper-VI) for more). | "" | No | +| `LVDCOM_OPTIONS` | LVDcom options for starting the vi (see [Creating IOC wrapper VI](../creation/Creating-IOC-wrapper-VI) for more). | 6 | No | +| `LVDCOM_USER` | LVDcom user; not to be added to `config.xml`. It can be set only through the local globals.txt (see [Creating IOC wrapper VI](../creation/Creating-IOC-wrapper-VI) for more). | "" | No | +| `LVDCOM_PASS` | LVDcom passphrase; not to be added to `config.xml`. It can be set only through the local globals.txt (see [Creating IOC wrapper VI](../creation/Creating-IOC-wrapper-VI) for more). | "" | No | + +## Specific Macros that we might need to set + +There are some macros that need to be set which have specific names, and do not fit into the convention which it might be good to know about easily for setting up an IBEX system. Or which might highlight things that should be changed to conform. The macros for SDTEST are not included as there are many of them to consider. + +| IOC | Macro | Description | Suggested Values | +| --- | --- | --- | --- | +| `global` | `SIMULATE` | This can be used to check whether or not you are a simulated instrument | `1` to simulate, `0` for a live system| +| `ISISDAE01` | `DAEDCOM` | This is whether ISISDAE uses DCOM to talk to isisicp, or loads ISISICP internally. | `1` is normal and recommended on instrument, `0` is if you do not want to run isisicp e.g. dcom problems on MUONFE | +| `ISISDAE01` | `DAEHOST` | This is the IP address of the DAE host | `localhost` | +| `GALIL_0n` | `GALILADDR` | This is the IP address of the Galil | `None` when simulating | +| `HVCAEN_0n` | `HVCAENIPn` | This is the IP address for the CAEN | | +| `FINS_0n` | `PLCIP` | The IP address for the PLC | | +| `TPG300_0n` | `TTY` | The COM port for the TPG 300 (should be changed to conform) | | +| `GALIL_0n` | `MTRCTRL` | The motor controller number, e.g. it's 08 in :MOT:MTR0805 | | +| `CONEXAGP_0n` | `MTRCTRL` | The motor controller number, e.g. it's 08 in :MOT:MTR0805 | | +| `SMC100_0n` | `MTRCTRL` | The motor controller number, e.g. it's 08 in MOT:MTR0805 | | +| `PIMOT_0n` | `MTRCTRL` | The motor controller number, e.g. it's 08 in MOT:MTR0805 | | +| `ECLAB_0n` | `IPADDR` | IP Address of the device | | +| `LKSH336_0n` | `IPADDR` | IP Address of the device | | + + diff --git a/doc/iocs/conventions/Manuals.md b/doc/iocs/conventions/Manuals.md new file mode 100644 index 000000000..f45378a2a --- /dev/null +++ b/doc/iocs/conventions/Manuals.md @@ -0,0 +1,3 @@ +# Manuals + +Manuals are stored on the network share, it is not always clear that they can be shared so they are just for our group. The directory name is `__`. diff --git a/doc/iocs/conventions/PV-Naming.md b/doc/iocs/conventions/PV-Naming.md new file mode 100644 index 000000000..6d84f8e65 --- /dev/null +++ b/doc/iocs/conventions/PV-Naming.md @@ -0,0 +1,174 @@ +# PV naming + +Suggestions for naming of PVs at ISIS. + +Basic idea is to describe function in the PV name, not hardware/technology - the PV name is the purpose of a channel and is abstracted from the underlying hardware; an individual IOC name can however reflect technology/hardware/implementation. Note: do not use the `:` character in an IOC name. + +All characters in names should be uppercase. We will use `:` as hierarchy separator, `_` to delimit distinct characters in device name + +Basic scheme format is `Domain:subdomain:technicalarea:device:subdevice:signal` + +PV names are restricted to alphanumerical, plus `_` and `:` so `[A-Z0-9_:]*` Items that might have multiple instances must not end with a number as this would be confused with a 01,02 etc suffix used to enumerate multiple instances. + +### PV Name length limit + +Several things can create PV names, which influence the PV limit. The limit of a PV name as far as channel access protocol is concerned is around the size of a UDP packet (so ~1500 chars). + +On the other hand, an epics record can only create PVs of a smaller size, increased from 40 to 60 in 3.14.12 - but that limit doesn't field names as well. + +Therefore, IOC PVs have a full record name limit of 60, but PVs on channel access servers can have a much longer name. + +### Private names + +Sometimes we have a need to define PVs which are only used internally by the IOC, and never by outside programs. These "private" names will use a leading underscore on the last logically private element of a hierarchy. + +For example: + +``` +# Usual temperature setpoint record +record(ao, "$(P)TEMP:SP") +{ + ... +} + +# Some calculation that does something to the setpoint before it is sent to the device +record(calc, "$(P)TEMP:SP:_CALC") +{ + field(CALC, "...") + ... +} + +# A record which actually writes to the device (after the calculation has been performed) +record(ao, "$(P)TEMP:SP:_RAW") +{ + ... +} +``` + +### Naming convention + +Having followed the [IOC-Naming](IOC-Naming) the PV will be: + + PR:INSTXXCR:DEVICEXX_NN:PARAMETER + + - PR (max 2): Instrument prefix TE - test, IN - Instrument + - INST__CR (max 8): Instrument name max 8 characters. If the instrument name is longer than 8 characters then truncate to 6 characters and use the 2 characters CR16 hash of the instrument name e.g. `IRIS_SETUP => IRIS_SE22` + - DEVICEXX_99 (max 11): Name of the device with the device index after it. Device index is a two digit number + - PARAMETER (max 36): Parameter in the IOC e.g. a temperature readback or set point + + +## IOCs with multiple devices + +For multiple devices, use a 2-letter prefix followed by an unpadded number (e.g. CH1). Note that this rule is not strict; there are a number of IOCs that don't follow it (e.g. Eurotherms: A01) + +## Signal Qualifiers +These are added after a signal to indicate it is a setpoint etc. So for the TEMP signal + +- `...:HEATER:TEMP`: Current temperature +- `...:HEATER:TEMP:SP`: Temperature set point (requested value) – this is the value that was input in software and sent to the equipment. Write to this PV to change a setpoint +- `...:HEATER:TEMP:SP:RBV` (read-only): This is the setpoint “readback” from hardware, which may differ from SP sent above if e.g. the hardware was unable to exactly match the requested value. Also if the SP was changed by some other mechanism (e.g. manually on the hardware) `:SP` would not reflect this, but `:SP:RBV` would + +Note: RBV suffix can be used more generally e.g. For P,I,D values you could have `...:P` and `...:P:RBV` + +Note: For comparison, the SNS are using [camel case](http://en.wikipedia.org/wiki/CamelCase) i.e. `Temp`, `TempSet` and `TempRB` + +## Macros + +All records should start `$(P)` for prefix which can be substituted at load time. Diamond have `$(P)$(Q)` for added flexibility and we may want to use this too + +## Top Level domain + +Some suggested top level domains – we may not want to use them all + +| Prefix | Function | +| --- | --- | +| AC | ISIS Accelerator/synchrotron related parameter | +| CS | Control system specific top level variables e.g. global IOC related variables | +| TG | ISIS Target related parameter | +| IN | Instrument related parameter | +| HA | Experimental hall/building related parameter (e.g. cabin temperature) | +| TE | Testing domain, used by local EPICS developers | +| ME | Movable equipment, equipment not tied to a specific beamline | +| BL | Beamline – used if multiple instruments are sharing a common set of equipment, such as on the muon beamlines pre kicker. | + +## AC Domain +- `TG:TS1:MOD:H2:TEMP`: TS1 hydrogen moderator temperature +- `TG:TS1:MOD:H2:TC01:TEMP`: Alternative to above as there may be more than one temperature value! Using TC for “thermocouple” | + +## TE Domain +Mostly free format - have developers fedid as second part, then whatever required. In many cases would want to suffix with above scheme to make final integration easier e.g. + +``` +TE:FAA59:TG:TS1:MOD:H2:TEMP +``` + + +## BL Domain +This needs to be followed by information about which beamline. For neutron instruments could use the MCR shutter port identifier (N1, N2, N3 etc.) – need to find out what they call the Muon beamline. + +## HA Domain +Should be followed by the hall identifier. We could use `HA:TS1:*` etc, or be more general for possible use with any building e.g. `HA:R55:* ` + +## IN Domain +Sub domain is full instrument name e.g GEM. If we wish to distinguish the instrument “front end” from the rest, could use `*:FE` sub-domain + +- `IN:GEM:*`: Prefix for all variables related to GEM instrument +- `IN:GEM:SB:*`: Where to record short SECI block (short/friendly) names if we use them +- `IN:GEM:MOT:*`: Motion control equipment - see below +- `IN:GEM:VAC:*`: Vacuum equipment +- `IN:GEM:CS:*`: Variables related to instrument control system software - see below +- `IN:GEM:PS:*`: Power supply +- `IN:GEM:CHOP:FERMI:*` +- `IN:GEM:SE:*`: Sample environment not covered elsewhere +- `IN:GEM:DET:*`: Detector related variables +- `IN:GEM:FE:*`: Instrument “fro``nt end” equipment +- `IN:GEM:DAE:*`: Data acquisition electronics related PVs + +## The IN:{INST}:CS sub domain + +- `IN:{INST}:CS:IOC:*`: Variables describing running IOCs provided by the IOC themselves +- `IN:{INST}:CS:IOC:{IOCNAME}:AS:*`: Autosave PVs for IOC {IOCNAME} +- `IN:{INST}:CS:IOC:{IOCNAME}:DEVIOS:*`: DevIOStats PVs for IOC {IOCNAME} +- `IN:{INST}:CS:IOC:{IOCNAME}:MOT:*`: Motion specific PVs e.g. `allstop` from `motorUtils` +- `IN:{INST}:CS:IOC:{IOCNAME}:PS:*`: ProcServCtrl PVs for IOC {IOCNAME} +- `IN:{INST}:CS:GATEWAY:EXTERNAL:*`: Gateway special PVs for the external gateway +- `IN:{INST}:CS:GATEWAY:BLOCKSERVER:*`: Gateway special PVs for the blockserver gateway +- `IN:{INST}:CS:SCAN:ACTIVE`: where to put scan related variables? +- `IN:{INST}:CS:PS:{IOCNAME}:*`: variables created by PROCSERVCTRL IOC for starting/stopping IOCs via procServ + +## The IN:{INST}:MOT:* sub domain + +- `IN:{INST}:MOT:MTRccmm`: Epics motor records for motor on controller number cc, motor number mm. These numbers are zero padded to two digits and star from 1 e.g. MTR0101 is the first motor on the first controller +- `IN:{INST}:MOT:JAWSmm`: First set of jaws e.g. JAWS01 +- `IN:{INST}:MOT:DMCcc`: Galil specific controller variables for controller cc +- `IN:{INST}:MOT:STOPALL`: stop all motion + +## Standard signal names + +If a value can fluctuate, these refer to the current measured value of a quantity and the suffixes SP and RBV are used to indicated the desired value software requested (setpoint) and the desired value being used in the hardware (RBV) + +| Signal Name | Meaning | Valid Units | +|---|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|---| +| POS | Position | M, mm, cm | +| STAT | Status, State | Open, Closed, On, Off, Ok, Error | +| CURR | Current | A, mA, uA | +| VOLT | Voltage | kV, V | +| CMD | Device command e.g. write to this to perform an action, such as start/stop a run | | +| SEL | Select mode or position | | +| TEMP | Temperature | | +| COUNT | Counter value, neutron counts | | +| COUNTD | Counter value as a distribution, i.e. divided by bin width - so neutron counts per microsecond for example | | +| P, I, D | P, I, D values on e.g. Eurotherm | | +| TOF | Time of flight axis for a spectrum | | +| TIME | An absolute timestamp , preferably in ISO8601 format | | +| FIELD | Magnetic field | | +| PRESSURE | Pressure | | +| FREQ | Frequency of a chopper | Hz | +| PHAS | Phase of a chopper | us | +| PHAS_ERR | Error in the phase | us | +| ERROR | Device error output | Integer or String values | +| COMP | A component for a given composition gradient. Common on high performance liquid chromatography devices. | | +| FLOWRATE | The flow rate of a pump | (SI prefix) + L | +| RAMPRATE | The ramp rate of a variable | (SI unit of variable)/ (s or min) | +| START | For a device with a singular operation (Ramp, Pump etc.,) but a complex initialisation this can be used to start the operation (using a sequenced record etc.,) | | +| STOP | For a device with a singular operation (Ramp, Pump etc.,) but a complex shutdown this can be used to stop the operation (using a sequenced record etc.,) | | diff --git a/doc/iocs/conventions/PV-Units-&-Standards.md b/doc/iocs/conventions/PV-Units-&-Standards.md new file mode 100644 index 000000000..69bdfee8b --- /dev/null +++ b/doc/iocs/conventions/PV-Units-&-Standards.md @@ -0,0 +1,99 @@ +# PV units + +## DBChecker + +The DBChecker Python script is a helper file that checks a number of things are true about the PVs used within the project and that the correct syntax is used in db files. + +### Usage + +From an EPICS terminal in `C:\Instrument\Apps\EPICS\ISIS\DbChecker\master`, run: +``` +%python3% check_db_file.py -d -r +``` + +### Capabilities + +Current pv error checks are: + +- PVs that are labelled as interesting and have type longin, longout, ai or ao must contain a unit field +- Description fields must contain less than 41 characters +- All units must conform to unit standards (see below) +- PVs that are labelled as interesting must have description fields +- The names of PVs that are labelled as interesting must be capitalised and contain only `A-Z 0-9 _ :` +- `calc` records that are marked as interesting must have their access security group (`ASG`) set to `READONLY`. This is because if you were to set it directly, the record would not execute its calculation, which is not what we want. + +Current pv warning are: + +- PVs may not have multiple unit fields +- PVs that are labelled as interesting and have type longin, longout, ai or ao may not have blank fields + +Current syntax errors are: + +- Colons not used as main separator +- Names use characters that are not alphanumeric, an underscore or colon +- Does not adhere to :SP and SP:RBV format +- If DUMMYPV and DUMMYPV:SP exists without DUMMYPV:SP:RBV (at least as an alias). +- If DUMMYPV:SP exists on its own but does not have a SP:RBV alias for DUMMYPV +- Underscores must not be used in place of : i.e. DUMMYPV_SP + +Current syntax warnings are: + +- Names not in uppercase + + + +The checker is run at the end of a build on Jenkins and unit tests are failed if any of the error checks fail. Failed warnings will be noted and displayed in the test report but will not result in an unstable build. Syntax errors are currently treated as warnings as some files currently contain large numbers of them. + +{#pv_units_standards} +## Unit Standards + +The unit standards apply to both DB files and [calibration files](/system_components/configurations/Calibration-Files). + +If the unit has a standard alphanumeric unit symbol that has been used. In the case where the usual symbol is not alphanumeric e.g. degree (°), angstrom (Å), the unit is written in full, lower case and singular. The unit may be shortened because CSS will only display 8 characters worth of unit so `degree/s^2` is to long. + +Standard prefixes, [T|G|M|k|m|u|n|p|f], are accepted before all units. + +Units can be constructed from a number of 'base' units using a space, forward-slash and caret for multiplication, division and powers respectively. For example a unit for work done could be "N m", a unit for velocity could be "m/s" and a unit for area "m^2". This is the same standard as used in [udunits](http://linux.die.net/man/3/udunits). + +For PVs where the units are contestable, for example NUMSPECTRA, a blank units field is acceptable. This will give a warning, but not a failure, when a test is run and so can be discussed at a later date. + +The units within the `support/optics/` path are not checked as they contained a number of ambiguities and are rarely used. + +### Supported Units + +The project currently contains the following base units: + +* A +* angstrom +* bar +* bit +* byte +* C +* cm +* count +* degree or deg for short +* eV +* event +* frame +* hour +* Hz +* inch +* interrupt +* K +* L +* m +* min +* minute +* ohm +* Oersted +* % +* Pa +* photon +* pixel +* radian +* rpm +* s +* torr +* step or stp for short +* T +* V diff --git a/doc/iocs/creation/Add-ioc-to-epics-hardware-list.md b/doc/iocs/creation/Add-ioc-to-epics-hardware-list.md new file mode 100644 index 000000000..7629329b9 --- /dev/null +++ b/doc/iocs/creation/Add-ioc-to-epics-hardware-list.md @@ -0,0 +1,16 @@ +# Add IOC to EPICS hardware support list + +It would be nice to add all IOCs to the EPICS list so that other facilities do not have to rewrite a driver. Most of our IOCs are in public git hub repositories so there should be no problems doing this. We have discussed with the relevant team and we need to add licences before we can publish the link see but until then follow the procedure: + + +1. Visit epics page https://epics.anl.gov/modules/manufacturer.php +1. Click `To request a new entry in this table for your Hardware Support module, use this form.` +1. Add the following + - Bus: the bus e.g. `RS-232 (Streams)` + - Manufacturer: the manufacturer + - Contact Name: `ISIS Computing Group` + - Contact Email: `ISISEPICS@stfc.ac.uk` + - Link URL: blank until licences are sorted out + - Link Text: blank until licences are sorted out + +NB Make sure that the support module [has a licence files](https://github.com/ISISComputingGroup/IBEX_device_generator/blob/master/templates/support/LICENCE) \ No newline at end of file diff --git a/doc/iocs/creation/Creating-IOC-wrapper-VI.md b/doc/iocs/creation/Creating-IOC-wrapper-VI.md new file mode 100644 index 000000000..a9cec1669 --- /dev/null +++ b/doc/iocs/creation/Creating-IOC-wrapper-VI.md @@ -0,0 +1,295 @@ +# LvDCOM IOCs (LabVIEW wrapper IOCs) + +```{warning} +These instructions are **NOT accurate** and will need careful editing before they can be used to create a fully functioning IOC and support module ⚠️ + +In their current form, it is recommended that they be followed with extreme caution and with expert advice on hand 😟 +``` + +These are instructions for creating an IOC wrapper for a VI using lvDCOM. It assumes you want to create a "support" type module for lvdcom in ISIS. The Mercury ITC is an example where the below was followed and then an IOC was created using this as a support module. + +## 1. Create ISIS Support Module + +I plan to make these into a template (so you will just need to run makeBaseApp.pl and not do the edits), but for now this is the manual way. + +Please take note of the [IOC naming convention](../conventions/IOC-Naming) before proceeding. + +1. Create a public repository to work in called EPICS-\. +1. In the `EPICS\ISIS` directory create a directory called \ +1. Add the repository as a submodule and clone it into this directory (i.e. with the command `git submodule add https://github.com/ISISComputingGroup/EPICS-.git master`) +1. Create a basic standard file and directory structure either by copying a similar existing module (and then executing `make clean uninstall` to remove unnecessary files) or by running `makeSupport.pl`. Check the results either way. +1. Copy a standard makefile into this directory + +Create the IOC in EPICS\IOC\master with (from an EPICS terminal) +1. In the `EPICS\IOC\master` directory create a directory called +1. In the new directory `EPICS\IOC\master\`, run the following two commands. + +``` + makeBaseApp.pl –t ioc + makeBaseApp.pl –i –t ioc +``` + +Edit `mynameApp/src/build.mak` add dbd file + +``` +$(APPNAME)_DBD += lvDCOM.dbd +``` + +and libraries + +``` +$(APPNAME)_LIBS += lvDCOM +$(APPNAME)_LIBS += asyn +``` + +and system library + +``` +$(APPNAME)_SYS_LIBS_WIN32 += msxml2 +``` + +Add the IOC to the makefile in `EPICS\ioc\master\Makefile` e.g. + +``` +## modules not to build if no Windows ATL present (depends on Visual Studio compiler) +ifneq ($(HAVE_ATL),YES) +DIRS_NOTBUILD += ISISDAE MERCURY_ITC STPS350 AG53220A STSR400 DELFTSHEAR DELFTDCMAG DELFTARDUSTEP LVTEST SCIMAG3D HIFIMAG + +endif +``` + +## 2. Create the xml configuration file + +This is a summary of [[more general LvDCOM instructions|http://epics.isis.stfc.ac.uk/doxygen/lvDCOM]]. + +1. Open the VI in lab view. +1. Export strings from the LabVIEW panel (In different version of LabVIEW this is different) + 1. 2010: Go to Tools menu, Advanced, Export Strings... and uncheck the wizard option for "block diagram strings" and save the export results to a text file e.g. controls.txt (Note: you may need write access to the VI itself to do this, so might have to make a local copy of the VI first) + 1. 2014: + 1. Menu -> Tools -> Advanced -> Export Strings... + 1. Then save file to (\/protocol/controls.txt) + 1. Yes to "Export captions for controls without captions?" + 1. No to "Export block diagram strings?" +1. Generate a corrected xml file. In an epics terminal run in the protocol directory: + ``` + C:\Instrument\Apps\EPICS\ISIS\lvDCOM\master\lvDCOMApp\src\fix_xml.cmd "controls.txt" "controls.xml" + ``` +1. Generate lvdcom file: + ``` + xsltproc C:\Instrument\Apps\EPICS\ISIS\lvDCOM\master\lvDCOMApp\src\lvstrings2input.xsl "controls.xml" > lv_controls.xml + ``` +1. Edit lv_controls.xml (see below for example) + 1. Check the path is correct for the external interface, it should be:`` + 1. Path in vi element needs path to be vi in the llb containing the vi e.g. `C:/LabVIEW Modules/Drivers/Oxford Instruments/Mercury/Mercury - Temperature.llb/` + 1. Look at TODO's in this file + 1. Remove unneeded controls or states of those controls (e.g. write for read only values) + 1. If the value is controlled by an event the `extint` value of set controls should be set to `"true"`. This will make it process the value. If one of the value is true you might as well set them all to be true the only advantage is if the are all false it does not need to load the external interface. +1. Add protocol file to `ISIS/App/protocol/Makefile` as + + ``` + DATA += lv_controls.xml + ``` + +## 3. Generate the DB File + +1. generate db file from an epics terminal in protocol + ``` + xsltproc C:\Instrument\Apps\EPICS\ISIS\lvDCOM\master\lvDCOMApp\src\lvinput2db.xsl lv_controls.xml > ..\Db\controls.db + ``` +1. Add db file to `App\Db\Makefile` + + ``` + DB += controls.db + ``` +1. Edit the DB file. I found I wanted many `_RBV` to be `:RBV` and to get rid of some of the records (see below for example) + +## 4. Edit to st.cmd + +1. Add `lvDCOMConfigure("lvfp", "frontpanel", "${TOP}/data/lv_controls.xml", "$(LVDCOM_HOST=)", $(LVDCOM_OPTIONS=1), "$(LVDCOM_PROGID=)", "$(LVDCOM_USER=)", "$(LVDCOM_PASS=)")` before loading common records and _after_ IOC initialization (to enable common macros to be used in `lv_controls.xml`). (see [lvDCOMConfigure documentation](http://epics.isis.stfc.ac.uk/doxygen/lvDCOM/lvDCOMDriver_8cpp.html#a90fdd61917374a2fed5dd1e2ba6da62b)). + +1. Load the DB file for the IOC using the `dbLoadRecords` command. + +## 5. Edit `config.xml` + +1. Add these macros to `config.xml` to allow them to be configured in the GUI: + + ``` + + + + ``` + * Don't add username and password, users can set these in globals.txt + + + +## 6. Run the IOC + +Run the IOC as normal. The IOC should start with no errors and typing "dbl" will list the PVs. Note: unless the VI was already open it will not be visible. If it is not visible, stop the IOC, load the VI and restart the IOC + +## 7. Finish the Workflow + +Now return to the IOC workflow to apply [finishing touches](IOC-Finishing-Touches) like units, PVs of interest and macros. +Once the ISIS IOC works you should probably now create an IOC linked to this one in ioc follow a similar pattern to a support modules. Remember that if you do this add your new ioc to the Makefile `IOCDIRS` and it does not build if there is no ATL so add it to this list too, i.e. edit `EPICS\ioc\master\Makefile` add to the line: +``` + ifneq ($(HAVE_ATL),YES) + DIRS_NOTBUILD += ISISDAE MERCURY_ITC STPS350 AG53220A STSR400 DELFTSHEAR DELFTDCMAG DELFTARDUSTEP LVTEST + endif +``` + + +## Example XML control definitions + +A numeric indicator: + +``` + + + +``` + +A numeric control: + +``` + + + + +``` + +A boolean indicator: + +``` + + + +``` + +A string control: + +``` + + + + +``` + +An array indicator: + +``` + + + +``` + +Push a button when a value is set: + +``` + + + + +``` + +## Example records + +A numeric indicator: +``` +record(ai, "$(P)NUM_IND") +{ + field(DTYP, "asynFloat64") + field(INP, "@asyn(ex1,0,0)param_name_from_xml") + field(PREC, "3") + field(SCAN, ".1 second") +} +``` + +A numeric control: + +``` +record(ao, "$(P)NUM_CON") +{ + field(DTYP, "asynFloat64") + field(OUT, "@asyn(ex1,0,0)param_name_from_xml") + field(PREC, "3") +} +``` + +A boolean indicator: + +``` +record(bi, "$(P)BOOL_IND") +{ + field(DTYP, "asynInt32") + field(INP, "@asyn(ex1,0,0)param_name_from_xml") + field(ZNAM, "FALSE") + field(ONAM, "TRUE") + field(SCAN, ".1 second") +} +``` + +A string control (two records - one for readback and one for writing): + +``` +record(stringin, "$(P)STR") +{ + field(DTYP, "asynOctetRead") + field(INP, "@asyn(ex1,0,0)param_name_from_xml") + field(SCAN, ".1 second") +} + +record(stringout, "$(P)STRW") +{ + field(DTYP, "asynOctetWrite") + field(OUT, "@asyn(ex1,0,0)param_name_from_xml") +} +``` + +A numeric array indicator: + +``` +record(waveform, "$(P)ARRAY_IND") +{ + field(DTYP, "asynFloat64ArrayIn") + field(INP, "@asyn(ex1,0,0)param_name_from_xml") + field(PREC, "3") + field(SCAN, ".1 second") + field(NELM, "10") + field(FTVL, "DOUBLE") +} +``` + +## Troubleshooting + +If you are getting file not found errors in your IOC when writing to PVs this will be because the path to the external interface in the generated xml is incorrect. + +You can also use the **dbior** command from the IOC prompt to print out the lvDCOM settings currently in use + +## Fixing LabVIEW 2010's Registry settings to allow remote DCOM access + +It appears that when LabVIEW 2010 is installed it does not register itself correctly in the Windows Registry. This means that it ignores any DCOM settings applied to it via dcomcnfg (see note below); as a result, it uses the general default settings instead. + +Note: dcomcnfg is the DCOM configuration tool which can be launched by typing "dcomcnfg" at the command prompt. + +Without the correction to the Registry, connecting to the machine from a remote lvDCOM IOC starts a second instance of LabVIEW rather than connecting to the existing instance - this is NOT what is wanted. + +- Find the CLSID of LabVIEW - you can use dcomcnfg for this or search the registry. +- Under HKEY_LOCAL_MACHINE\SOFTWARE\Classes\AppID, find the LabVIEW.exe and copy the AppId +- Under HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{LabVIEW GUID from step 1}, add a string called !AppId and set the value to the AppId from above. +- Open dcomcnfg as Administrator, locate LabVIEW and click properties. Under Identity, set it to launching user then click OK. + +## Connecting an IOC to a VI on a different PC + +To connect to a lvDCOM IOC to a VI on a different PC is very straightforward. + +Edit the st.cmd of the IOC to change the lvDCOMConfigure line from something like this: + +``` +lvDCOMConfigure("ex1", "example", "$(TOP)/lvDCOMApp/src/examples/example_lvinput.xml", "", 6) +``` + +To something like this: + +``` +lvDCOMConfigure("ex1", "example", "$(TOP)/lvDCOMApp/src/examples/example_lvinput.xml", "ndxtestmjc", 6, "", "account_name", "password") +``` + +Of course, the account_name and password should be changed to match your system. diff --git a/doc/iocs/creation/Creating-New-Submodules-in-EPICS.md b/doc/iocs/creation/Creating-New-Submodules-in-EPICS.md new file mode 100644 index 000000000..9af66eed6 --- /dev/null +++ b/doc/iocs/creation/Creating-New-Submodules-in-EPICS.md @@ -0,0 +1,18 @@ +# Creating new submodules in EPICS + +You will want to make sure you have run [IOC Generator](IOC-Generator) first, if this is for a new device's IOC. + +This script will create the IOC, OPI, support (including lewis emulator and system tests), and Makefiles for these various folders as well. + +## Write the IOC, OPI, emulator, system tests, etc. + +Consult the device's manual, as well as any documentation the instrument scientists may have on their needs/wants. More info can be found here: +* [Creating an ISIS StreamDevice IOC](Creating-an-ISIS-StreamDevice-IOC) +* [OPI Creation](/client/opis/OPI-Creation) +* [Emulating Devices](../testing/Emulating-Devices) +* [Test naming](/client/testing/Test-naming) +* [IOC Test Framework Documentation](https://github.com/ISISComputingGroup/EPICS-IOC_Test_Framework/blob/master/README.md) + +## Final Push to EPICS top + +Once everything is written, and final commits and pull requests have been created in the `EPICS-` and `EPICS-ioc` repositories, navigate to the top of EPICS, from the branch created by `device_generator.py` (Ticket_XXXX_Add_IOC_) add, commit, and push the newly added submodule \ No newline at end of file diff --git a/doc/iocs/creation/Creating-an-ISIS-StreamDevice-IOC.md b/doc/iocs/creation/Creating-an-ISIS-StreamDevice-IOC.md new file mode 100644 index 000000000..a36f1ee4e --- /dev/null +++ b/doc/iocs/creation/Creating-an-ISIS-StreamDevice-IOC.md @@ -0,0 +1,263 @@ +# Creating an ISIS StreamDevice IOC + +## Before you begin + +Is the a support module already available? Check https://epics.anl.gov/modules/manufacturer.php. If it's not listed there, email tech talk https://epics.anl.gov/tech-talk/. + +## First step + +The easiest way to create a StreamDevice is to use the script [here](https://github.com/ISISComputingGroup/IBEX_device_generator) but if for some reason you want to create it manually the instructions are as follows: + +```{warning} +The IOC creating script is improving but still under testing. Git operations should be happening correctly. +Tickets related to the issues: +- https://github.com/ISISComputingGroup/IBEX/issues/3431 +- https://github.com/ISISComputingGroup/IBEX/issues/3588 +- https://github.com/ISISComputingGroup/IBEX/issues/4659 +- https://github.com/ISISComputingGroup/IBEX/issues/8249 +``` + +## Create a StreamDevice support module + +Note: The support module is put in the `EPICS\support` directory, but the actual IOC(s) are put in the `EPICS\ioc\master` directory + +First, set up the Support Module (Using the Hameg 8123 as an example.): + +Make the main directory: + +``` +cd ...EPICS\support +mkdir Hameg_8123 +cd Hameg_8123 +``` + +Get an admin to [create a git repository](/processes/git_and_github/Adding-new-modules-via-Git). + +Create a stream support module: + +``` +cd ...EPICS\support\Hameg_8123\master +makeSupport.pl -t streamSCPI Hameg_8123 +``` + +Edit the protocol file in `Hameg_8123Sup`, so it reads like: + +``` +Terminator = '\r\n'; +ReplyTimeout = 2000; + +getIDN { + out "*IDN?"; + #Read no more than 39 chars (EPICS limit) + in "%/(.{0,39})/"; + ExtraInput = Ignore; +} + +getTriggerLevel { + out "LV\$1?"; + in "%s"; + ExtraInput = Ignore; +} + +setTriggerLevel { + out "LV\$1%s?"; +} + +resetCounts { + out "RES"; +} +``` + + +Note: I have only included a very small section of the command set for this device for brevity. + + + +Now add the directory name to the support make file (`C:\Instrument\Apps\EPICS\support\Makefile`), ie to DIRS at the top. Also add dependencies if needed. + +Next the db file needs to created. The db file should now be stored with the proto file in `hameg8123Sup` to aid portability. For the Hameg the part of the db file looks like this: +``` +record(ai, "$(P)CHAN_A:TRIG_LVL") +{ + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@Hameg_8123.proto getTriggerLevel(A) $(PORT)") + field(PREC, "3") + field(EGU, "V") +} + +record(ao, "$(P)CHAN_A:TRIG_LVL:SP") +{ + field(DTYP, "stream") + field(OUT, "@Hameg_8123.proto setTriggerLevel(A) $(PORT)") + field(PREC, "3") + field(EGU, "V") +} +``` + +`Hameg_8123.proto` is the name of the protocol file for the Hameg created here. + +Note that the db file should conform to the naming standards detailed in [PV-Naming](../conventions/PV-Naming), and that ANY value which might be read and set as a block must have a `:SP` as well as a non post-fixed name entry. + +Also note that ALL streamdevice PVs must have the field `DTYP` set to `"stream"`, otherwise they will not correctly communicate. + + +Modify the Makefile in the same directory as the protocol and db files to have lines like: + +``` +DB += .db +DATA += +``` + +If the Makefile has a line that reads `DATA += $(patsubst ../%, %, $(wildcard ../*.proto))`, delete it. + + +## Creating the IOC + +All IOCs used at ISIS reside in the `EPICS\ioc\master` directory. + +For this example we are using the Hameg 8123. + +Please take note of the [IOC naming conventions](../conventions/IOC-Naming). The same name (with the same casing) should be used both for the top-level IOC directory under `EPICS\ioc\master` and in the `makeBaseApp` command, as in the example below. + +First create the necessary directory structure and IOC boilerplate, starting from the `EPICS\ioc` directory (we're going to create at least two IOCs): +``` +mkdir HAMEG8123 +cd HAMEG8123 +makeBaseApp.pl -t ioc HAMEG8123-IOC-01 +makeBaseApp.pl -t ioc HAMEG8123-IOC-02 +makeBaseApp.pl -i -t ioc HAMEG8123-IOC-01 + +makeBaseApp.pl -i -t ioc HAMEG8123-IOC-02 + +``` + +The next step is to adjust the Makefile in `HAMEG8123-IOC-02App\src:` +``` +cd HAMEG8123-IOC-02App\src +del build.mak +notepad Makefile +``` + +In notepad, adjust the Makefile as outlined in the in-line comments, namely change the line: +`include $(TOP)/HAMEG8123-IOC-02App/src/build.mak` to `include $(TOP)/HAMEG8123-IOC-01App/src/build.mak`. +Don't forget to save it! + +Now it is time to edit the `build.mak` file in `HAMEG8123-IOC-01App\src` - this is the master build file for all the HAMEG8123 IOCs. In notepad (or similar) add any require DBD files and LIBS to the respective listings. For example, for the Hameg it is necessary to add the DBDs for stream, asyn and the communication protocols: +``` +... +## ISIS standard dbd ## +$(APPNAME)_DBD += devSequencer.dbd +$(APPNAME)_DBD += icpconfig.dbd +$(APPNAME)_DBD += pvdump.dbd +$(APPNAME)_DBD += asSupport.dbd +$(APPNAME)_DBD += devIocStats.dbd +$(APPNAME)_DBD += caPutLog.dbd +$(APPNAME)_DBD += utilities.dbd +## add other dbd here ## +$(APPNAME)_DBD += stream.dbd +$(APPNAME)_DBD += asyn.dbd +$(APPNAME)_DBD += drvAsynSerialPort.dbd +$(APPNAME)_DBD += drvAsynIPPort.dbd +... +``` + +Likewise, the LIBs need to be added too: +``` +## ISIS standard libraries ## +$(APPNAME)_LIBS += seq pv +$(APPNAME)_LIBS += devIocStats +$(APPNAME)_LIBS += pvdump easySQLite sqlite +$(APPNAME)_LIBS += caPutLog +$(APPNAME)_LIBS += icpconfig +$(APPNAME)_LIBS += autosave +$(APPNAME)_LIBS += utilities +## Add other libraries here ## +$(APPNAME)_LIBS += stream +$(APPNAME)_LIBS += pcre +$(APPNAME)_LIBS += asyn +$(APPNAME)_LIBS += calc +``` + +The final step is to rationalise the st.cmd files for each IOC. There will be a default `st.cmd` for each IOC which will call a common file in the 01 directory. The top files for IOC-YY should look like the following (XXXX is the name of the IOC): + +``` +#!../../bin/windows-x64/XXXX-IOC-YY + +## You may have to change XXXX to something else +## everywhere it appears in this file + +< envPaths + +cd "${TOP}" + +## Register all support components +dbLoadDatabase "dbd/XXXX-IOC-YY.dbd" +XXXX_IOC_YY_registerRecordDeviceDriver pdbbase + +## calling common command file in ioc 01 boot dir +< ${TOP}/iocBoot/iocXXXX-IOC-01/st-common.cmd + +``` + +The a common file, `st-common.cmd` should look like (NOTE: the support files location is a macro defined in the `\configure\RELEASE` file): + +``` +epicsEnvSet "STREAM_PROTOCOL_PATH" "$(AMINT2L)/data" + +##ISIS## Run IOC initialisation +< $(IOCSTARTUP)/init.cmd + +## For recsim: +$(IFRECSIM) drvAsynSerialPortConfigure("L0", "$(PORT=NUL)", 0, 1, 0, 0) + +# For dev sim devices +$(IFDEVSIM) drvAsynIPPortConfigure("L0", "localhost:$(EMULATOR_PORT=57677)") + +## For real device use: +$(IFNOTDEVSIM) $(IFNOTRECSIM) drvAsynSerialPortConfigure("L0", "$(PORT=NO_PORT_MACRO)", 0, 0, 0, 0) +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "baud", "$(BAUD=9600)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "bits", "$(BITS=7)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "parity", "$(PARITY=even)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "stop", "$(STOP=1)") +## Hardware flow control off +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", 0, "clocal", "Y") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"crtscts","N") +## Software flow control off +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixon","N") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixoff","N") + +##ISIS## Load common DB records +< $(IOCSTARTUP)/dbload.cmd + +## Load our record instances +dbLoadRecords("[support_module_path]/db/devAMINT2L.db","P=$(MYPVPREFIX)$(IOCNAME):, PORT=L0, RECSIM=$(RECSIM=0), DISABLE=$(DISABLE=0)") + +##ISIS## Stuff that needs to be done after all records are loaded but before iocInit is called +< $(IOCSTARTUP)/preiocinit.cmd + +cd "${TOP}/iocBoot/${IOC}" +iocInit + +## Start any sequence programs +#seq sncxxx,"user=hgv27692Host" + +##ISIS## Stuff that needs to be done after iocInit is called e.g. sequence programs +< $(IOCSTARTUP)/postiocinit.cmd + +``` + +Now is a good time to add everything into Git. Once that is done it is time to build it and run it: + +``` +cd c:\Instrument\Apps\EPICS\ioc\master\HAMEG8123 +make +cd iocBoot\iocHAMEG8123-IOC-01 +runIOC.bat st.cmd +``` + +Hopefully, the IOC will start and the `dbl` command will list all the PVs. + +Now it builds add a reference to the IOC make file in `C:\Instrument\Apps\EPICS\ioc\master\Makefile` add the directory name to `IOCDIRS`. If this gets to long split with `IOCDIR +=` + +Before the IOC is complete you will need to finish all the [relevant workflow steps](/IOCs) diff --git a/doc/iocs/creation/Duplicating-IOCs.md b/doc/iocs/creation/Duplicating-IOCs.md new file mode 100644 index 000000000..5c87b030f --- /dev/null +++ b/doc/iocs/creation/Duplicating-IOCs.md @@ -0,0 +1,56 @@ +# Duplicating IOCs +## IOC Duplication script +There is a script in ibex utils to duplicate IOCs, more information can be found [here.](#ioc_copier_script) + +The manual instructions below can also be used. +Manual instructions
## Creating IOC 2 +If you need more than 1 IOC (i.e. you are making the second IOC) there is only a process to follow, rather than a script, as various IOCs can have their own nuances: so take this with a grain of salt. Let us refer to the IOC you are duplicating as `newIOC`, for clarity. + +Navigate to an IOC folder which has two or more IOCs: here we have two folders to focus on, `/iocBoot/` and `/-IOC-0App` (where `n` is the number of IOCs). Make sure this IOC has a `st-common.cmd` file (to make your life easier!). Let us refer to this as `refIOC`. + +Now, let's get cracking! + + +### Making IOC 2: `iocBoot/ioc-IOC-02>` +1. Create a new **`ioc-IOC-02`** folder in `/iocBoot/`. +2. Navigate to the **`config.xml`** file of `refIOC`. In `01` you will see macros being defined, in `02` you will see the `01` config file being referenced. Create a `config.xml` file in `ioc-IOC-02` and do the same. +3. Check that there is a **`st-common.cmd`** file in `ioc-IOC-01` + - if there **is**, note how `st.cmd` references this. Copy this file into `ioc-IOC-02` and refactor all instances of `01` to `02`. Then, where the file calls `st-common.cmd`, you will need to add a line that navigates to the `ioc-IOC-01` directory above it (there will be an example of this in `ioc-IOC-02`). + - if there **is no**t, note how the `st.cmd` and `st-common.cmd` are set up in `refIOC`. In `ioc-IOC-01`, transition the `st.cmd` contents to a +`st-common.cmd` file and refactor the `st.cmd` to reference this in a similar way. Now follow the step above! +4. Copy across the **`Makefile`** - this stays unchanged. +5. Copy across **`envPaths`**, changing `ioc-IOC-01` to `ioc-IOC-02` if it appears. +6. Copy across **`dllPath<...>`** and **`relPaths`** files. These also stay unchanged. + + +### Making IOC 2: `-IOC-02App` +This one may be slightly less straightforward. There may be nuances and additional things in this folder to deal with that aren't mentioned below - either try find another IOC with similar oddities or ask someone! + +1. Create a new **`-IOC-02App`** folder in `//` +2. Navigate to `-IOC-01App` and copy across **`Db`** to the `02App` folder. + - Empty the `O.` folders of all `.db` files. + - Delete all `.substitutions` files from top level. + - In `Db\O.windows-x64\Makefile`, refactor the line `DB += something.db ...` with `#DB += xxx.db` (e.g just comment it out) +4. Navigate to `-IOC-01App` and copy across **`src`** to the `02App` folder. + - Empty the contents of both `O.` folders. + - Delete the `build.mak` file + - Rename the `<...>Main.cpp` file with the correct IOC number, and rename the header in the file itself. + - In the `Makefile`, update `APPNAME` with the correct IOC number - but the `include ...` line needs to stay the same. +5. Check whether `-IOC-01App` has a **`protocol`** file + - If it does, just copy this across. As far as I can see, the Makefiles and folder contents seem to be the same. +
+ +### After duplication +After either using the script or creating duplicates manually be sure to make and test to IOC. + +### Making the IOC +1. **`make`** the `` folder +2. **`make iocstartups`** in EPICS top +3. Try to run your new IOC! + +### Testing the IOC +Navigate to `IOCTestframework` or the `ioc/` folder (wherever the IOC tests live). +Refactor `DEVICE_PREFIX` to `_02` and you will need to refactor any calls to `get_default_ioc_dir()` with additional parameter `iocnum` (or whatever the equivalent is in your test module, this should be pretty intuitive). + +****WARNING:**** If you will also be making more IOCs via the method below, you should be _very_ confident that your new `_02` behaves the same as `_01` before duplicating: any issues with ` \ No newline at end of file diff --git a/doc/iocs/creation/IOC-Finishing-Touches.md b/doc/iocs/creation/IOC-Finishing-Touches.md new file mode 100644 index 000000000..4fe544a61 --- /dev/null +++ b/doc/iocs/creation/IOC-Finishing-Touches.md @@ -0,0 +1,159 @@ +# IOC Finishing Touches + +This page documents the finishing touches to make to your IOC to make it function within the IBEX environment. + +{#ioc_finishing_touches_interesting_pvs} +## 1. Interesting PVs + +To have a PV appear in the interest list in IBEX Configurations add the following to the PV record: + + info(INTEREST, "") + +where the level is HIGH, MEDIUM, LOW. + +For records that are of no "interest" do not add an interest info field. For example, intermediate CALC records, SIM records etc. + +Any calc record which is interesting needs `field(ASG, "READONLY")` added so it cannot be set by accident. + +{#ioc_finishing_touches_archive_pvs} +## 2. Archive PVs + +To have a PV automatically archived add the following to the PV record + + info(archive, "VAL") + +This will archive the value of the `VAL` field once per second. The general form is + + info(archive, " ") + +Where + + * period (defaults to 1): + * when +ve - sign up to monitor the pv use the period field to determine a typical delay between samples in seconds + * when -ve - monitor the value the value is the deadband for the system + * field: the field to monitor on the record + +Find more information at [Logging from the archive](/system_components/Logging-from-the-archive). + +{#ioc_finishing_touches_alarm_pvs} +## 3. Alarm PVs + +If you want a PV to appear in the alarm view (and there should be at least one per IOC so that it can show disconnected) then add: + + info(alarm, "") + +{#ioc_finishing_touches_autosave_pvs} +## 4. Autosave PVs + +PVs can be autosaved so that they save their values and are reloaded when the IOC starts. The value is set before the record is initialised so is only processed using this value if `PINI` is Yes; although waveform records do this differently and will process the record. To do this mark the field with the following info line: + + info(autosaveFields, "VAL") + +The second argument is the field which is autosaved within the record. + +{#ioc_finishing_touches_disable_records} +## 5. Disable records + +It is very useful to be able to turn an IOC on and off by simply writing to a special PV, as it is a lot faster and more convenient than actually restarting the whole IOC. The instructions can be found [here](../testing/Disable-records) . + +{#ioc_finishing_touches_essential_fields} +## 6. PVs Have Essential Fields + +All PVs should have if appropriate: + +* Description (`DESC` field) +* Unit fields if representing a value (`EGU` field) which may be blank + * Units must be in ... +* Precision (`PREC`) for records with floating point numbers - is this set correctly for what a user/technician requires + +{#ioc_finishing_touches_setpoints} +## 7. Initialising Setpoints + +Setpoint pvs should have "undefined field" initialised by adding the following record to the PV: + + field(UDFS, "NO_ALARM") + +This means that they can have alarm sensitive borders but will not alarm if they have never been set. + +{#ioc_finishing_touches_dbchecker} +## 8. Compliance with DBChecker + +The build in Jenkins will fail if the rules of the [DBChecker](../conventions/PV-Units-&-Standards) script are not satisfied. You might as well check them beforehand to save yourself time later. See linked page for additional information & instructions. + +{#ioc_finishing_touches_macros} +## 9. Macros and Details + +Macros where possible should follow the [standard names](../conventions/Macro-Naming). If a macro can be set as part of the IOC (and can be reasonably set in the GUI) then a config file should be added to the run directory which contains a list of macros (i.e. `..\EPICS\ioc\master\\iocBoot\\config.xml`). Common macros should be included from `..\EPICS\ioc\common\`.The file is of the form: + +```xml + + + +Eurotherm temperature controller + + + + + + + + + + + + + +``` + +where +- `ioc_desc` is a short description of the IOC e.g Lakeshore 218 for LKSH218. This field is shown alongside the IOC name in the GUI when adding or editing IOCs. +- `ioc_details` is more details about the IOC, e.g. link to docs. +- `macro` describes a macro settable by a user. + - containing `name`, is the name of the macro; + - `pattern`, the regex for the macro's value; Useful regex for macros: + - `^-?[0-9]+\.?[0-9]*$`: float with sign + - `^[0-9]+\.?[0-9]*$`: float no sign + - `^[0-9]+$`: integer no sign + - `description`, a plain text description which is shown to the user. + - `defaultValue`, the default value of the macro, if it exists (this attribute is not required) + - `hasDefault`, if the macro has a default, either `"YES"`, `"NO"`, or `"UNKNOWN"`. (Note that `UNKNOWN` exists for legacy reasons, new IOCs should not use it) + +`config.xml` support include so if you have several iocs with the same set of macros you don't need to repeat the file contents. Example GALIL02 (see below) uses GALIL01's config: + +```xml + + + + +``` + +Either a full make of the server or running `make iocstartups` from the EPICS folder will then make the contents of these XML files available to the GUI (after restarting the instrument). + +**Tips** +* If you want a macro that restricts input to be a byte, so 0-255, then you can use ^$|^([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$ . You can change this so it allows any range of integers of you desire. [This website](https://regex101.com/) is a good resource for checking regex expressions. + +## 10. PV Limits + +If a limit on a set point is well defined (i.e., given by a device manual) then the fields `DRVH` "Drive High" and `DRVL` "Drive Low" should be used to constrain the PV set point. The behaviour of these fields is that if a limit is 10.0 and a user inputs 11.0, then the PV will constrain the input to 10.0 and process that value. Records that use limits should also be robustly tested to ensure the behave as expected. An example test: + +```python + @parameterized.expand([ + ("lt_low_limit", CURR_LOW_LIMIT-1, "low_limit", CURR_LOW_LIMIT), + ("gt_high_limit", CURR_HIGH_LIMIT+1, "high_limit", CURR_HIGH_LIMIT)]) + @skip_if_recsim("Behaviour cannot be simulated in Recsim") + def test_WHEN_voltage_setpoint_is_set_outside_max_limits_THEN_setpoint_within(self, case, case_value, limit, limit_value): + self.ca.set_pv_value("CURRENT:SP", case_value) + self.ca.assert_that_pv_is("CURRENT:SP", limit_value) +``` + +## 11. Directories added to Makefiles +Type +``` +make checkdirs +``` +at EPICS top level and make sure it completes ok + +## 12. Add IOC to EPICS hardware list + +Once the IOC is reviewed and tested with hardware, [add it to the EPICS hardware list](../creation/Add-ioc-to-epics-hardware-list) + diff --git a/doc/iocs/creation/IOC-Generator.md b/doc/iocs/creation/IOC-Generator.md new file mode 100644 index 000000000..47db53f65 --- /dev/null +++ b/doc/iocs/creation/IOC-Generator.md @@ -0,0 +1,62 @@ +# IOC Generator + +Generates the boilerplate structure required for developing an IBEX [IOC](../../IOCs). + +## How you use it + +Use these [instructions](https://github.com/ISISComputingGroup/IBEX-device-generator). + +## How to edit it/how it works/file locations + +Clone the [repository](https://github.com/ISISComputingGroup/IBEX-device-generator) into C:\Instrument\Dev. + +This program makes git branches and pushes to the remote repo, you can stop this by adding a return statement at the start of each method in git_utils (including the RepoWrapper constructor). + +The program starts in IBEX_device_generator.py, so we'll start there. +generate_device calls create_component for each component you might want, passing in an instance method. +These instance methods are grouped into classes: + - support_utils (generic IOC config) + - ioc_utils (ISIS specific IOC config) + - ioc_test_framework_utils (testing) + - emulator_utils (emulation) + - gui_utils (opi template creation) + +### support_utils + +Makes files in C:\Instrument\Apps\EPICS\support + +Templates for support module: C:\Instrument\Apps\EPICS\support\asyn\master\templates\streamSCPI +These templates are **generated from** templates in: C:\Instrument\Apps\EPICS\support\asyn\master\makeSupport\streamSCPI + + +### create_submodule + +Creates the top level directory in C:\Instrument\Apps\EPICS\support\, adds a makefile and tries to create a new git repo. + +### apply_support_dir_template + +Requires the directories made by create_submodule. +This will call a Perl script which generates the .db and .proto files for the support module. +The Perl script is C:\Instrument\Apps\EPICS\support\asyn\master\bin\windows-x64\makeSupport.pl. +The Perl script uses the templates in C:\Instrument\Apps\EPICS\support\asyn\master\templates\streamSCPI\_NAME_Sup. +These templates are **generated from** templates in: C:\Instrument\Apps\EPICS\support\asyn\master\makeSupport\streamSCPI\_NAME_Sup + + +## ioc_utils + +Makes files in C:\Instrument\Apps\EPICS\ioc\master + +Templates for ioc configuration: C:\Instrument\Apps\EPICS\base\master\templates\makeBaseApp\top +These templates are **generated from** templates in: C:\Instrument\Apps\EPICS\base\master\src\template\base\top +Run make in C:\Instrument\Apps\EPICS\base\master\src to regenerate the templates. + +## ioc_test_framework_utils + + +## emulator_utils + + + +## gui_utils + +Generates an opi template for the IOC. \ No newline at end of file diff --git a/doc/iocs/creation/Writing-An-ISIS-Stream-Device.md b/doc/iocs/creation/Writing-An-ISIS-Stream-Device.md new file mode 100644 index 000000000..300efce30 --- /dev/null +++ b/doc/iocs/creation/Writing-An-ISIS-Stream-Device.md @@ -0,0 +1,56 @@ +# Writing an ISIS StreamDevice IOC + +Note: The support module is put in the EPICS\\support directory, but the actual IOC(s) are put in the EPICS\\ioc directory + +Using the Hameg 8123 as an example. + +Make the main directory:: +``` + cd C:\EPICS\support + mkdir Hameg_8123 + cd Hameg_8123 + ``` +Create a version directory:: +``` + mkdir 1-0 +``` +Create a stream support module:: +``` + cd 1-0 + makeSupport.pl -t streamSCPI Hameg_8123 +``` + +Edit the protocol file in Hameg_8123Sup, so it reads like:: +``` + Terminator = '\r\n'; + ReplyTimeout = 2000; + + getIDN { + out "*IDN?"; + #Read no more that 39 chars (EPICS limit) + in "%/(.{0,39})/"; + ExtraInput = Ignore; + } + + getTriggerLevel { + out "LV\$1?"; + in "%s"; + ExtraInput = Ignore; + } + + setTriggerLevel { + out "LV\$1%s?"; + } + + resetCounts { + out "RES"; + } +``` +Note: I have only included a very small section of the command set for this device for brevity. + +Delete the db file in Hameg_8123Sup and modify the Makefile so it not longer refers to it. + +The next step is to create the IOC, the instructions are [here](Creating-an-ISIS-StreamDevice-IOC) + + + diff --git a/doc/iocs/creation/ioc-pcaspy.md b/doc/iocs/creation/ioc-pcaspy.md new file mode 100644 index 000000000..c2b9cbe86 --- /dev/null +++ b/doc/iocs/creation/ioc-pcaspy.md @@ -0,0 +1,74 @@ +# Creating a pcaspy IOC + +IOCs can be served via python using pycaspy. These let you create IOCs with PVs using channel access. They do not work quite like the normal EPICs so this the start of a guide to help create one. + +## Main Loop + +To Do + +## Allow IBEX to start the IOC + +The easiest way to let ibex start the pycaspy server (or in fact any python script) you need: + +1. Create dir (use same [naming rules](../conventions/IOC-Naming) as for other IOCs) +1. Add an empty makefile. Copy the one in LSICORR +1. Create the directory tree: `\iocBoot\ioc-IOC-01` + 1. Add a [standard `config.xml` to the folder](#ioc_finishing_touches_macros) + 1. Add a `runIoc.bat` file which runs the pycas py server with options. +1. If needed create a similar tree for a second IOC. + +## Registering IOC and PVs with IBEX + +To make sure that IBEX knows both that your IOC has started and what PVs should be archived and interesting use: + +``` +sys.path.insert(2, os.path.join(os.getenv("EPICS_KIT_ROOT"), "ISIS", "inst_servers", "master")) +from server_common.helpers import register_ioc_start, get_macro_values + +register_ioc_start(REFL_IOC_NAME, STATIC_PV_DATABASE, PREFIX) +``` + +where +- `STATIC_PV_DATABASE`: is a pv database used to construct the pv. To add a pv info field use entries in the pv for PV_INFO_FIELD_NAME. For example `{'pv name': {'info_field': {'archive': '', 'INTEREST': 'HIGH'}, 'type': 'float'}}` which will archive the val field and register the pv as high interest. +- `PREFIX`: is the prefix for the IOC e.g. `IN:NDXLARMOR:REFL_01` + +## Accessing PVs + +Most IOCs which need to access PVs use the `CaChannelWrapper` in `instservers` file `server_common/channel_access.py`. To do this: + +``` +import sys +import os +sys.path.insert(2, os.path.join(os.getenv("EPICS_KIT_ROOT"), "ISIS", "inst_servers", "master")) + +ChannelAccess.caget("{}BLAH".format(mypvprefix)) +``` + +## Accessing Macros on Start + +If you want to access the macros on start of a pv then add the following to your runIOC.bat: + +- `call dllPath.bat`: this adds the dll paths needed to run the icp config command. For example of this file see the reflectometry IOC. +- Nearer the end put the macros into an environment variable (`icpconfigGetMacros` is defined in [icpconfig](../tools/icpconfig)) + + ``` + set "GETMACROS=C:\Instrument\Apps\EPICS\support\icpconfig\master\bin\%EPICS_HOST_ARCH%\icpconfigGetMacros.exe" + set "MYIOCNAME=" + + if "%MACROS%"=="" ( + REM need this funny syntax to be able to set eol correctly - see google + for /f usebackq^ tokens^=*^ delims^=^ eol^= %%a in ( `%GETMACROS% %MYIOCNAME%` ) do ( set "MACROS=%%a" ) + echo Defining macros + ) else ( + echo Macros already defined + ) + + echo Macro JSON is %MACROS% + ``` + +- Inside the IOC you can access the macros with: + + ``` + from server_common.helpers import register_ioc_start, get_macro_values + get_macro_values() + ```` diff --git a/doc/iocs/testing/Add-sim-records-script.md b/doc/iocs/testing/Add-sim-records-script.md new file mode 100644 index 000000000..6a9233eaf --- /dev/null +++ b/doc/iocs/testing/Add-sim-records-script.md @@ -0,0 +1,7 @@ +# Add sim records script + +Records can be added to the db files by running: + +`python C:\Instrument\Apps\EPICS\ISIS\DbChecker\master\add_sim_records.py -o ` + +this will produce sim_ which includes sim records for various values as well as a disable record for the comms on all values. To not add the disable record specify the `-nd` flag and to specify an external PV to turn on/off record sim use the `-s` flag. \ No newline at end of file diff --git a/doc/iocs/testing/Disable-records.md b/doc/iocs/testing/Disable-records.md new file mode 100644 index 000000000..3bec0a27e --- /dev/null +++ b/doc/iocs/testing/Disable-records.md @@ -0,0 +1,38 @@ +# Disable records + +It would be convenient to be able to turn an IOC on/off by accessing a PV. We should allow this in IOCs that we write, but cannot guarantee that it will be always be possible. This could allow us to start all IOC's up in "disabled" mode, and then enable/disable by PV as per user choice, but for the moment the code below will start an IOC with comms enabled. The IBEX backend provide a macro setting the disable value this is `DISABLE` this need passed in dbload records. There is a helper script that will do this for you, as well as add `RECSIM`, see [here](Add-sim-records-script). + +The work required is similar to that for adding RecordSimulation - we add an '''SDIS''' field to any records that access hardware (i.e. that are not "soft channel" records) and point this at a PV to control the mode + +``` +record(ai, "$(P)CURRENT") +{ + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@kepco.proto readActualCurrent $(PORT)") + field(PREC, "3") + field(EGU, "A") + field(SDIS, "$(P)DISABLE") +} +``` + +``` +record(bo, "$(P)DISABLE") +{ + field(DESC, "Disable comms") + field(PINI, "YES") + field(VAL, "$(DISABLE=0)") + field(OMSL, "supervisory") + field(ZNAM, "COMMS ENABLED") + field(ONAM, "COMMS DISABLED") +} +``` + +Add to the IOC in the db load: + +``` +DISABLE=$(DISABLE=0) +``` +## Motors + +If you are looking to disable motors from display in the motors view you need to set `MTRXXXX_able.VAL 1` or 0 if you want to enable them. \ No newline at end of file diff --git a/doc/iocs/testing/Emulating-Devices.md b/doc/iocs/testing/Emulating-Devices.md new file mode 100644 index 000000000..b6d99af90 --- /dev/null +++ b/doc/iocs/testing/Emulating-Devices.md @@ -0,0 +1,155 @@ +# Emulating devices + +## Introduction + +You've created an IOC to talk to a device, and you want to test it: just borrow an actual piece of hardware and test with that. What if that's not possible? + +*The only way to know an IOC will work with an actual device is to use an actual device.* + +However, we can try and get as close as possible at the development stage. We also might want to make minor changes to an IOC we know that works without all the effort of tracking down an actual piece of hardware. The above principle still applies, but we can still take steps to improve our odds. + +Our emulators are written within the Lewis framework developed at ESS. The purpose of this page is not to replicate the full Lewis documentation, which can be found [here](https://isiscomputinggroup.github.io/lewis/index.html), but to give quick pointers to common actions and describe how it all fits within IBEX. + +Due to some staff turnover Lewis is now maintained by ISIS and ESS collaboratively. Dom has access to pushing release versions of Lewis to pypi. + +## Get the framework + +Lewis is included as an installed module in genie_python (for Python 3). + +## Set up a new emulator + +1. Create a subdirectory for your new emulator under `support/my_device/master/system_tests/lewis_emulators/`, for an example see the CCD100. +1. Documentation for how to write a Lewis emulator can be found [here](https://isiscomputinggroup.github.io/lewis/developer_guide/writing_devices.html), and you can refer to the examples in the Lewis library (i.e. `C:\Instrument\Apps\Python3\Lib\site-packages\lewis\devices` and `...\examples`). +1. NOTE: the simple examples `simple_device` and `example_motor` have all the code in a single `__init__.py` file, but we should stick to a consistent tidy structure like that of the `linkam_t95` emulator, i.e. with separate files for the device itself, its states (if it's a state machine), and its interfaces. +1. Don't forget to add `__init__.py` files in all of your folders! +1. At the time of writing, the Lewis `StreamAdapter.handle_error()` method does nothing. Please make sure your interface class deriving from `StreamAdapter` prints the content of the error, which makes it easier to understand what's going on (see for example the `iris_cryo_valve` emulator). + +## Run the emulator + +To run from the command line, use + +``` +%PYTHON3% -u -m lewis -p "stream: {bind_address: localhost, port: 57677}" -r 127.0.0.1:10000 -a C:\Instrument\Apps\EPICS\support\cryValve\master\system_tests -k lewis_emulators iris_cryo_valve +``` + +where we have picked port 57677 (see Lewis's doc for defaults). Note that the lewis executable is located in `%PYTHON3DIR%\Scripts`. + +**Note:** emulators used to be stored in one repository in `support\DeviceEmulator\` since https://github.com/ISISComputingGroup/IBEX/issues/6555 emulators should start being moved to live in the support directory for the IOCs that they are testing. + +Congratulations! Your emulator is now running. You can test it by connecting to it via a telnet client such as PuTTY (please see the troubleshooting note below) or with a simple Python script like so: + +```python +import socket + +OUT_TERMINATOR = "\r" + +s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +s.connect(("127.0.0.1", 57677)) + +while True: + cmd = raw_input() + s.sendall(cmd + OUT_TERMINATOR) + data = s.recv(4096) # Needs to be longer than the returned message + print data + +s.close() +``` + +### The backdoor + +It's possible to modify the device's state on the fly as it's running in case you want to push it into a specific state (as a backdoor). The backdoor can also be used to alter simulation parameters, e.g. to simulate a loss of connection or speed up the simulation time. Full documentation can be found [here for device access](https://isiscomputinggroup.github.io/lewis/user_guide/remote_access_devices.html) and [here for simulation access](https://isiscomputinggroup.github.io/lewis/user_guide/remote_access_simulation.html). + +The host and port for the backdoor are specified in the `-r` argument at startup: + +``` +%PYTHON3% -u -m lewis -p "stream: {bind_address: localhost, port: 57677}" -r 127.0.0.1:10000 -a C:\Instrument\Apps\EPICS\support\cryValve\master\system_tests -k lewis_emulators iris_cryo_valve +``` + +NOTE: at the time of writing, you can't type `localhost` for the `-r` argument. + +Once the emulator is running, the backdoor can be operated either via the command line through `lewis-control`: +``` +%PYTHON3DIR%\scripts\lewis-control.exe -r localhost: device "" "" "" +``` +or can be scripted, as described in the Lewis documentation. + +Replace each of the sections in `<>`s with the relevant values, not including the angle brackets. For example, to call the `backdoor_set_channel_property(self, channel_id, property_name, value)` method on an emulator running on port `59254`, you would write: + +``` +%PYTHON3DIR%\scripts\lewis-control.exe -r localhost:59254 device backdoor_set_channel_property "MB1.H0" "voltage" "10" +``` + +If you want to control an emulator running from a test, look at the lewis log for the device in `Instrument\Var\logs\IOCTestFramework` and use the port that the `ControlServer` is listening on. + +**NOTE**: If an argument is a string that contains words separated by white space characters, then you need to use "'argument'" instead of "argument", since otherwise the command line will not interpret it correctly and crash with a SyntaxError. + +**NOTE**: The simulation command `disconnect_device` seems to simulate the device not responding to the port, which is different from a lost connection: the IOC reports `No reply from device within xxx ms`. When the emulator is actually stopped, with the simulation `stop` command, the IOC detects that there is really no connection and reports `Can't connect to localhost:`. + +**NOTE**: The backdoor does not give access to private variables, so anything prefixed with `_` can not be changed in this way, or through the backdoor in python scripts. + +## Connecting your IOC + +We've got our emulator running, now we need to get our IOC talking to it. For this to work it needs to use the standard st.cmd setup so it works with the IOC testing framework. Then in your `globals.txt` do the following: + + - Add a line to set the IOC into dev sim `__DEVSIM=1` (this can go in the configuration) + - Set the port it should be communicating on (must be free) `__EMULATOR_PORT=57677` + +IOC name is the name of the ioc e.g. `EUROTHRM_01` + +## GO! + +Start the IOC as normal by running `runIOC.bat st.cmd`. If everything's hooked up correctly, you should see a `Client connected` message in the emulated device console. At the time of writing, Lewis emulators don't echo requests from the client, but this should be implemented soon. With any luck, the data from the emulator should then be updated to your PVs. + +## Connecting an Emulator to LabView + +If we have a pre-existing VI it might be useful to connect it to an emulator to test the emulator functionality is as expected. If the VI talks TCP then just ensure that the port it is talking to is the one that Lewis is served on. If the VI is attempting to talk to a serial connection this is a bit harder and you must do the following: +1. Find a MOXA box +2. Create a physical loopback on the MOXA by connecting one port into another using one male serial cable connected to one female serial cable (a plain network cable won't do) +3. Use NPort (which can be found under `\kits$\CompGroup\Utilities`) to connect to these two ports, noting which COM ports correspond to the loopback +4. Run the emulator (instructions above in "Run the emulator" section) +5. Run the com2tcp.py script found in https://github.com/ISISComputingGroup/EPICS-DeviceEmulator/ to create a connection between one COM port and Lewis e.g. `python com2tcp.py 57677 COM12` +6. Connect the VI to the COM port that you haven't run com2tcp.py on + +If the above is not working check that the baud rates/stop bits etc. set in the VI, NPort and com2tcp are all the same. otherwise, look in NI MAX and confirm that the COM ports are showing up under devices and interfaces on the left, if they are not you may need to update NI MAX. Also, note that the port you want to set in the driver VI is the one highlighted below (which can occasionally differ from the actual COM port) +![NI MAX](ni_max.JPG) + +If you want to test against the LabVIEW VI and you aren't in a position to create a loopback in the MOXA, there is a VI available which you can use within the communication VI instead of the serial connections. +1. Create some room inside the main case +2. On the block diagram, move the reply to the far right of the main case, the read reply button and the command to the far left of the main case +3. Drop a `disable diagram` structure around the serial communications section +4. In the enabled state, drop in an instance of `IBEX Integration - Connect to TCP.vi`, which can be found in the `IBEX Integration.llb` in the `Labview Modules\Common\Utilities` directory (if the VI isn't there, then you need to get an updated version of the repo) +5. Connect the errors, reply, command and so on in the obvious fashion, connect the COM port to the Port In, and create a constant for the mode +6. Change the COM port in the setup dialog to the port of the emulator (this has to be running on localhost at the moment) +7. Run the emulator and VI and they should be talking to each other + +## Troubleshooting + +We haven't done much with emulators yet, so not much has gone wrong, so please add to this section as you can. + +* Telnet server is running, but is not receiving any data from the IOC: Is your st.cmd correct? Try removing the 4 `< $(IOCSTARTUP)...` lines, and the `drvAsyn{IP,Serial}PortConfigure` lines and run `runIOC.bat st.cmd`. Are you getting any error or warning messages? Sort those out first. +* When connecting to a Lewis emulator via a Telnet client such as PuTTY, beware that Telnet uses `\r\n` as a terminator. If your emulator interface has a different one (like for the `linkam_t95`), the protocol won't work. You could temporarily use the Telnet terminator instead. Note also that PuTTY sends some extra characters at the start of the communication, so the very first command you send probably won't work. +* Note that lewis can't deal with not having a termination character. If your device doesn't use a termination character then you will have to temporarily use one while talking to the emulator. +* `An error occurred: +The setup 'default' you tried to load does not specify a valid device type, but +the device module 'neocera_ltc21' provides multiple device types so that no meaningful default can be deduced.`. Possible solutions: + - Add device to `__init__` file of package so it can be imported. + - Ensure that the initial state is one of the states returned by get_state_handlers. +* When I try to launch `lewis.exe` I get the error `Fatal error in launcher: Unable to create process using '"'`. When you build Python on Windows, the Python path is baked into the `lewis.exe` executable. If you subsequently say move `Python-build` to `Python` then the path will be incorrect and the executable doesn't know where to launch from. You can either run lewis as a module e.g. `%PYTHON3% -u -m lewis` or run it by importing it into a python script. +* When I try to print something from the device emulator, nothing happens. Why? +Print statements in the device emulator can not print anything to a console when they are ran as part of the IocTestFramework. +* I want to log something how do I do that? + 1. include `@has_log` at the top of the class (don't forget to `from lewis.core.logging import has_log`) + 1. use self.log.info(message), self.log.warning(message), self.log.error(message), etc +* When I try to run a device I get the error `Failed to find protocol stream...`. This is due to one of the imports in the stream_interface not being valid. Check that they are all correct. +* When I try to access a variable that I know exists in my emulator, I get an error saying that variable does not exist? + 1. The lewis backdoor does not give access to private variables, so anything prefixed with `_` cannot be changed in this way. +* If you are using `CmdBuilder` be aware that you should use `.eos()` before `.build()`, _especially_ if you have commands that 'overlap'. And example of this would be on the Keithley 2700, which has a buffer auto clear setting command, `TRAC:CLE:AUTO`, and a buffer clear command, `TRAC:CLE`. `.eos()` essentially tells the built regex to match the exact command string, rather than some of it. +* When running the IOC tests with `make ioctests` you get no output until all the tests are run. This is set in general for all `Makefiles` to avoid interleaved printing when doing a parallel build. To fix this you need to remove the `-Otarget` from the `MAKEFLAGS` environment variable. e.g. run `set "MAKEFLAGS=-w -j 6"` before running the test. This environment variable will be reset back every time you start a new EPICS terminal. +* Use of `@property` python decorator is not supported within `stream_interface.py` and will cause the emulator to fail. Please try to avoid using such decorators in your python scripts for use with emulators as they will cause issues when trying to construct Func-object instances during the build process. + +#### When using an emulator with a VI + +* If you are having problems getting data into or out of your emulator when using a VI, it could be a comms issue. + * Check the baud rate and other serial parameters if using serial + * Double check the port that your VI is connected to +* The VI may be polling/looping too fast for your emulator. Some VIs are written to go flat out as fast as possible. This may be faster than your python emulator can handle. If you are getting no data from/into the VI, try slowing it down and increasing its loop delays. e.g. the VI controlling the Keithley 2700 was looping every 2ms, and when data was inserted into the emulated device buffer, the VI showed no change. This is because the loop was too fast for the data to be processed properly by the emulator and then VI. The delay was increased to 20ms (still extremely fast for the intended purpose), and the VI and emulator worked (make sure that the increased delay is not unreasonable and the device can still be expected to work properly). diff --git a/doc/iocs/testing/IOC-Testing-Framework.md b/doc/iocs/testing/IOC-Testing-Framework.md new file mode 100644 index 000000000..509c82b8c --- /dev/null +++ b/doc/iocs/testing/IOC-Testing-Framework.md @@ -0,0 +1,5 @@ +# IOC Testing framework + +See documentation at [https://github.com/ISISComputingGroup/EPICS-IOC_Test_Framework/blob/master/README.md](https://github.com/ISISComputingGroup/EPICS-IOC_Test_Framework/blob/master/README.md) + +As of [6555](https://github.com/ISISComputingGroup/IBEX/issues/6555) IOC system tests are being moved from a single location to being next to the device they test. The tests next to the device can be run using `run_tests.bat` in `support/device/master/system_tests`. This bat file will pass through any command line arguments to the IOC_Test_Framework and so arguments documented above can still be used. You can also run `make ioctests` in the `support/device/master` to run the tests but this will not let you pass through command line arguments and will not display output until the whole test is finished. diff --git a/doc/iocs/testing/Record-Simulation.md b/doc/iocs/testing/Record-Simulation.md new file mode 100644 index 000000000..e110bca3d --- /dev/null +++ b/doc/iocs/testing/Record-Simulation.md @@ -0,0 +1,94 @@ +# Record simulation + +EPICS supports the concept of running IOC records in "simulation mode" - here rather than reading input/output from devices +according to the INP/OUT fields the record will bypass hardware and use an alternative value. We should only need to add this support to records that access hardware, other records should be able to work as normal on these supplied simulated values. + +Simulated values can either be a constant, or can be read/written to another PV location. The principle we will use here is to create a set of dummy records named $(P)SIM:* to which we will read and write values, hence allowing us to join a set and a read together. Initially these dummy records will be "Soft Channel" analogue/binary records which will be joined so that readback and set are connected, in future these records could be pointed at a dummy ioc mechanism to e.g. smoothly ramp. + +Though a simulated motor exists, it is probably still worth doing record simulation for e.g. jaws as it provides a convenient way to test GUIs without having to start additional IOCs + +The EPICS record has to initialise first before simulation mode is enabled internally, and this is where problems can occur. Some drivers will try to read a value from the hardware at record initialisation, and if this fails they leave the record in an error state that stops simulation mode working. If you see PACT is stuck at 1 this is typical of a failed record initialisation. You will wither need to fix the device driver, or avoid this record when using recsim. + +There are several cases where recsim doesn't work properly (or not without significant extra work): +- Records with `PINI="YES"` - initialisation runs before channel access is active which causes issues. +- For I/O interrupt records there will be no hardware to trigger, but you can use the SSCN (simulation scan mode) +- Records that get pushed to from a protocol file, as the protocol file will not be run. +- MBBI/MBBO records with Soft channel device support. Soft channel doesn't populate `RVAL`, it is possible to work around this in some simple cases but often the benefit of adding recsim to these records is not worth the time to get it to work. + +There is a [script to help](Add-sim-records-script). +First add the following record that will be used to indicate if simulation mode is being used (this will have been done for you if you used the [device generator script](../creation/IOC-Generator)): + +``` +record(bo, "$(P)SIM") +{ + field(SCAN, "Passive") + field(DTYP, "Soft Channel") + field(ZNAM, "NO") + field(ONAM, "YES") + field(VAL, "$(RECSIM=0)") +} +``` + +Add to the IOC in the db load (this will have been done for you if you used the [device generator script](../creation/IOC-Generator)): + +``` +RECSIM=$(RECSIM=0) +``` + + +next you need to modify any records that talk to real hardware (i.e. those where DTYP is not "Soft Channel" or "Raw Soft Channel"). You add the SIML field (to tell the record whether it should run in simulation mode) and the SIOL fields (to tell it where to read/write values when in simulation mode). For example: + +``` +record(ai, "$(P)CURRENT") +{ + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@kepco.proto readActualCurrent $(PORT)") + field(PREC, "3") + field(EGU, "A") + field(SIML, "$(P)SIM") + field(SIOL, "$(P)SIM:CURRENT") +} + +record(ao, "$(P)CURRENT:SP") +{ + field(SCAN, "Passive") + field(DTYP, "stream") + field(OUT, "@kepco.proto writeCurrent $(PORT)") + field(EGU, "A") + field(PREC, "3") + field(SIML, "$(P)SIM") + field(SIOL, "$(P)SIM:CURRENT:SP PP") +} + +record(ai, "$(P)CURRENT:SP:RBV") +{ + field(SCAN, "1 second") + field(DTYP, "stream") + field(INP, "@kepco.proto readSetpointCurrent $(PORT)") + field(PREC, "3") + field(EGU, "A") + field(SIML, "$(P)SIM") + field(SIOL, "$(P)SIM:CURRENT:SP:RBV") +} +``` + + + +We now need to add, and join if necessary, the relevant $(P)SIM dummy records. For the moment we will just do a simple join of the records by creating an alias - however by using an alias rather than pointing at a single PV we allow a future option of using a soft IOC to e.g. ramp values smoothly. + +``` +record(ao, "$(P)SIM:CURRENT") +{ + field(SCAN, "Passive") + field(DTYP, "Soft Channel") +} + +alias("$(P)SIM:CURRENT","$(P)SIM:CURRENT:SP") + +alias("$(P)SIM:CURRENT","$(P)SIM:CURRENT:SP:RBV") +``` + +To link the simulation to a record that is passively scanning you should add `field(FLNK, "")` to the simulation record. + +For more information on fields in simulation records see [here](https://epics.anl.gov/base/R7-0/6-docs/dbCommonInput.html) for input records, or [here](https://epics.anl.gov/base/R7-0/6-docs/dbCommonOutput.html) for output records. \ No newline at end of file diff --git a/doc/iocs/testing/Running-IOCs.md b/doc/iocs/testing/Running-IOCs.md new file mode 100644 index 000000000..83aab12be --- /dev/null +++ b/doc/iocs/testing/Running-IOCs.md @@ -0,0 +1,61 @@ +# Running IOCs locally + +## Start the IOC + +Either: + +* start the IOC through the GUI. + * If this is a new IOC you will need to run `make iocstartups` in the EPICS folder for the IOC to be displayed in the GUI. +* start the IOC through the console (ctrl + x) +* switch off/on auto start (ctrl + t) +* start the IOC by running `runIOC.bat st.cmd` (in `iocBoot/Appdir`) + +## IOC in procServ using the Console + +Open an epics terminal a list all running instruments with + + console -M localhost -x + +To interact with an IOC use + + console -M localhost + +console will attempt to complete the name if you only give part of it and will give you possible options. Once in the console: +* `ctrl-t` : toggles the auto-restart setting +* `ctrl-x` : starts and stops the IOC +* `crtl-e` `c` `.`: exits the console + +# Simulation Mode + +TO switch an IOC to simulation mode the default is + + caput :SIM 1 + +## Macros + +It is possible to set macros for an IOC either through the IBEX GUI or using the globals.txt file. Loaded in using [icpconfig](../tools/icpconfig). + +### Globals.txt + +The globals.txt is file is held in `C:\Instrument\Settings\config\NDW_____\configurations`. In here, global macros are set with `=`, and macros specific for an IOC are set with `__=` + +e.g. + + GALIL_01__GALILADDR=130.246.51.169 + GALIL_01__MTRCTRL=2 + GALIL_03__MTRCTRL=3 + GALIL_03__DEVSIM=1 + EGXCOLIM_01__RECSIM=1 + +## Reading a Compressed Hex PV + +Use the following to read a waveform PV of a compressed hexed string + + caget -t -S |uzhex + +(uzhex - think unzip hex) + +## Troubleshooting + +[See IOC and device Troubleshooting](../Troubleshooting) + diff --git a/doc/iocs/testing/Setting-up-googleTest-to-work-with-EPICS-build-process.md b/doc/iocs/testing/Setting-up-googleTest-to-work-with-EPICS-build-process.md new file mode 100644 index 000000000..80a6c45ba --- /dev/null +++ b/doc/iocs/testing/Setting-up-googleTest-to-work-with-EPICS-build-process.md @@ -0,0 +1,125 @@ +# Google Test + +This is a guide to set up googleTest with EPICS at ISIS. This allows you to write C++ unit tests for your C/C++ code in an IOC. Examples of IOCs that run tests like this are the [Keithley 2001](https://github.com/ISISComputingGroup/EPICS-Keithley_2001) and the [cryosms](https://github.com/ISISComputingGroup/EPICS-Cryosms). + +## Contents +* [Introduction](#googletest_intro) +* [Release file](#googletest_release_file) +* [Building the test runner](#googletest_building_test_runner) +* [Running your tests](#googletest_running_tests) +* [Sample Test](#googletest_sample) +* [Adding a target to run tests](#googletest_external_dependencies) +* [Troubleshooting](#googletest_troubleshooting) + + +{#googletest_intro} +## Introduction + +You will need to have the `googleTest` support submodule and built the master branch. This will create a `gtest.lib` which you can link against. + +More information on googleTest can be found at https://github.com/abseil/googletest. We are using version 1.8.x currently at ISIS. + +Good places to start on how to write tests using googleTest is [here](https://github.com/abseil/googletest/blob/master/googletest/docs/primer.md). + +More advanced usage (including `ASSERT_THROWS`, `ASSERT_NO_THROW` and `ASSERT_DOUBLE_EQ`) can be found [here](https://github.com/abseil/googletest/blob/master/googletest/docs/advanced.md). + +Examples can be found [here](https://github.com/abseil/googletest/blob/master/googletest/docs/samples.md). + +{#googletest_release_file} +## Release file + +Remember to add the path to the Google Test support module in your support module Release file (in `configure/Release`). + +``` +GTEST=$(SUPPORT)/googletest/master +``` + +{#googletest_building_test_runner} +## Building the test runner + +Make sure you include the following lines in your Makefile alongside the source code to create a test runner executable. + +```Makefile +ifeq ($(findstring 10.0,$(VCVERSION)),) +# googleTest Runner + +GTESTPROD_HOST+= runner +GTESTS += runner +endif +``` + +Note: you should exclude the line `GTESTS += runner` if you are using `run_tests.bat` as described below (i.e. if your tests need external dependencies). This is the line that tells the default `runtests` makefile target to include this test. + +You then need to include all the files you need for your tests using +```Makefile +runner_SRCS += #names of test files and source files +``` + +If your tests are in a different directory to your source files, you can add that directory to your SRCs path using + +```Makefile +SRC_DIRS += #path to your tests directory +``` + +{#googletest_running_tests} +## Running your tests + +Once built you can run the tests by doing `make runtests` in your submodule. This will print the test results and also save them to an xml file in `src/O.$(EPICS_HOST_ARCH)`. This file will be picked up by Jenkins. + +{#googletest_sample} +## Sample Test + +The following below is a sample test file + +```C++ +#include "gtest/gtest.h" + +namespace { + TEST(Sample, this_is_a_sample_test){ + ASSERT_EQ(1+1, 2); + } +} // namespace + +``` + +{#googletest_external_dependencies} +## Tests with external dependencies + +In some cases you may want to write unit tests that depend on other EPICS modules e.g. `asyn`. To do this you will need to create the required dlls and run the tests manually. First in your makefile in `src` add the following lines at the bottom: +```Makefile +ifdef T_A +install: dllPath.bat + +dllPath.bat: + $(CONVERTRELEASE) -a $(T_A) -T $(IOCS_APPL_TOP) $@ +endif +``` + +Then in the top of your submodule create `run_tests.bat` that looks like: +```bat +@echo off +setlocal +set "ARCH=%1" +set "TESTPATH=%~dp0App/src/O.%ARCH%" +if exist "%TESTPATH%\runner.exe" ( + call %TESTPATH%\dllPath.bat + %TESTPATH%\runner.exe --gtest_output=xml:./test-reports/TEST-.xml +) else ( + @echo No tests to run +) +``` +Then in the top makefile you will need to override the `runtests` rule by adding in: +```Makefile +.PHONY: runtests + +runtests: + run_tests.bat $(EPICS_HOST_ARCH) +``` +An example of an IOC that does this is the [cryosms](https://github.com/ISISComputingGroup/EPICS-Cryosms). + +{#googletest_troubleshooting} +## Common issues + +### Tests pass but error afterwards "Cannot detect source of runner.run" + +You may need to include the line `-include $(GTEST)/cfg/compat.RULES_BUILD` at the end of your Makefile, note the leading `-` which is to make the include optional, this is to stop failures on a global `make clean` when `$(GTEST)/cfg` will get removed before the module is processed. \ No newline at end of file diff --git a/doc/iocs/testing/ni_max.JPG b/doc/iocs/testing/ni_max.JPG new file mode 100644 index 000000000..143a0dd0a Binary files /dev/null and b/doc/iocs/testing/ni_max.JPG differ diff --git a/doc/iocs/tips_tricks/Communication-layer-likes-and-dislikes-(protocol,-new-devices).md b/doc/iocs/tips_tricks/Communication-layer-likes-and-dislikes-(protocol,-new-devices).md new file mode 100644 index 000000000..7e1be4a09 --- /dev/null +++ b/doc/iocs/tips_tricks/Communication-layer-likes-and-dislikes-(protocol,-new-devices).md @@ -0,0 +1,69 @@ +# Comms layer likes & dislikes + +This page documents an "idealised" communication protocol for communicating with new sample environment drivers at ISIS. It is intended as a useful starting point if new protocols are being developed from scratch and as a set of nice-to-haves for new equipment being purchased. + +## Transport layer + +### Serial (RS232, RS422, RS485) + +This is the most common transport layer at ISIS. Serial settings are flexible in all of our drivers but the most common configuration is: + +- Baud rate: 9600 +- Data bits: 8 +- Parity: None +- Stop bits: 1 +- Flow control: Off + +RS232 is preferable, RS422 and RS485 are avoided as far as reasonable. Note that distances can be extreme if higher baud rates are required. Preferred socket is DB-9, but DB-25 also acceptable. + +### Ethernet + +Ethernet is a commonly used and well-supported transport layer at ISIS using RJ45 ports (AKA 8P8C sockets). Any other port type is to be avoided. It is also preferable that devices use DHCP to find their IP address rather than being issued a static address. + +### USB + +USB devices are difficult to integrate into the Experiment Control System, so should be avoided. + +### Manufacturer software / DLLs + +We prefer not to use these interfaces. However, in some cases, we may be able to work with these. + +### MODBUS + +MODBUS is more difficult for us than ASCII protocols but we can deal with this. + +### Connection Hardware + +Use “standard”, easily maintainable and available connection hardware where possible, so no unusual or bespoke sockets, cables or adapters preferred. + +## Protocol + +### Termination characters + +It is very helpful for devices to terminate their messages with a unique set of characters that do not appear elsewhere in the message. A common example of a terminator is a carriage-return, line-feed pair (``, `/r/n`, HEX: `0D 0A`). + +### Readability/encoding + +Commands to a device are ideally human-readable so that they can be used via terminal emulators and the device interacted with. Many protocols use an ASCII encoding. + +For example, a temperature controller might implement commands similar to the following: + +``` +Driver to device: +temp? +(HEX: 74 65 6D 70 3F 0D 0A) + +Device to driver: +temp=30.5 +(HEX: 74 65 6D 70 3D 33 30 2E 35 0D 0A) +``` + +### Replies + +All commands to a device should ideally return some form of reply. This helps the control system to distinguish between a device which is unplugged and an incorrect or invalid command. + +Where no data needs to be returned, a device could generate an "ACK" or "OK" or similar response. Commands which are invalid or rejected could return a "NAK" or "NOK" or "error" or similar responses. + +### Unsolicited messages + +We prefer not to receive unsolicited messages. It is better for a driver to explicitly ask for a parameter each time it is required. diff --git a/doc/iocs/tips_tricks/Email-Text-Alerts-from-IOCS.md b/doc/iocs/tips_tricks/Email-Text-Alerts-from-IOCS.md new file mode 100644 index 000000000..988b2c48a --- /dev/null +++ b/doc/iocs/tips_tricks/Email-Text-Alerts-from-IOCS.md @@ -0,0 +1,3 @@ +# Email & Text alerts from IOCs + +See https://github.com/ISISComputingGroup/EPICS-ioc/blob/master/RUNCTRL/README.md \ No newline at end of file diff --git a/doc/iocs/tips_tricks/Flow-control.md b/doc/iocs/tips_tricks/Flow-control.md new file mode 100644 index 000000000..0d0ad8627 --- /dev/null +++ b/doc/iocs/tips_tricks/Flow-control.md @@ -0,0 +1,36 @@ +# Flow control + +Flow control is used in serial devices to allow them to better communicate when they are ready to send/receive data. For most devices it should be switched off, unless we know for sure that the device needs to use it. We have seen a number of cases in the past where having it on when not necessary has caused issues. + +All IOCs should set this because when connecting to a port if you don't set it the previous value may be used, i.e. if it had the flow control on for the last device the moxa may remember that setting for the next device. + +Flow control comes in two varieties, [hardware](https://en.wikipedia.org/wiki/Flow_control_(data)#Hardware_flow_control) and [software](https://en.wikipedia.org/wiki/Software_flow_control). + +## Hardware flow control + +Set *off* (probably the most used) +``` + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", 0, "clocal", "Y") + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"crtscts","N") +``` +Set *on* using +``` + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", 0, "clocal", "N") + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"crtscts","Y") +``` +in the device's `st.cmd`. To turn this off set it to `N` or remove the line completely. Under windows there is a further option to set it to `D` which ensures the hardware lines are floating (this is usually unnecessary). + +## Software flow control + +In the device's `st.cmd`: + +Set *off* using +``` + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixon","N") + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixoff","N") +``` +Set *on* using +``` + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixon","Y") + $(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0",0,"ixoff","Y") +``` diff --git a/doc/iocs/tips_tricks/Handling-Invalid-Values.md b/doc/iocs/tips_tricks/Handling-Invalid-Values.md new file mode 100644 index 000000000..e7bf06799 --- /dev/null +++ b/doc/iocs/tips_tricks/Handling-Invalid-Values.md @@ -0,0 +1,71 @@ +# Handling invalid values + +From time-to-time, IBEX encounters problems determining the value of a PV (e.g. because of a communication error on a device). The purpose of this page is to set out the conventions used by IBEX to indicate to the user that the value of a PV or block should be considered invalid. + +Blocks are, essentially, aliases for PVs. Therefore, if the value of a PV cannot be determined, the value of a corresponding block is equally invalid. + +## Situations Giving Rise to Invalid Values +The following situations can cause process variables (PVs) to be considered invalid. +* communication error between a device and IBEX +* device reports that a value is invalid (i.e. over range) + +There may be other situations which can cause process variables (PVs) to be considered invalid. If so, define them here. + +### Communication Errors +When EPICS detects a communication error with a device, it does two things: +1. the PV continues to hold the last-known value +1. the alarm status of the PV is changed to `INVALID` + +### Calibration Errors +Some devices return "raw" values from sensors. Such values have to be converted into meaningful physical values via some kind of calibration algorithm (in most cases, the algorithm is merely a simple linear conversion or a table lookup plus linear interpolation). For example, a Eurotherm returns the raw voltage generated by a thermocouple, which is converted into a temperature using a lookup table. + +All calibration algorithms have a range of validity - if the "raw" sensor value falls outside that range the calibrated physical value should be considered invalid. Calibration calculations are performed in the IOC, therefore, any out of range errors have to be signalled by the IOC. In this situation, it is suggested that the IOC behaves as follows: +1. the PV holds the converted value (even though it might be invalid) +1. the alarm status of the PV is changed to `[can-we-invent-a-calibration-alarm?]` + +## IBEX Conventions for Invalid Values +For each type of invalid value (described above), IBEX will behave according to the conventions described below. There are specific conventions for different features of IBEX - the dashboard area of the IBEX GUI (i.e. the IBEX client), log files, genie_python, run control, the Web Dashboard. + +### Communication Errors: +In this situation IBEX cannot read a value from a device. In practice, this means EPICS will signal an `INVALID` alarm on a PV. On encountering an `INVALID` alarm, IBEX will behave as follows: +#### IBEX GUI +1. In the dashboard, blocks will display the text "N/A" and be surrounded by a purple border +1. In OPIs, readback fields (which display PVs, not blocks) will display the last-known-good-value and be surrounded by a purple border. On hovering the mouse over a readback field, IBEX will display a pop-up box showing the PV name, the last-known-good-value and the alarm status. + +#### Log & Data Files +1. In the NeXus data file, blocks will be logged with their last-known-good-value and the alarm status. +1. In the IBEX archive, PVs will be logged with their last-known-good-value and the alarm status. + +Applications or scripts which process log & data files are responsible for checking the alarm status of a block or PV and taking appropriate action. + +#### genie_python +In genie_python, `cget` is used to obtain the value of a block or PV. `cget` always +1. returns the value of a block or PV +1. returns the alarm status of the block or PV +1. prints a warning message if there is an alarm on the block or PV. + +It is the responsibility of the script author to check the alarm status and take any subsequent actions required. + +#### Run Control +1. If a block has run control applied to it and that block goes into alarm (including `INVALID` alarm) then + * if the last-known-good-value was within the run control limits, data collection will continue + * if the last-known-good-value was outside the run control limits, data collection will not resume until the alarm is cleared. + +#### Web Dashboard +1. In the web dashboard, blocks will display the text "INVALID". + +### Calibration Errors +At the current time, there is no agreed convention. + +### Alternative Conventions +IBEX has adopted the conventions defined above. Other systems and devices have adopted different conventions for signalling that a readback value might be invalid. We have deliberately decided not to follow these conventions to avoid the risk of ambiguity or clashes in interpretation. + +#### Why haven't we used NaN? +NaN, which means "not a number", is a concept used in computing to represent the results of calculations that are somehow "indeterminate". Superficially, therefore, it might seem that NaN is a good candidate for representing an invalid value. +However, there are several reasons for not choosing NaN for this purpose: +1. Specifically, NaN, as defined by the IEEE-754 standard, applies only to floating-point numbers. There is no concept of NaN for integer-valued variables. Nor is there an equivalent concept for string-valued variables. So, NaN is a only a partial solution, at best. +1. Some devices use NaN to signal that a value has not been defined or initialized. Strictly speaking this is an abuse of the IEEE-754 standard. Nevertheless, because some device manufacturers have adopted this convention, it means that there is a risk of confusion - if a device returns NaN does it mean that the value was never initialized, or has it become invalid (having previously been valid)? +1. In arithmetical expressions, NaNs behave as follows: + * `anything + NaN = NaN` - i.e. NaNs propagate through arithmetical operations + * all comparisons involving NaNs return `FALSE` (except the `!=` operator, which returns `TRUE`). +1. If NaNs appear in data or log files, then any code reading such files might need modification to take appropriate action on encountering a NaN. diff --git a/doc/iocs/tips_tricks/IOC-Tips-and-Tricks.md b/doc/iocs/tips_tricks/IOC-Tips-and-Tricks.md new file mode 100644 index 000000000..7344e9361 --- /dev/null +++ b/doc/iocs/tips_tricks/IOC-Tips-and-Tricks.md @@ -0,0 +1,115 @@ +# IOC Tips & Tricks + +## DBLoadRecordsLoop and DBLoadRecordLoop + +You can load Db records in a loop using `dbLoadRecordsLoop`. + +``` +# Example usage of dbLoadRecordsLoop() and iocshCmdLoop() etc. +# Define a loop or list variable, then references to \$(VAR) will be replaced by loop/list values. +# You can use \$(VAR) in any of the values of the parameters, including the db file name. +# The loop/list variable "I" is passed to the DB file along with any other specified macros. +# We now loop "I" over a range of 1 to 3. +dbLoadRecordsLoop("db/utilitiesTest.db", "P=$(MYPVPREFIX),Q=A\$(I)", "I", 1, 3) +``` + +You can also iterate over a list using `dbLoadRecordsList`. + +``` +# Here S is our list variable with values x, Y and Z. +dbLoadRecordsList("db/utilitiesTest.db", "P=$(MYPVPREFIX),Q=A\$(S)", "S", "X;Y;Z", ";") + +# As the loop/list variable is always passed though, we can instead iterate over Q. +dbLoadRecordsList("db/utilitiesTest.db", "P=$(MYPVPREFIX)", "Q", "BX;BY;BZ", ";") +``` + +You can execute IOC commands in loops as well. + +``` +# We can execute iocs commands in a similar way. +iocshCmdLoop("dbgrep $(MYPVPREFIX)A\$(I)*", "", "I", 1, 3) +iocshCmdList("dbgrep $(MYPVPREFIX)A\$(S)*", "", "S", "X;Y;Z", ";") + +# We can also load files. The macro Q and I will be defined as temporary environment. +# variables for use in the script and then reset back to their pre-script values. +iocshCmdLoop("< st\$(I).cmd", "Q=Hello\$(I)", "I", 1, 2) +``` + +More details can be found at http://epics.isis.stfc.ac.uk/doxygen/main/support/utilities/dbLoadRecordsFuncs_8cpp.html#a32071967b99f42356b1e04b06746cc73. + +**N.B.** If you need to parameterise your macro names, you will need to use a template and not a `dbLoadRecordsList` or `dbLoadRecordsLoop`. + +## Toggle multiple PV's from a single set-point + +Sometimes you might want to have a single record set point that processes multiple records. An example of this would be for a device that requires different commands to be sent for Start and Stop. Rather than having to process two distinctly different records for each command you can use this template to toggle these from a single record: + +``` +record(bi, "$(P)TOG:RECORD:SP") { + field(DESC, "Toggle record set point") + + field(ZNAM, "Record1") + field(ONAM, "Record2") + info(INTEREST, "HIGH") +} + +# Add 1 to the value of the toggle set point to map the fanout +# forward link index. +record(calcout, "$(P)_TOG:CALC:RECORD") { + field(INPA, "$(P)TOG:RECORD:SP CP") + + field(CALC, "A+1") + + field(OUT, "$(P)_TOG:RECORD.SELN PP") +} + +record(fanout, "$(P)_TOG:RECORD") { + field(SELM, "Specified") + + field(LNK1, "$(P)_RECORD1:SP PP") + field(LNK2, "$(P)_RECORD2:SP PP") +} + +record(bo, "$(P)_RECORD1:SP") { + field(DESC, "Process record1") + field(DTYP, "stream") + field(SCAN, "Passive") + + field(OUT, "@device.proto set_record1() $(PORT)") +} + +record(bo, "$(P)_RECORD2:SP") { + field(DESC, "Process record2") + field(DTYP, "stream") + field(SCAN, "Passive") + + field(OUT, "@device.proto set_record2() $(PORT)") +} +``` + +This takes the set points binary value, shifts it by +1 in a calc record which allows for it to be mapped with the forward link outputs of the fanout record, these forward links are then processed. This can be easily extended with additional forward links or sequence records. + +## Processing of calc records + +There are two major ways in which your calc records can be processed: + +1)The first way is to either use a forward link or have the name of your calc record in the OUT field of another PV, followed by the PP flag. In this way, the calc record will not be processed when the IOC starts. This way is better for when the other PV is a setpoint, since you want the calc record to process only when the setpoint is explicitly set, not also when the IOC starts and when the setpoint is initialized. + +2)The other way is to have the PV that should trigger the processing of the calc record in the an INP field of the calc record, followed by the CP MS flags. The CP flag enables monitoring of that PV, and then the MS flag means any alarm of that PV will propagate to the calc record. The calc record will be processed every time the other PV is processed, including when the IOC starts. This way is better for when the other PV is a readback, because you want the calc to be processed each time the readback changes. + +## Creating many IOC aliases + +If you would like to create a set of PV aliases in an IOC from the `st.cmd` then that is possible using `dbAliasRecordsPrefix` (to simply swap a prefix) or `dbAliasRecordsRE` (to use a more complicated regular expression) + +To create aliases for all records with e.g. the `ME:` (movable equipment) prefix that instead have the local pv prefix +``` +dbAliasRecordsPrefix("ME:", "$(MYPVPREFIX)") +``` +The equivalent using a regular expression would be +``` +dbAliasRecordsRE("ME:(.*)", "$(MYPVPREFIX)\1") +``` +but much more complex rewrites are possible + +## For general information on commands see: +* http://epics.isis.rl.ac.uk/doxygen/main/support/utilities/index.html +* https://epics.anl.gov/base/R3-14/12-docs/AppDevGuide/node19.html \ No newline at end of file diff --git a/doc/iocs/tips_tricks/PV-with-a-limited-range.md b/doc/iocs/tips_tricks/PV-with-a-limited-range.md new file mode 100644 index 000000000..9c4c5f94b --- /dev/null +++ b/doc/iocs/tips_tricks/PV-with-a-limited-range.md @@ -0,0 +1,4 @@ +# PV with a limited range + +If a setpoint on a device has a range set by the device it should be set up by using DRVH and DRVL. This will limit the value that can be set and set it to the largest possible value which will allow scripts to do the best possible. [See [Decision Log 12](/processes/Decision-Log)] + diff --git a/doc/iocs/tips_tricks/Some-Design-thoughts-for-a-serial-or-Ethernet-IOC.md b/doc/iocs/tips_tricks/Some-Design-thoughts-for-a-serial-or-Ethernet-IOC.md new file mode 100644 index 000000000..a23ceb6a4 --- /dev/null +++ b/doc/iocs/tips_tricks/Some-Design-thoughts-for-a-serial-or-Ethernet-IOC.md @@ -0,0 +1,19 @@ +# Design thoughts for a serial or ethernet IOC + +## Serial Devices +Applicable - For IOCS (or any code) which ultimately drives something at the end of a real wire or a fibre where the comms. are very much an end-to-end affair. + +Not applicable - Some IOCS are to drive things in crates (with hardware slots interrupt lines etc.) this is not about these, also it is not about things that communicate with other bits of manufacturers code (unless this ultimately ends up driving a serial device). + +## Different models for device communication +There are generally two sorts of "serial" devices. + +Those that "chatter" to the computer regardless of whether you talk to them, for those an IOC should treat this chatter as a heartbeat (i.e. timeout and report a comms. error if it doesn't receive what it should within a time). We’ve generally not preferred these sort of comms. (if we have a choice) because they can be a bit awkward to program and are by definition asynchronous with the code talking to them (unfortunately, hardware people often quite like these). + +Alternatively, there are "obedient" devices which only speak when they are spoken to. These must be polled because this is the only way we will know they still are working. In fact, they must really be polled so as to return at least one data value or response packet. Devices which accept silent commands need to checked for data coming back (one wire may not be connected). The rate of polling for data can then be set lower than this (or on demand) if values are not needed very often. An IOC is simplified if heartbeat and data needs are satisfied at the same rate which is normal. For "chatterers" it may be possible to alter the rate of chatter with a device command. + +As far as I can think, this will apply to any device connected to any simple serial "bus" RS232, Ethernet, MODBUS etc. There is a point to be made that between two bits of code on a local system, polling is generally a poor idea (software should use events/interrupts). When talking to real hardware without its own interrupt lines, it is essential. + +The network load of polling is generally very low for real RS232 speeds even at 115Kb (the CPU load of over-polling serial calls might be higher though). On the Ethernet, saturation of an Ethernet link is slightly possible, but as long as the device is fairly dumb and one waits for a reply, the chances of a very tight loop are quite low (I would always put in a delay though). + +Just some thoughts as you continue into the design of your IOC. diff --git a/doc/iocs/tips_tricks/Stream-Device-Tips-and-Tricks.md b/doc/iocs/tips_tricks/Stream-Device-Tips-and-Tricks.md new file mode 100644 index 000000000..b54d31a1e --- /dev/null +++ b/doc/iocs/tips_tricks/Stream-Device-Tips-and-Tricks.md @@ -0,0 +1,148 @@ +# Stream Device Tips and Tricks + +## Waveforms + +With any waveform record protocol, you need to specify a separator to split the input or output string. For example, to split on commas use `separator = ",";` within the protocol function for the record. See https://paulscherrerinstitute.github.io/StreamDevice/tipsandtricks.html + +### Reading in a string into a waveform of strings + +If you have a waveform record of `FTVL` `STRING`, then you have to be careful about which formatter you use to read in that string. + +Instead of using `%s` as your formatter, you need to use a character set, e.g. `%[A-Z0-9a-z.+-]`, which does not include the separator you are splitting on. If you use `%s`, then the whole string will be read into the zeroth element of the array, and won't be split on the separator. + +### Dealing with enums + +The record type for enums in EPICS is mbbo/mbbi. However, StreamDevice also provides [support for enums within the protocol file](https://paulscherrerinstitute.github.io/StreamDevice/formats.html#enum). + +The way that the `mbbo` works is that it takes a user set value into the `VAL` field and uses it to lookup a value to output based on `ZRVL`, `ONVL` etc. this is outputted to the `RVAL` field. The value in `RVAL` is then passed to the protocol file, which could also have an enum specified. This effectively means that you have two enums stacked on top of each other and is BAD. Not only does it reduce readability it has historically caused issues when StreamDevice was modified to take into account `RVAL`, see [here](https://github.com/ISISComputingGroup/IBEX/issues/5263). It is up to the developer whether the enum should be defined in the protocol or the record but you should only use one. + +On the record side the `mbbi` works in the same way as the `mbbo` but in reverse. However, the behaviour for an `mbbi` in StreamDevice is not symmetrical with `mbbo`, which you can see by comparing [here](https://paulscherrerinstitute.github.io/StreamDevice/mbbi.html) and [here](https://paulscherrerinstitute.github.io/StreamDevice/mbbo.html) under the ENUM behaviour. This effectively means the enum defined in `mbbi` is ignored in StreamDevice so you are forced to use the protocol for defining this. + +## Multi-value Protocols + +### Dealing with multi-value stream device protocols + +It often happens that a single read query to a device returns multiple values, which we ultimately need to store in separate PVs. Similarly, sometimes a single write command requires multiple values. Below are some tips on how to deal with multi-value read protocols, but the same kind of tricks can be applied to write protocols too. + +If all the values are the same format, you could read them into a `waveform` or `aai` record using the `separator` as described above, and then split them or manipulate after than e.g. `acalcout`, `calcout` or `scalcout` + +### Passing the PV names as protocol parameters + +Here is an example of record redirection, where a single status read query returns three different values, which correspond to three different PVs. The first PV triggers the protocol, and in doing so passes the name of the second and third PVs as arguments to the protocol: + +``` +## Read the status value 1. This also populates the status value 2 and 3. + +record(ai, "$(P)VALUE1") { + field(DESC, "Status value 1") + field(DTYP, "stream") + field(INP, "@ls336.proto getValues($(P)VALUE2,$(P)VALUE3) $(PORT)") + field(SCAN, "$(SCAN) second") +} + +## Read the status value 2. +record(bi, "$(P)VALUE2") { + field(DESC, "Status value 2") + field(ZNAM, "Off") + field(ONAM, "On") +} + +## Read the status value 3. +record(bi, "$(P)VALUE3") { + field(DESC, "Status value 3") + field(ZNAM, "OK") + field(ONAM, "Error") +} +``` + +The corresponding protocol would be: + +``` +getValues { + out "VALUES?"; + in "%f,%(\$1)d,%(\$2)d"; +} +``` + +where `\$1` and `\$2` indicate the first and second argument, respectively. The protocol will redirect the second and third value to the PVs specified. + +As our PV prefix `$(P)` tends to be quite long, often the `INP` field calling the protocol ends up being longer than the max number of allowed characters. We can shorten the `INP` field by passing the prefix as a separate argument (and remove blank spaces between arguments!): + +``` +... +field(INP, "@ls336.proto getValues($(P),VALUE2,VALUE3) $(PORT)") +``` + +and the prefix can be pre-pended to the PV names inside the protocol: + +``` +... +in "%f,%(\$1\$2)d,%(\$1\$3)d";} +``` + +You need to keep in mind that PVs that are using this method can not be tested in RecSim. This is because with this method only one PV has a SCAN and INP field, while the other updated records do not have these fields. Therefore, they can not be updated unless the protocol method is called. But in RecSim the protocol is not used at all, so these PVs can not be set in RecSim. + +If the protocol returns many values, even this approach may result in too long `INP` fields. One solution would be to pass the prefix `$(P)` as an argument and hard-code the rest of the PV names inside the protocol, but we try to avoid this as much as possible as it introduces extra coupling between the protocol and the db file. Read on for other tricks! + +### Reading into the inputs of a calc record + +If you can't fit all the PV names in a single INP field, you can redirect them to the input fields of a `calc` record instead. We need a trigger PV which calls the protocol, and a buffer `calc` record to temporarily store the parsed values. The final PVs can then fetch their values from the `calc` record: + +``` +## Trigger the read protocol. Parsed values are stored in a buffer calc PV. +record(bi, "$(P)READ_DO") { + field(DTYP, "stream") + field(INP, "@ls336.proto getValues($(P)BUFFER) $(PORT)") + field(SCAN, "$(SCAN) second") +} + +## Store the parsed values +record(calc, "$(P)BUFFER") { + field(CALC, "0") +} + +## +## Read the status value 1. +## +record(ai, "$(P)VALUE1") { + field(INP, "$(P)BUFFER.A CP") + ... +} + +## +## Read the status value 2. +## +record(bi, "$(P)VALUE2") { + field(INP, "$(P)BUFFER.B CP") + ... +} + +## +## Read the status value 3. +## +record(bi, "$(P)VALUE3") { + field(INP, "$(P)BUFFER.C CP") + ... +} +``` + +and the protocol becomes: + +``` +... +in "%(\$1.A)f,%(\$1.B)d,%(\$1.C)d"; +``` + +If you have to read strings as well as numbers, you can always use a `scalcout` record instead of a `calc`. + +### ... and if everything fails + +Another way is to read the entire input as a string into a `string` record, and then use various `scalcout` or `asub` records to parse the individual bits and pieces. An example of this usage can be found in the Linkam95 IOC. + +### Other considerations + +The error on disconnection is not passed through from stream and so you may consider doing this via the error setter; see [IOC Utilities](#ioc_utils_error_setter) for details. + +Useful links: + +https://paulscherrerinstitute.github.io/StreamDevice/index.html \ No newline at end of file diff --git a/doc/iocs/tips_tricks/Using-IP-rather-than-nport-with-MOXA-(RFC-2217).md b/doc/iocs/tips_tricks/Using-IP-rather-than-nport-with-MOXA-(RFC-2217).md new file mode 100644 index 000000000..7350ff9f2 --- /dev/null +++ b/doc/iocs/tips_tricks/Using-IP-rather-than-nport-with-MOXA-(RFC-2217).md @@ -0,0 +1,13 @@ +# Using IP rather than NPORT (RFC 2217) + +NPort will create serial ports on the computer and these are accessed using `drvAsynSerialPortConfigure()` in the `st.cmd` + +An alternative is to directly talk IP to the moxa, which bypasses the NPort windows software. This option can be configured on a port by port basis, though the moxa will restart when changes are made. + +To enable this option you need to change the mode of the moxa port from "Real COM" (which uses windows NPort) to "RFC 2217". On the moxa web page look under "operational settings" for the port to change the mode. + +Port 1 of the MOXA is served at IP port 4001, so to connect with asyn you would use something like: +``` +drvAsynIPPortConfigure("L0", "130.246.49.42:4001 COM", 0, 0, 0, 0) +``` +The "COM" after the address:port indicates the RFC 2217 mode, this mode means that `asynSetOption()` will work as before and there is no need to set baud etc. via the moxa web page. \ No newline at end of file diff --git a/doc/iocs/tools/Asyn-Interpose-Functions.md b/doc/iocs/tools/Asyn-Interpose-Functions.md new file mode 100644 index 000000000..6e7406a49 --- /dev/null +++ b/doc/iocs/tools/Asyn-Interpose-Functions.md @@ -0,0 +1,17 @@ +# Asyn Interpose functions + +ASYN interpose functions can apply logic to data read from the device that is then passed to e.g. stream device. + +## Throttle Interpose + +Sometime a device has a maximum rate at which it can accept commands and/or that commands must be separated by a minimum delay. Though it is possible to add delays into Db files and/or stream device protocol, but this can get messy or things get missed. The throttle interpose function allows you to specify a minimum time between writes to the device, if another write occurs too quickly a sleep is inserted to bring up to minimum delay. + +syntax is +``` +asynInterposeThrottleConfig(port, address, min_delay_between_commands) +``` + +so you would add e.g. `asynInterposeThrottleConfig("L0", 0, 0.1)` to the `st.cmd` after port `L0` is created. + +To see when a delay is being inserted, enable the filter debug output as per [asyn trace masks](/iocs/troubleshooting/ASYN Trace Masks (Debugging IOC, ASYN)) so e.g. set trace mask to `0xD` rather than `0x9` + diff --git a/doc/iocs/tools/AsynEchoDriver.md b/doc/iocs/tools/AsynEchoDriver.md new file mode 100644 index 000000000..277653152 --- /dev/null +++ b/doc/iocs/tools/AsynEchoDriver.md @@ -0,0 +1,37 @@ +# Asyn Echo Driver + +Asyn provides an echo driver that will return on a read what was last provided by a write. This can +be used for some very basic stream device testing / simulation e.g. if the real function is + +``` +getFrequency { + out "FREQ?"; in "%f"; +} +``` + +Then by connecting the device to the echo driver you could write + +``` +Terminator = ETX; + +getFrequency { + out "10.0"; in "%f"; +} +``` + +Note, you must have a terminator otherwise the echo driver gets confused and won't work. + +Potentially you could have both a real `getFrequency()` and a dummy `getFrequencyTest()` function in the protocol file and then control with one was used by writing `getFrequency$(TEST=)` in the DB file and setting `TEST=Test` with `dbLoadRecords() ` + +To enable support for the echo driver in a Makefile: +* Add **`asynEcho`** to $(APPNAME)_LIBS +* Add **`drvAsynEcho.dbd`** to $(APPNAME)_DBD + +And then in st.cmd use e.g.: + +``` +echoDriverInit(“L0”, 0.1, 0, 0) +``` + +Where L0 is the asyn port name and 0.1 is, in this case, how long to take to return a reply (**this must be greater than zero**). The other options allow selecting `noautoconnect` or multi-device if required. + diff --git a/doc/iocs/tools/Autosave.md b/doc/iocs/tools/Autosave.md new file mode 100644 index 000000000..9a532560e --- /dev/null +++ b/doc/iocs/tools/Autosave.md @@ -0,0 +1,30 @@ +# Autosave + +## Purpose + +The purpose of autosave is to capture tunable parameters in a config-independent way. Examples of things we autosave are things like motor settings (resolutions, speeds, etc.). + +## Snapshots + +In the future, we may use an autosave-like mechanism to apply snapshots of motor settings at a particular time to an IOC again. However this is not currently implemented nor required on any beamline that we support. + +## Dangers + +Mixing autosave with IOCs that load a dynamic number of axes/sensors can be dangerous. If the IOC is accidentally started with a macro that makes the IOC load no axes, autosave will save a file containing no values. When the IOC is subsequently restarted with the correct macros, it will read the autosave file (which has no values in it). This causes the IOC to "lose" its previously saved values. This was seen in https://github.com/ISISComputingGroup/IBEX/issues/2180. + +## How to Add + +To add autosave to a field see [autosave PVs in finishing touches](#ioc_finishing_touches_autosave_pvs). + +## Forcing autosave to save at a given point (useful for testing) + +You can force autosave to save at a given point by writing a command into the iocsh. The command: `manual_save(".req")`. + +You can find `".req"` from the ioc startup logs `create_monitor_set(".req", ...)`. + +If you wish to do this in the IocTestFramework you can with the ProcServLauncher. First you need the ioc to access: `self._lewis, self._ioc = get_running_lewis_and_ioc("", DEVICE_PREFIX)` and then you can run `self._ioc.telnet.write("manual_save(\"KEPCO_01_info_settings.req\")\n")`. + +## Troubleshooting + +See [here](/iocs/troubleshooting/IOC-And-Device-Trouble-Shooting) + diff --git a/doc/iocs/tools/Convert-Record.md b/doc/iocs/tools/Convert-Record.md new file mode 100644 index 000000000..f9b602027 --- /dev/null +++ b/doc/iocs/tools/Convert-Record.md @@ -0,0 +1,29 @@ +# Convert record + +The convert record is used to convert values when you have a lookup table or formulae. The details for the record can be found at http://www-csr.bessy.de/control/SoftDist/csm/cvtRecord.html. These can be a little sketchy so these are some extra notes. + +I (John) used the record in the barndoors support modules (..\EPICS\support\barndoors\master), Kathryn has used it on the Eurotherm. + +To use this record in an IOC you must include in the build.mak file the following (both in support and the ioc): + +1. `$(APPNAME)_DBD += cvtRecord.dbd` in the dbd section +1. `$(APPNAME)_LIBS += cvtRecord csmbase` in the libraries section +1. `CSM=$(SUPPORT)/csm/master` in the RELEASE file in configuration directory + +If you are using a calibration file both the base path (`TDIR` field) and the filename (`SPEC` field) must be less than 40 characters; If you really need an extra 40 characters the `BDIR` field can also be used. You should use the standard name for these macros of `CALIB_PATH` and `CALIB_FILE` (see (Macro-Naming)). The suggested location for calibration files is `C:\Instrument\Settings\calib\`. Place an example in your IOC that can be copied in settings. + +The convert records will load calibration data when it is initialised. To reload the calibration data use: + + caput %MYPVPREFIX%.INIT 1 + +## Spline hack + +A hack is being introduced into `csmbase` and the convert record to allow developers to use bespoke conversion routines along with (1D) lookup tables. The way this is currently implemented is to write a conversion function (in C or C++) in the support directory and name it `User1DTableSub`. It must be named this due to a limitation in the convert record wherein a developer cannot specify _both_ a user defined subroutine name _and_ and a lookup table. Hence the way that the function is located by the convert record is to search the epics function registry for a function called `User1DTableSub`. + +This is currently only in use with the Keithley 2700 as used for the HIFI Cryomag. The user defined subroutine uses the `gsl` library's cubic spline implementation to interpolate from a resistance measurement provided by a 4 wire carbon RTD to a temperature in Kelvin (in the range 0-200 or so). + +## Tips, Tricks and Gotchas + +1. The record will linearly interpolate the value this includes at either end of the record so make sure you set `DRVH` and `DRVL` if you want to avoid this. +1. Make sure your calibration (in the way in the direction you are using it) has only one value per input otherwise it is not always clear what the answer will be. For example in the barn doors I have two calibration file one represents the gap for a given response (this includes inwards and outwards swing which have the same gap measurement) and one file which is just inward swing for when I am converting a gap to a motor movement. +1. The spline hack uses gsl, and the spline interpolation implementation allocated memory for the fit and an accelerator. This should normally be freed, but in the spline hack implementation is it not because of the way that an evaluation is called. If the convert record (when using a spline hack) develops a memory leak, this could be why. diff --git a/doc/iocs/tools/Creating-a-State-Machine-(Sequencer).md b/doc/iocs/tools/Creating-a-State-Machine-(Sequencer).md new file mode 100644 index 000000000..b9c675e72 --- /dev/null +++ b/doc/iocs/tools/Creating-a-State-Machine-(Sequencer).md @@ -0,0 +1,125 @@ +# Creating a State Machine (Sequencer) + +## Contents +- [Introduction](#sequencer_intro) +- [Create State File](#sequencer_snl_file) + - [Call State file from IOC](#sequencer_call_snl_file) +- [Passing macros into the sequencer](#sequencer_macros) + +{#sequencer_intro} +## Introduction +Does your IOC require definitive states? If the logic of your IOC will go beyond calc fields, you may wish to implement a Finite State Machine. + +EPICS supports State Machine creation with a set of [Sequencer tools.](https://epics-modules.github.io/sequencer/Manual.html) The state file, which controls the states the system can be in and the transitions between them, is written in C-based State Notation Language. +This is a how-to guide to create and implement a state machine into your IOC. + +{#sequencer_snl_file} +## Create State File + +Enter the support directory of your IOC, e.g.: +``` +C:\Instrument\Apps\EPICS\support\HFMAGPSU\master\HFMAGPSUSup +``` +Create a state file in this directory (e.g. `fsm.st`). +The EPICS manual for SNL files can be found [here.](https://epics-modules.github.io/sequencer/Manual.html) + +If you are reading or changing PV values, the following must be included in your `fsm.st` file: +``` +#include "ibexSeqPVmacros.h" +``` +You may also need to add a line to your `configure/RELEASE` file to allow inclusion of these PV utility macros: +``` +UTILITIES=$(SUPPORT)/utilities/master +``` + +Create a `fsm.dbd` file, which contains the following: +``` +registrar(fsmRegistrar) +``` +Make sure that your `.dbd` file has a different name to the IOC `.dbd` file. Otherwise, your IOC `.dbd` file will be overwritten. + +Edit the `Makefile` in this folder, to reference your new `.dbd` and `.st` files and add the libraries needed for them. +``` +TOP=.. +include $(TOP)/configure/CONFIG +#======================================= + +# Install .dbd and .db files +DATA += devHFMAGPSU.proto +DBD += fsm.dbd + +# Sequence file +HFMAGPSU_SRCS += fsm.st +LIBRARY_IOC = HFMAGPSU +HFMAGPSU_LIBS += seq pv +HFMAGPSU_LIBS += $(EPICS_BASE_IOC_LIBS) + +#======================================= +include $(TOP)/configure/RULES +``` + +Be sure to include the library location for seq in the support `RELEASE` file. +``` +# Macros required for basic ioc/stream device +SNCSEQ=$(SUPPORT)/seq/master +``` + +{#sequencer_call_snl_file} +### Call State file from IOC +Enter IOC folder, e.g. +``` +(cd …EPICS\ioc\master\HFMAGPSU\iocBoot\iocHFMAGPSU-IOC-01) +``` +Add the following to the end of the `st-common.cmd` file: +``` +< $(IOCSTARTUP)/postiocinit.cmd +epicsEnvSet(P,$(MYPVPREFIX)$(IOCNAME)) + +## Start any sequence programs +seq fsm, "P=$(P)" +``` +In the above example, we are passing the name of the IOC through to the `fsm.st` file so we can reference PVs in it with `{P}:`. + +Enter the IOC source folder (e.g. ``HFMAGPSU-IOC-01App\src``) + +Edit `build.mak` to ensure the needed libraries are included: +``` +# Add all the support libraries needed by this IOC +## ISIS standard libraries ## + +$(APPNAME)_LIBS += seq pv +``` +### Notes + +Whenever you make an edit to the `fsm.st` file, rebuild the support module: +``` +cd …EPICS\support\HFMAGPSU\master +make +``` + +{#sequencer_macros} +## Passing macros into the sequencer + +You can pass macros into your sequencer by including them in brackets after you define the program. E.g. + +``` +program keithley_2001 ("P, channels") +``` + +Here `P` and `channels` are passed as macros to the state machine. You pass these to the state machine in the `st.cmd` file when calling the state machine. E.g. + +``` +seq keithley_2001, "P=$(MYPVPREFIX)$(IOCNAME):, channels=10" +``` + +To access these macros in your state machine, create a variable to hold your macro, e.g. `char *P` for the macro `P`, and call `macValueGet` on your macro, e.g. `P = macValueGet("P")`, within your state machine to allow you to use the macro "P" within your code as the variable "P". + +Note that the macros you pass into the state machine **must** match up with those in your `.db` files. Otherwise your state machine will not be able to assign variables to PVs and the state machine won't run. + +## Using `epicsThreadSleep` + +From https://www-csr.bessy.de/control/SoftDist/sequencer/Tutorial.html#common-pitfalls-and-misconceptions + +*If your action statements have any sort of polling loops or calls to `epicsThreadSleep` you should reconsider your design. The presence of such operations is a strong indication that you’re not using the sequencer as intended.* + +Long sleeps will hang the thread and then other things may happen. An example was an SNL program to check that a setpoint had been actioned by waiting 30 seconds and then comparing setpoint and readback. While it was waiting, the setpoint may change again, and the wait is now redundant, and if it doesn't check for a change in original setpoint it may do the wrong thing. Using delay() is better as that does not block the thread and allows other checks in the state to continue. \ No newline at end of file diff --git a/doc/iocs/tools/IOC-Utilities.md b/doc/iocs/tools/IOC-Utilities.md new file mode 100644 index 000000000..cf20f425c --- /dev/null +++ b/doc/iocs/tools/IOC-Utilities.md @@ -0,0 +1,190 @@ +# IOC Utilities + +The utilities comprise of useful IOC db templates and IOC shell utilities. + +{#ioc_utils_templates} +## DB Templates + +Below are available utility templates for [substitution](Template-Substitutions). In order to allow Make to build these, you need to add the following to your `Device/master/configure/RELEASE` file: `UTILITIES=$(SUPPORT)/utilities/master`. + +### Field setter +This copies the specified field from a PV and sets it on a different PV. E.g. + + file $(UTILITIES)/db/field_setter.template { + pattern + {P, FROM, TO, FIELD} + + {"\$(P)", "UNITS", "READING", "EGU"} + {"\$(P)", "UNITS", "SP", "EGU"} + + } + +This example copies the `EGU` field from `$(P)READING` to `$(P)UNITS.EGU` and `$(P)SP.EGU`. + +{#ioc_utils_unit_setter} +### Unit setter + +This copies units from a PV and sets them on a different PV. E.g. + + file $(UTILITIES)/db/unit_setter.template { + pattern + {P, FROM, TO} + + {"\$(P)", "UNITS", "READING"} + {"\$(P)", "UNITS", "SP"} + + } + +This will copy units from the value of the PV `$(P)UNITS` to `$(P)READING` and `$(P)SP`. + +{#ioc_utils_error_setter} +### Error Setting + +Creates a raw PV that can be written to by a stream protocol and then transfer the stream protocols PV error and value to the real PV. + +For example: + + file $(UTILITIES)/db/error_setter.template { + pattern {P, STREAM_PV, PV_NAME} + + {"\$(P)", "FREQ:REF", "FREQ:SP:RBV"} + {"\$(P)", "FREQ:REF", "FREQ"} + + } + +In this example, the PV `FREQ:REF` reads the values from a status and then set the values, via the protocol file, in the `FREQ:SP:RBV:RAW.A` PV. This value and any error that occurs in the `FREQ:REF` is then set on the `FREQ:SP:RBV` PV. This allows you to easily show a disconnected error in PVs that are set from the protocol file. + +If you need to use the error setter for PVs that are defined in a .db file instead of .template file, the .substitutions file for that .db file needs to have a different name than the .db file, otherwise the error setter will not work. + +### Calibration Range + +Calculates the maximum and minimum values of a selected calibration file. You can load this db with macros using +``` +dbLoadRecords("$(UTILITIES)/db/calibration_range.db","P=$(P),BDIR=TEMP.BDIR,TDIR=TEMP.TDIR,SPEC=TEMP.SPEC,HIGH_PV=TEMP:RANGE:OVER.B,LOW_PV=TEMP:RANGE:UNDER.B") +``` +Where `TEMP` is a `cvt` record which uses the calibration file. The max value is outputted to `HIGH_PV` and the minimum to `LOW_PV`. + +### check stability + +This is a generic utility for verifying that a value has been within tolerance of a setpoint and without any invalid alarms for N samples. + +The implementation is defined in `$(UTILITIES)/db/check_stability.db`. + +First load the DB records, e.g. + +``` +dbLoadRecords("$(UTILITIES)/db/check_stability.db", "P=$(MYPVPREFIX)$(IOCNAME):,INP_VAL=$(MYPVPREFIX)$(IOCNAME):FIELD,SP=$(MYPVPREFIX)$(IOCNAME):FIELD:SP:RBV,NSAMP=5,TOLERANCE=$(TOLERANCE=0)") +``` + +And then, from an existing DB, forward link to `$(P)STAB:SCANNOW` when it should take new samples (e.g. Forward-link from the readback PV). + +Whether the value is stable or not is then published in `$(P)STAB:IS_STABLE` - 1 if stable, 0 otherwise. + +## Shell Utilities + +There are some IOC shell utilities defined in `C:\Instrument\Apps\EPICS\support\utilities` which can be used in an IOC shell to help startup IOCs. The doxygen docs are here http://epics.isis.rl.ac.uk/doxygen/main/support/utilities/. + +{#pausing_an_ioc_at_startup} +### Pausing an IOC at startup + +You can pause an IOC at startup in the st.cmd using `msgBox`. This is imported in `IOC_NAME_registerRecordDeviceDriver pdbbase` so must come after that line. + +Usage: `msgBox "title" "text"`. Bring up a message box at the point in your startup that it is placed and pause the boot until you click the button. + +This could be useful if you want to run a debugger on the IOC, which you attach whilst the IOC boot is paused. + +## calc + +Performs an arithmetic operation on an expression and return the integer value to a specific environment variable: + +`calc("ENV1","1+1",1,2)` + +The arguments are as follows: + +1. The output environment variable +2. The expression to be evaluated +3. Options +4. The output length + +My best guess is that the expression is evaluated using the `calcPerform` method from the EPICS standard library so the expression should match the syntax as used in a calc record. + +The options are detailed in `ioccalc.cpp` in the utilities directory. + +Examples can be seen in the Galil and McLennan motor records. + +## dcalc + +As calc, but returns a double value. The 4th argument is for the number of decimal places, not the value length. + +`dcalc("ENV1","0.1*0.2",1,2)` + +An example can be seen in the Eurotherm IOC, file `st-timing.cmd`. + +## stringtest + +TODO + +## `stringiftest(resultvar, lhs, operation, rhs)` + +Defines an environment variables as empty or a comment depending on if lhs is empty. The variables defined are: +- `IF` '#' if empty; otherwise ' ' +- `IFNOT` ' ' if empty; otherwise '#' + +parameters + +1. `resultvar` - the basename of the environment variable to set +1. `lhs` - the string to test +1. `operation` - set the first bit for verbose mode +1. `rhs` - used in `lhs==rhs` operation + +The operation argument is given as a decimal representation of binary flags: + +| Operation | Flag | +|:---------------:|:-----:| +| Verbose | `0x1` | +| length > 0 | `0x2` (default) | +| lhs == rhs | `0x4` | +| Inverse output | `0x8` | + +To get the operation that you require, add the flag value in decimal. For example, to check if a string has finite length, your operation would be `2`. However, if you would like your flag to check if a string has a zero length, then add the inverse flag value `8`, meaning your operation would be `10 (8+2)`. To add a log for this operation, add the verbose flag of value `1`, so the total value is `11`. + +For debugging purposes it is advisable to add the verbose/logging flag of value `1` to your operation. + +### Examples +A simple use case might be to only run certain sections of `st.cmd` for a serial port (`PORT` macro defined) and others only if `IPADDR` macro is defined. So we would use lines like: +``` +stringiftest("SERIAL", "$(PORT=)") +stringiftest("IPADDR", "$(IPADDR=)") +$(IFIPADDR) drvAsynIPPortConfigure("$(DEVICE)", "$(IPADDR):12345") +$(IFSERIAL) drvAsynSerialPortConfigure("$(DEVICE)", "$(PORT)", 0, 1, 0, 0) +``` + +From the DKFPS IOC: +``` +stringiftest("POLAR" "$(POLARITY="BIPOLAR")" 5 "BIPOLAR") +``` +The operation value is `5`, or `4+1`, so this checks the lhs (`$(POLARITY)`, which defaults to `"BIPOLAR"`) equals the right hand side `"BIPOLAR"`, and puts the result in the `$(POLAR)` also creating `$(IFPOLAR)` and `$(IFNOTPOLAR)` with appropriate space or `#` character. So if you wanted to execute some lines of `st.cmd` if `POLARITY` equals `BIPOLAR` and some if it wasn't you would type +``` +$(IFPOLAR) do something +$(IFNOTPOLAR) do something else +``` + +You'll see this used a lot in the form `$(IFDEVSIM)` and `$(IFNOTDEVSIM)` to either connect to the real hardware or an emulator + +Search the `EPICS-ioc` repository for `stringiftest` to see many other examples of usage + +## setIOCName + +TODO + +## getIOCName + +TODO + +## getIOCGroup + +TODO + +## mkdir + +TODO \ No newline at end of file diff --git a/doc/iocs/tools/IOC-access-security.md b/doc/iocs/tools/IOC-access-security.md new file mode 100644 index 000000000..af6e8b488 --- /dev/null +++ b/doc/iocs/tools/IOC-access-security.md @@ -0,0 +1,47 @@ +# IOC Access Security + +## What it is for + +Access security, alternatively known as "manager mode", is used to prevent users from writing values to an IOC while letting instrument scientists assign values. + +## How to set it up + +Here are the basic steps to follow: + +- For each record in the IOC which should be protected, add an `ASG` field +- For instrument specific DB records, it is ok to hard-code the ASG (for example, `field(ASG, "MANAGER")`) +- For general IOCs, the access security group should be defined as a macro (e.g. `field(ASG, "$(ASG=DEFAULT)")`). Giving the `ASG` macro a default value of `DEFAULT` will mean that existing instruments using this IOC will be unaffected. +- The `ASG` macro can then be supplied from an `st.cmd`, or a `globals.txt` as appropriate. The two useful values for the ASG macro for setting up access security are `DEFAULT` (acts as normal) and `MANAGER` (with access security). +- OPIs that might be protected by access security should have the access security widget in them. This can be copy-pasted from the template OPI. +- The access security widget in the OPI depends on a PV provided from the IOC in the following way: +``` +record(scalcout, "$(P)MANAGERMODE") +{ + field(ASG, "READONLY") + field(DESC, "Non-zero if manager is required for this IOC") + field(PINI, "YES") + field(INPA, "$(PVPREFIX)CS:MANAGER CP") + field(BB, "$(ASG)") + field(CALC, "A = 0 && BB = 'MANAGER'") + field(OOPT, "Every Time") +} +``` +- Note that you will need to pass in `$(PVPREFIX)` from your `st.cmd` for this to work! + +_(If you need a "real world" example, the jaw set on Polaris has been set up using access security.)_ + +## Further information + +The access security rule is defined in `C:\Instrument\Apps\EPICS\support\AccessSecurity\master\default.acf`. + +This looks at a PV which is `$(P)CS:MANAGER`. If the PV is set to 1 then manager mode is enabled, if it is 0 then manager mode is disabled. The GUI simply sets this PV to 1 or 0 as you enter or exit manager mode. + +To check whether access security is working properly it can be useful to use a `cainfo `, this tells you whether you currently have read, write, or both permissions. + +## Even more information + +- [Access Gateway](/system_components/Gateway) + +## Past approaches / future extensions + +In future it may be desirable to implement a "baton" system which allows users to request exclusive access to an instrument. There was a small amount of front-end code related to this removed in [this commit](https://github.com/ISISComputingGroup/ibex_gui/pull/615/commits/f3ea01d0cb4d192d5b6f22990540718c650bb8c2). The back end for this was never implemented. \ No newline at end of file diff --git a/doc/iocs/tools/ISIS-modules-for-file-handling.md b/doc/iocs/tools/ISIS-modules-for-file-handling.md new file mode 100644 index 000000000..d7d90bca4 --- /dev/null +++ b/doc/iocs/tools/ISIS-modules-for-file-handling.md @@ -0,0 +1,51 @@ +# ISIS modules for file handling + +There are currently two ISIS modules that may help select and read files via PVs. + +## FileList + +This module will take a base directory and PCRE expression. It will return a compressed JSON list of the files that match the expression within that directory (non-recursively). + +Currently, the constructor for the module must be given the starting PCRE and directory. This method was chosen, rather than defaults being set via PVs, as the directory must be set prior to the PCRE. + +There are three FileList variables that can be connected to PVs: + +- DIRBASE - Gives the base directory to search +- SEARCH - Gives the PCRE expression to search for +- JARR - Gives the JSON compressed list of valid files +To connect the list of files to an OPI there is a common script called UpdateFileList.py which should be attached to a combo control. For an example of this see the Kepco OPI. + +There is also a db file for selecting calibration files for a record, which is used in the Eurotherm and the Kepco. + +For examples of this modules use see the test IOC within the FileList folder or see the DAE IOC where this is used to select wiring, detector and spectra tables. + +## ReadASCII + +This module has two main functions: +* Ramp a set point linearly from it's current value to a new one. This can be controlled through a ramp rate (in EGU/min) and a step rate (in steps/min) +* Change a selection of other PVs when a set point changes. This is done by opening a file at a given location and using the table within it to change PVs. The module uses one PV as a lookup on the first column of the table and subsequently changes other PVs to match the values in the other columns. + +These functions can be turned off and on independently but can be used in conjunction to ramp the lookup PV and change other PVs when it crosses a threshold of the table. Both functions are used in the Eurotherm IOC to change PID and MaxHeater values when the temperature SP is changed. The Kepco IOC only uses the ramping functionality. + +To configure ReadASCII you must have the following in your `*.cmd`: +``` +ReadASCIIConfigure(port_name, PID_folder, steps_per_minute, quiet_on_SP) +``` +where: +* `port_name` is a string of the port name, used to refer to ReadASCII in the db +* `PID_folder` is the initial folder used to search for PID lookup files (can be blank if just using ramping) +* `steps_per_minute` is the number of steps the ramp will take per minute (note that the step size is dependant on the rate set at runtime) +* `quiet_on_SP` *optional* can be set to 1 to stop ReadASCII logging on every new set point. This can be useful if you think the set point may end up being changed rapidly e.g. the zero field controller. Defaults to 0 i.e. logging on every set point + +### Behaviour +Upon startup, ReadASCII will use lookup file to create asynPortDriver parameters and associate them with +settings table values created also from lookup value. +IMPORTANT - for ReadASCII to work correctly, the column names (headers) in lookup file must match PV names from .db file. +For example given field(INP, "@asyn($(READ),0,1)MH") the correct header in lookup file should be 'MH'. +.db file can be usually found in master/db folder, otherwise no PV will be read/written to. +ReadASCII has many checks written into it to prevent crashing if lookup file is incorrectly formatted or other unexpected situations occur and allows continued use without crashing, however there are currently no warnings displayed to the user - only errors printed in logs. + +### Extending +It is possible to extend functionality of ReadASCII to use more parameters than currently used (1 setpoint and 4 values - P, I, D, Max). +If the lookup file contains more columns with proper header names then corresponding PVs will be written to using lookup value table. +There can be only one setpoint column however. Other parameters, such as LUTON (Check if reading lookup file is on) or RAMPON (Check if ramping is on) are currently hard-coded. \ No newline at end of file diff --git a/doc/iocs/tools/Labview-Remote.md b/doc/iocs/tools/Labview-Remote.md new file mode 100644 index 000000000..500bf66ff --- /dev/null +++ b/doc/iocs/tools/Labview-Remote.md @@ -0,0 +1,11 @@ +# LabVIEW Remote + +IOC to allow remote connection LabVIEW VI's via ISIS Remote LabVIEW services. + +## Templates +There are pre-existing templates that can be implemented via substitution files for doubles, buttons, strings, and enums. +Once a DB file has been generated, create a file called `st-.cmd` in the devices folder of the ioc that loads the db, and set the `DEVCMD1` macro to the name of this file to load it. + +## Ports +This IOC requires two ports for full functionality, if you are only interacting with string values then it only requires TCP port 64008. if it only uses binary (i.e. doubles or booleans) then it only requires 64009, Enums and combinations of both require both ports be opened however. + diff --git a/doc/iocs/tools/Lua.md b/doc/iocs/tools/Lua.md new file mode 100644 index 000000000..6d8faa65f --- /dev/null +++ b/doc/iocs/tools/Lua.md @@ -0,0 +1,134 @@ +# Lua + +I've added the EPICS Lua support module to our build system. Lua is a scripting language +designed to be embedded, it has a small footprint and is reasonably powerful. It would +provide an alternative to jumping through hoops in st.cmd syntax but also provides a +few other options. There are examples of a Lua script used in the DETADC, Attocube and OERCONE IOCs. + +## To add the Lua support module to an IOC + +``` +add LUA=$(SUPPORT)/lua/master to configure/RELEASE +add luaSupport.dbd to the IOC Makefile DBD list +add lua and asyn to the IOC Makefile _LIBS list +``` + +## How to use it + +All `iocsh` commands are imported into Lua and so you can do things +like: + +``` +for index=1,10,1 +do + print(string.format("Loading instance: %d", index)) + iocsh.dbLoadRecords("test.db", string.format("P=xxx:,Q=%d", index)) +end +``` + +You execute files from st.cmd using: + +``` +epicsEnvSet("LUA_PATH", "${UTILITIES}/lua") +epicsEnvSet("LUA_SCRIPT_PATH","${TOP}/iocBoot/${IOC}") +luash("file.lua") +``` + +or just typing `luash` puts you into an interactive Lua shell. + +The Lua script record is like a calcout record but can execute Lua script. It might be +an alternative to e.g. aSub records for parsing stream device strings when writing C is +a bit overkill. + +As well as being able to read/write PVs there is also some asyn integration into Lua, +so you can read/write/set asyn parameters +from Lua command line or script record, or even talk to a device by creating an asyn IP port +and sending strings. See the documentation directory in Lua support module and the +example scripts directory in iocBoot + + +There is a powerpoint about Lua here: https://indico.cern.ch/event/766611/contributions/3438291/attachments/1856812/3050126/Lang-Lua_Integrating_Scripting_Language.pdf + +See also the documentation on our [epics-lua module](https://github.com/ISISComputingGroup/EPICS-lua) or the actual [epics module](https://github.com/epics-modules/lua) for more information on using Lua in EPICS. + +## Importing Lua Functions from Other Files + +When importing functions from other files you must be very careful not to pollute your scope as by default anything declared in lua is in the global scope. This means that if I have a file `importable_script.lua` of + +``` +function my_func() + print("Hello world") +``` + +and I import this file from elsewhere using: + +``` +require "importable_script.lua" +my_func() +``` + +I can call `my_func` as it will be in the global variables of my new script. This means the require statement is polluting my namespace, to get around this we can change `importable_script.lua` to read: + +``` +local available_functions = {} + +local function available_functions.my_func() + print("Hello world") + +return available_functions +``` + +when I do the import I can do: + +``` +my_import = require "importable_script.lua" +my_import.my_func() +``` + +now the only think in my global namespace will be `my_import`, which I specifically put in there, which contains all the functions I've imported for later reference. + +We have a few Lua utility functions available in our utilities submodule. For usage and how to add to them see [this page](lua/Our-Lua-Utility-Functions). + +## Style Guide + +We are using the style guide from LuaRocks as documented in https://github.com/luarocks/lua-style-guide#conditional-expressions + +## LuaCheck + +For installation, usage and troubleshooting see the [luacheck page](lua/LuaCheck) + +## Issues with Epics Lua and some answers + +Whilst during the conversion of the oercone IOC to Lua we came across a few issues with the epics Lua library. + +The process for resolving names in epics Lua starts with looking for a variable with that name, if it does not find one it looks in the running EPICS environment and then if it cannot find anything there it looks for a matching function name. + +This is great so that we can access macros easily by instead of doing `$(MYMACRO)` from cmd we can just do `MYMACRO`. However, we cannot stipulate a default. There is a Lua utility function we have written, `getMacroValue(options)`, which expects to be called like this `getMacroValue{macro="MYMACRO", default="VAL"}` or `getMacroValue{macro="MYMACRO"}`. + +The way that EPICs calls lua scripts causes local variables to fall out of scope between lines such that a script where you have written +``` +local text = "Hello" +print(text) +``` +will print `nil` as `text` has not been defined but +``` +local text = "Hello";print(text) +``` +will successfully print `Hello`. + +Fortunately, if we have a multiline chunk such as in a function we can use local variables. Thus to limit the amount of global variables we have to use we have decided to surround our lua code in functions. This means that: + +- Anything outside of a function must be declared as global for it to be used later on in the script. We think [this](https://epics-lua.readthedocs.io/en/latest/using-lua-shell.html#common-lua-environments) might `fix` the use of local variables outside of functions but it's effectively putting them in a global scope anyway. +- This includes the functions themselves, so we must take care to not give the same names to functions in the same iocBoot. +- I suggest if we have a main function for each lua file we give it the name `__main` e.g. for the oercone devices st.lua: `oercone_st_main` and it's st-common.lua: `oercone_stcommon_main`. + +You may see that a variable was once the value you expect and then changes to something like `func_meta: 008...`. This is because the variable name has dropped out of scope and epics Lua attempts to look for a macro and fails, so then looks for a function and returns the func_meta string. + +## More details + +```{toctree} +:glob: +:titlesonly: + +lua/* +``` \ No newline at end of file diff --git a/doc/iocs/tools/Serve-file-contents-over-EPICS.md b/doc/iocs/tools/Serve-file-contents-over-EPICS.md new file mode 100644 index 000000000..9c252f6a4 --- /dev/null +++ b/doc/iocs/tools/Serve-file-contents-over-EPICS.md @@ -0,0 +1,11 @@ +# Serve file contents over EPICS + +## FileContentsServer + +You can serve the raw contents of files in a directory over EPICS by using the "FileContentsServer" component of https://github.com/ISISComputingGroup/EPICS-FileServer + +This is done by calling `FileContentsServerConfigure()`, with an asyn port name as well as a directory path in an `st.cmd` alongside loading `FileContentsServer.db` with the same asyn port name. + +## Device screen + +There is an accompanying device screen("File Editor" in the list) for viewing and editing the file served by this mechanism. \ No newline at end of file diff --git a/doc/iocs/tools/Template-Substitutions.md b/doc/iocs/tools/Template-Substitutions.md new file mode 100644 index 000000000..18acfb01d --- /dev/null +++ b/doc/iocs/tools/Template-Substitutions.md @@ -0,0 +1,30 @@ +# Template Substitution + +Template substitution allows you to take a database template file and substitute the macros in it for a list of chosen values, the documentation for this can be found [here](https://epics.anl.gov/base/R7-0/1-docs/msi.html). This is used for example to create the same set of records for `North`, `East`, `South`, and `West` jaw blades. This helps to avoid writing a lot of duplicate PVs. + +This requires two files: a `.template` and a `.substitutions` file +- The `.substitutions` file loads the `.template` (or several) and runs a list of macro substitutions on it. This lives inside the IOC under `\\App\Db\.substitutions`. +- The `.template` file looks the same as a `.db`. It can live anywhere, you just need to specify the right path in the `.substitutions` file. + +To build a `.db` from this, you need to include a reference to the `.substitutions` file in the `makefile` in the same folder, e.g.: `DB += .db` (this instruction will look, in order, for a .db, then a .template, then a .substitutions file named ``) + +The general format of a substitutions file is as follows: +``` +file { + pattern + {MACRO_1, MACRO_2, (...) MACRO_N} + + {"MACRO_1_SUB_1", "MACRO_2_SUB_1", (...) "MACRO_N_SUB_1"} + {"MACRO_1_SUB_2", "MACRO_2_SUB_2", (...) "MACRO_N_SUB_2"} + (...) + {"MACRO_1_SUB_M", "MACRO_2_SUB_M", (...) "MACRO_N_SUB_M"} +} + +file { + ... +} +``` + +This will create M sets of records, with each of the N macros replaced with what is specified in each row. Note that you need to substitute every macro that occurs in the `.template` file, otherwise the substitution procedure will fail. If it's something you don't want to replace (e.g. the PV Prefix) you can just replace it with itself. + +See [IOC Utilities](#ioc_utils_templates) for some practical examples. diff --git a/doc/iocs/tools/Utilities-Library.md b/doc/iocs/tools/Utilities-Library.md new file mode 100644 index 000000000..8d04d1426 --- /dev/null +++ b/doc/iocs/tools/Utilities-Library.md @@ -0,0 +1,12 @@ +# Utilities Library + +The `utilities` support module contains some C/C++ functions useful to IOC and support module developers + +## putDbAndWait + +This is based on `dbtpn` from the epics base `dbNotify.c`. It allows setting a PV from within C/C++ and waiting for a completion callback, if you do not need such a callback then using `dbNameToAddr` and `dbPutField` is sufficient. +``` +putDbAndWait(const std::string& pvName, const void *value, double timeout) +``` +`value` is interpreted as appropriate for the data type of `pvName` + \ No newline at end of file diff --git a/doc/iocs/tools/aSub-records.md b/doc/iocs/tools/aSub-records.md new file mode 100644 index 000000000..85142130f --- /dev/null +++ b/doc/iocs/tools/aSub-records.md @@ -0,0 +1,343 @@ +# Asub records + +## Introduction + +EPICS aSub records are a powerful tool that allows you to write arbitrary C or C++ code that interacts with EPICS DB records. + +Circumstances where you may need to use aSub records include: +- A device responds with a very complex status string which is too complex to decode using streamdevice +- There are complex checks required before a command can be sent to a device, or complex logic required to interpret the state of a device. +- The device responds with completely different responses based on its physical configuration, and the IOC needs to handle all possible circumstances +- You need to interact with files from within an IOC (first consider whether [autosave](Autosave) or [ReadASCII](https://github.com/ISISComputingGroup/EPICS-ReadASCII) cover the functionality you need) + +### Examples +- `FZJ Fermi Chopper` (`EPICS/ioc/master/FERMCHOP`) + * This contains five scripts: Two to parse sets of status packets from a device, two to choose values to send to a device, and one to check that a command is allowed to be sent at the current time. +- `Keithley 2700` (`EPICS/support/Keithley_2700/master`) + * This contains a script to parse buffer readings from a device and put them into the appropriate DB records. +- `Keyence TM-3001P` + * This contains scripts to parse complex response strings from a device. +- ReadASCII metadata (`EPICS/support/ReadASCII/master`) + * This contains scripts to read metadata from a filepath provided in EPICS DB records. + +## IOC files + +### DB records +Define some aSub records which may look like: +``` +record(stringin, "$(P)STATUS:RAW") +{ + field(DESC, "Reads from the device") + field(DTYP, "stream") + field(INP, "...") + field(SCAN, "1 second") + field(FLNK, "$(P)STATUS:PARSE") +} + +record(aSub, "$(P)STATUS:PARSE") +{ + field(SNAM, "function_name") + + field(INPA, "$(P)STATUS:RAW") + field(FTA, "STRING") + + field(OUTA, "$(P)STATUS PP") + field(FTVA, "STRING") + +} + +record(stringin, "$(P)STATUS") +{ +} +``` + + +### Build files + +Create a `.dbd` file with the names of all the functions declared in the `C` file which will be called by EPICS. Your DBD file should look like the following: + +``` +function(function_name) +function(another_function_name) +[...] +``` + +>__**NOTE: Make sure there is a newline at the end of this file**__ + +In the same directory as your C source code and the DBD file, adjust the makefile to reference the newly created `.dbd` and `.c` files, as well as adding necessary dependencies. It should look something like the following: + +``` +LIBRARY_IOC = APPNAME +DBD += appname.dbd + +APPNAME_DBD += asyn.dbd +APPNAME_DBD += asubFunctions.dbd + +APPNAME_SRCS += c_source.c + +APPNAME_LIBS += $(EPICS_BASE_IOC_LIBS) +APPNAME_LIBS += asyn utilities +APPNAME_LIBS += asubFunctions +``` +Where `APPNAME` is to be replaced with the name of the new support module being built. + +In the `configure/RELEASE`, ensure you have listed all required dependencies. A minimum set for an aSub record is shown below, but your support module may also require additional dependencies. + +``` +ASYN=$(SUPPORT)/asyn/master +ONCRPC=$(SUPPORT)/oncrpc/master +SNCSEQ=$(SUPPORT)/seq/master +UTILITIES=$(SUPPORT)/utilities/master +ASUBFUNCTIONS=$(SUPPORT)/asubFunctions/master +``` + +Finally, the **IOC** makefile (`build.mak`) will now need to be updated to include the DBD and Library produced by the support module. The following lines will add the dependency: + +``` +$(APPNAME)_DBD += LIBRARY_NAME.dbd +$(APPNAME)_LIBS += LIBRARY_NAME +``` + +Here, `$(APPNAME)` _is_ a macro, and `LIBRARY_NAME` will need to be replaced with the names of the library and DBD file that you have created above. + +You will also need to ensure that the support module location is mentioned in `ioc/{iocname}/configure/RELEASE`. However, for most IOCs, this should already be the case, as they will get their stream protocol from that location. + +## Writing an aSub Function + +### C code + +Define a C function which will be called by the aSub record. This must have the same name as the `SNAM` field in EPICS DB. You also need to tie this function in to the epics database with a call to `epicsRegisterFunction`. At this stage your C source file should look like the following: + +```C +#include +#include +#include +#include + +static long function_name(aSubRecord *prec) +{ + /* Either call a C++ function (see below) or implement logic here. */ + return 0; +} + +epicsRegisterFunction(function_name); +``` + +### Escaping to C++ + +Note: escaping to C++ is not strictly necessary, it is possible to implement the logic in C. However it is often easier to work with C++ due to it's larger standard library and [boost](https://www.boost.org/). + +#### Create a c/c++ header file + +In the same directory as the `C` source code file, create a header (`.h` extension) like the following: + +```C++ +#ifdef __cplusplus +extern "C" { +#endifit + +extern long function_name_impl(aSubRecord *prec); +extern long another_function_name_impl(aSubRecord *prec); + +#ifdef __cplusplus +} +#endif +``` + +#### Reference the header file from the C code + +In C code which is called by the EPICS records, call the C++ functions which will be parsing the data. Your C source file should now look something like the following: + +```C +#include +#include +#include +#include + +#include "header_file.h" + +static long function_name(aSubRecord *prec) +{ + return function_name_impl(prec); +} + +epicsRegisterFunction(function_name); +``` + +Note that C and C++ functions cannot share the same name. Here we have appended `_impl` to the name of the C++ function to indicate that it is the implementation. + +#### Reference the header file from C++ + +Create a C++ source file (`.cpp` extension, cannot share the same name as the `.c` file). The following is an example of a skeletal C++ source file which simply sets the output data (`VALA`, which goes to `OUTA`) to the input value (`A`, which is pulled from `INPA` in the EPICS database file): + +```C++ +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "header_file.h" + +long function_name_impl(aSubRecord *prec) +{ + /* + * Returns the first input value back, assuming both have been + * declared as LONG in the db file. You can check the fta and + * ftva fields to confirm the data types + */ + *(epicsInt32*)prec->vala = *(epicsInt32*)prec->a; + return 0; /* process output links */ +} + +``` + +The function takes one argument, `prec`, which is as a pointer to the EPICS aSub record. Changes made to `prec` will set the value of the corresponding EPICS fields once the aSub function returns (return 0 to indicate success, or 1 to indicate failure). + +#### Add the C++ source file to the support module Makefile. + +The makefile should now contain a line that looks like: + +``` +APPNAME_SRCS += c_source.c cpp_source.cpp +``` + +## aSub Record Function Tips and Tricks + +It is advised that when writing an aSub record, you escape to C++ as soon as possible so that you can use the C++ standard libraries and error handling capacity to improve the robustness of your IOC to errors arising from the aSub record. + +### GoogleTest unit tests + +GoogleTest can be used to write unit tests for you aSub functions. Details on how to set up GoogleTest to compile under EPICS can be found [here](#googletest_intro). + +### Error catching + +To reduce the chances of the IOC crashing due to an exception being, it is best practice to wrap the logic of your function in a `try-catch` block to log these exceptions and return a non-zero value. E.g. to catch a stand +```C++ +try { + // Your code here +} +catch (std::exception& e) { + errlogSevPrintf(errlogMajor, "%s exception: %s", prec->name, e.what()); + return 1; +} +``` + +### Defensive Type Checking + +As any field of the aSub record can be written to, it is best practice to check types of input and output fields before reading and writing. + +To check that the type of the input field `a` is a `STRING`, use + +```C++ +if (prec->fta != menuFtypeSTRING) + { + errlogSevPrintf(errlogMajor, "%s incorrect input argument type A", prec->name); + return 1; + } +``` +To check that the type of the output field `a` is of type `STRING` use + +```C++ +if (prec->ftva != menuFtypeSTRING) + { + errlogSevPrintf(errlogMajor, "%s incorrect output argument type A", prec->name); + return 1; + } +``` + +### Reading from a waveform of strings + +If you need to read from a waveform of strings, then you need to be extra careful with types. The best way we have found is to read the waveform into a `vector` in the following way: + +```C++ +std::vector args; +for (unsigned int i = 0; i < prec->noa; ++i) { + std::string s(((epicsOldString*)(prec->a))[i], sizeof(epicsOldString)); + args.push_back(s); +} +``` + +This takes into account that the type of each element of the waveform is not a `char*` but a` epicsOldString`. + +### Decoupling from the aSub record pointer + +You may find it useful to decouple the logic of writing to the output fields of the aSub record from the main aSub record it self. This makes it easier to test how you write to the aSub record without needing a pointer to a real aSub record. You can encapsulate the values you need in a `struct`. + +```C++ +struct aSubOutputParameters{ + void* outputPointer; + epicsEnum16 outputType; + + // Struct constructors + aSubOutputParameters() {} + aSubOutputParameters(void* output_pointer, epicsEnum16 output_type) + : outputPointer(output_pointer), outputType(output_type) {} +}; +``` +You can then use this `struct` to create a map which associates a value (a integer channel number in the example below) to the specific aSub output data that value corresponds to. + +```C++ +std::map asub_channel_output(aSubRecord *prec) { + std::map channel_outputs; + + channel_outputs.insert(std::pair(1, aSubOutputParameters(prec->vala, prec->ftva))); + channel_outputs.insert(std::pair(2, aSubOutputParameters(prec->valb, prec->ftvb))); + channel_outputs.insert(std::pair(3, aSubOutputParameters(prec->valc, prec->ftvc))); + channel_outputs.insert(std::pair(4, aSubOutputParameters(prec->vald, prec->ftvd))); + channel_outputs.insert(std::pair(5, aSubOutputParameters(prec->vale, prec->ftve))); + channel_outputs.insert(std::pair(6, aSubOutputParameters(prec->valf, prec->ftvf))); + channel_outputs.insert(std::pair(7, aSubOutputParameters(prec->valg, prec->ftvg))); + channel_outputs.insert(std::pair(8, aSubOutputParameters(prec->valh, prec->ftvh))); + channel_outputs.insert(std::pair(9, aSubOutputParameters(prec->vali, prec->ftvi))); + channel_outputs.insert(std::pair(10, aSubOutputParameters(prec->valj, prec->ftvj))); + + return channel_outputs; +} +``` + +When testing you need to create mock versions of the `asub_channel_output` map rather than the whole aSub record. + +### Parsing functions as arguments + +If you find yourself needing to pass a function as an argument, you can do so using: + +1. `boost::function func` (remember to include `boost/function.hpp`) +1. `std::function func` (remember to include `functional, C++11 and later only) +1. Using a `template` and passing the function as a parameter `F func`. **N.B.**: Types are checked at runtime not compile time when using templates (more details [here](https://stackoverflow.com/questions/1174169/function-passed-as-template-argument)). +1. Function pointers - try to use the above safer ways first. + +This technique can be useful if you have multiple aSub functions that are similar apart from the logic to parse a value from the waveform. You can then pass a different functions as arguments to a base aSub record function to generate your aSub record functions. + +### Record doesn't seem to work + +Add in some debug lines: + +``` +errlogSevPrintf(errlogMajor, "Start"); +``` + +If you don't see this in the log (works even under the testing framework) do a `caget -a` and make sure all the links are valid; an invalid link specification will cause the record not to process. + +### Tips + +* LONG in epics DB world is 32 bit, while the C long type is 32bit on Windows and 64bit on Linux. When casting from aSub you should use the `epicsInt32` type and not "long" +* Do not assign directly to aSub record argument such as "prec->vala = result". Rather copy into the pre-allocated space as shown below +* Remember all aSub arguments are arrays of the type even if the type is an array, so `epicsOldString*` +``` +epicsInt32 i = *(epicsInt32*)prec->a; +epicsOldString* result = (epicsOldString*)prec->vala; +strcpy_s(*result, MAX_STRING_SIZE, "Unknown") +``` + +## aSub Functions + +* [`splitCharWaveform`](https://github.com/ISISComputingGroup/EPICS-asubFunctions/blob/master/asubFunctionsApp/src/splitCharWaveform.c#L108) diff --git a/doc/iocs/tools/icpconfig.md b/doc/iocs/tools/icpconfig.md new file mode 100644 index 000000000..b45cbda16 --- /dev/null +++ b/doc/iocs/tools/icpconfig.md @@ -0,0 +1,10 @@ +# ICP Config + +ICP config is a set of C functions that are called by every IOC on startup to define the macros used within the IOC. The code for this is [here](https://github.com/ISISComputingGroup/EPICS-icpconfig). Macros can come from the following: +* The current configuration +* `\Instrument\Settings\globals.txt` - See [here](../testing/Running-IOCs) +* `\Instrument\var\tmp\test_macros.txt` - A file create by the IOC_test_framework to create macros for testing. Only used if the environment variables `TESTDEVSIM=yes` or `TESTRECSIM=yes`. + +Macros are loaded in the order above, such that `globals.txt` will override macros in the configuration and test macros will override all others. See diagram below for an idea of how this works. + +![](loading_macros.png) \ No newline at end of file diff --git a/doc/iocs/tools/loading_macros.png b/doc/iocs/tools/loading_macros.png new file mode 100644 index 000000000..6d68ffddc Binary files /dev/null and b/doc/iocs/tools/loading_macros.png differ diff --git a/doc/iocs/tools/lua/LuaCheck.md b/doc/iocs/tools/lua/LuaCheck.md new file mode 100644 index 000000000..a188a2dc8 --- /dev/null +++ b/doc/iocs/tools/lua/LuaCheck.md @@ -0,0 +1,41 @@ +# LuaCheck + +## Installing LuaCheck + +Either install via the hererocks python script or luacheck by itself. The hererocks way is longer but will integrate with your environment and favourite text editor nicely as it sets up parts such as luacheck.path and luarocks + +Via the hererocks python script: +- Download hererocks: + - Open powershell and navigate to somewhere you would like hererocks to download to + - Run `wget https://raw.githubusercontent.com/mpeterv/hererocks/latest/hererocks.py -OutFile hererocks.py` +- Install and activate lua and luarocks: + - In an EPICS terminal navigate to where you downloaded hererocks.py and run `python hererocks.py lua53 -l5.3 -rlatest` which installs lua 5.3 into `$current directory$\lua53` + - Add `$current directory$\lua53\bin` to your PATH where current directory is the directory you installed lua 5.3 in + - Close and reopen your EPICS terminal to allow your PATH to update +- Install luacheck: + - In your new EPICS terminal run `luarocks install luacheck` +- Luacheck is now installed + +OR install luacheck by itself: +- Go to the [luacheck GitHub page](https://github.com/mpeterv/luacheck/tree/76bb56736702e8651537b2a9c10ae55ab7dc1d5d) and under Windows binary download click the download link which will download the file luacheck.exe +- Place the luacheck.exe file in a useful place e.g. Program Files and add that location to your PATH. +- Restart any command lines you have open. +- Luacheck is now installed + +## Using luacheck: + +Post-install step: +- Set luacheck config: + - Create a new file `.luacheckrc` in `%LOCALAPPDATA%\Luacheck` as this is where luacheck looks for config files + - Add the below information to the file to set the config + - Note: This ignores setting, mutating and accessing undefined global variables. We are doing this because epics Lua uses an interactive Lua shell which requires us to use global variables in a different way to regular Lua. + +``` +ignore = {"111", "112", "113"} +``` + +Basic usage: `luacheck file.lua` + +For more detailed instructions on the use of luacheck see the [luacheck documentation](https://luacheck.readthedocs.io/en/stable/). + +The [luacheck GitHub page](https://github.com/mpeterv/luacheck/tree/76bb56736702e8651537b2a9c10ae55ab7dc1d5d) also has information about how to use luacheck in your favourite editor. \ No newline at end of file diff --git a/doc/iocs/tools/lua/Our-Lua-Utility-Functions.md b/doc/iocs/tools/lua/Our-Lua-Utility-Functions.md new file mode 100644 index 000000000..3c01cdfe0 --- /dev/null +++ b/doc/iocs/tools/lua/Our-Lua-Utility-Functions.md @@ -0,0 +1,79 @@ +# Lua Utility Functions + +## Using our Lua utility functions + +Currently we have four functions available as utilities for booting iocs using lua: `getMacroValue`, `setAsynOptions`, `setHardwareFlowControl`, `setSoftwareFlowControl`. + +To be able to use these you need to add this to the top of your lua file: + +``` +package.path = package.path .. ';' .. os.getenv("UTILITIES") .. '/lua/luaUtils.lua;' +ibex_utils = require "luaUtils" +``` +### `getMacroValue(options)` + +Getting a macro from the environment. If the macro cannot be retrieved from the environment the default options value is returned. If there is no default given and the macro cannot be retrieved an error is raised. + +- macro: String, the macro to look up in the +- default: String, the value to return if the macro if the macro is not set in the environment + +Example call: `ibex_utils.getMacroValue{macro="RECSIM", default="0"}` +CMD equivalent: `$(RECSIM=0)` + +### `setAsynOptions(device, port, baud, bits, parity, stop)` + +Note: This automatically sets asyn address port to -1. If you need to set it differently please use the calls `iocsh.drvAsynSerialPortConfigure` and `iocsh.asynSetOption` in your script. + +For a real device configure the asyn serial port and set the baud, bits, parity and stop options for it. + +- device: String, the name of the asyn port +- port: String, the name of the physical port the device is connected to +- baud: Integer, The baud rate of the device +- bits: Integer, the number of data bits +- parity: String, the device parity +- stop: Integer, the number of stop bits. + +Example call: +``` +if (not isRecsim and not isDevsim) then + local port = ibex_utils.getMacroValue{macro="PORT", default="NO_PORT_MACRO"} + local baud = ibex_utils.getMacroValue{macro="BAUD", default="9600"} + local bits = ibex_utils.getMacroValue{macro="BITS", default="8"} + local parity = ibex_utils.getMacroValue{macro="PARITY", default="none"} + local stop = ibex_utils.getMacroValue{macro="STOP", default="1"} + ibex_utils.setAsynOptions(device, port, baud, bits, parity, stop) +... +``` + +CMD equivalent: +``` +$(IFNOTDEVSIM) $(IFNOTRECSIM) drvAsynSerialPortConfigure("$(DEVICE)", "$(PORT=NO_PORT_MACRO)", 0, 0, 0, 0) +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("$(DEVICE)", -1, "baud", "$(BAUD=9600)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("$(DEVICE)", -1, "bits", "$(BITS=8)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("$(DEVICE)", -1, "parity", "$(PARITY=none)") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("$(DEVICE)", -1, "stop", "$(STOP=1)") +``` + +### `setHardwareFlowControl(device, flowControlOn)` + +Note: This automatically sets asyn address port to -1. If you need to set it differently please use the call iocsh.asynSetOption in your script the way you want. + +Set the hardware flow control for when it is off or on. + +- device: String, the name of the asyn port. +- `flowControlOn`: Boolean, true if hardware flow control is on. + +### `setSoftwareFlowControl(device, flowControlOn)` + +Note: This automatically sets asyn address port to -1. If you need to set it differently please use the call iocsh.asynSetOption in your script the way you want. + +Sets the software flow control for when it is off or on. + +- device: String, the name of the asyn port +- `flowControlOn`: Boolean, true if software flow control is on. + +## Adding to our Lua utility functions + +Things to do: +- Add documentation in this page (the plan is to have documentation closer to the code, but that also needs to be worked out first) +- Add it to the imports in "Using our Lua utility functions" section above \ No newline at end of file diff --git a/doc/iocs/troubleshooting/ASYN-Trace-Masks-(Debugging-IOC,-ASYN).md b/doc/iocs/troubleshooting/ASYN-Trace-Masks-(Debugging-IOC,-ASYN).md new file mode 100644 index 000000000..c4d0cfeef --- /dev/null +++ b/doc/iocs/troubleshooting/ASYN-Trace-Masks-(Debugging-IOC,-ASYN).md @@ -0,0 +1,100 @@ +# Asyn Trace Masks + +For asyn based drivers, such as stream device, additional information printing can be enabled on a port via commands from the ioc shell. The commands let you see what bytes are being sent and received, and also the flow of control/logic. + +## Print commands and responses to screen + +The two commands you need are: + +``` +asynSetTraceIOMask("L0", -1, 0x2) +asynSetTraceMask("L0", -1, 0x9) +``` + +Note: if the device's messages are longer than 80 characters, you should increase the buffer size by also running: + +``` +asynSetTraceIOTruncateSize("L0", -1, 1024) +``` + +Where 1024 is the maximum message length you expect - could be increased if your device requires it. + +to turn off use +``` +asynSetTraceMask("L0", -1, 0x1) +``` + +## ASYN trace mask + +This determines what you see and is set to 0x1 (ASYN_TRACE_ERROR) by default. The following additional values +can be ORed together and optionally set: + +* `0x1`: ASYN_TRACE_ERROR - Print error messages (default) +* `0x2`: ASYN_TRACEIO_DEVICE - Device support reports I/O activity. +* `0x4`: ASYN_TRACEIO_FILTER - Any layer between device support and the low level driver e.g. asyn interpose functions +* `0x8`: ASYN_TRACEIO_DRIVER - Low level driver reports I/O activity +* `0x10`: ASYN_TRACE_FLOW - Report logic flow +* `0x20`: ASYN_TRACE_WARNING - Report warnings, i.e. conditions between ASYN_TRACE_ERROR and ASYN_TRACE_FLOW + +If L0 if your asyn port, then + +`asynSetTraceMask("L0",-1,0x9)` + +will enable ASYN_TRACE_ERROR and ASYN_TRACEIO_DRIVER for all addresses (-1) on port "L0" which is a good place to start for debugging. You may also want to use ASYN_TRACE_FLOW but this can print a bit too much depending on the driver and you may loose your IO device messages amongst them. If you want to try this separately then e.g. + +`asynSetTraceMask("L0",0,0x11)` + +will enable ASYN_TRACE_ERROR and ASYN_TRACE_FLOW on port "L0" but just for address 0. + +To see output of interpose functions such as `asynInterposeThrottle` add `0x4`, so `0x9` become `0xD` for example + +## ASYN traceIO mask + +If ASYN_TRACEIO_DRIVER has been enabled, then this mask determines the format of the output produced when reporting I/O. The default is ASYN_TRACEIO_NODATA so the count of bytes transferred is reported, but the bytes themselves are not printed. + +* `0x0` (ASYN_TRACEIO_NODATA) - Print count of bytes transferred, but not bytes themselves +* `0x1` (ASYN_TRACEIO_ASCII) - Print bytes with a "%s" style format +* `0x2` (ASYN_TRACEIO_ESCAPE) - Calls `epicsStrPrintEscaped()` to display bytes +* `0x4` (ASYN_TRACEIO_HEX) - Print hexadecimal values of each byte with " %2.2x" + +So + +`asynSetTraceIOMask("L0",-1,0x2)` + +would enable ASYN_TRACEIO_ESCAPE style printing of all bytes read/written by the asyn driver via port "L0", which is a good place to start + +## Additional trace information + +This mask determines the information printed at the start of each message above. The default is `ASYN_TRACEINFO_TIME` which just adds a timestamp, normally this is sufficient but it can be changed by calling `asynSetTraceInfoMask` with a combination of flags: + +* `0x1` `(ASYN_TRACEINFO_TIME)` prints the date and time of the message (default) +* `0x2` `(ASYN_TRACEINFO_PORT)` prints `[port,addr,reason]`, where port is the port name, `addr` is the asyn address, and reason is pasynUser->reason. These are the 3 pieces of "addressing" information in asyn. +* `0x4` `(ASYN_TRACEINFO_SOURCE)` prints the file name and line number, i.e. `[__FILE__,__LINE__]` where the asynPrint or asynPrintIO statement occurs. +* `0x8` `(ASYN_TRACEINFO_THREAD)` prints the thread name, thread ID and thread priority, i.e. `[epicsThreadGetNameSelf(), epicsThreadGetIdSelf(), epicsThreadGetPrioritySelf()]`. + +e.g. to print time and port details at start of each message + +`asynSetTraceInfoMask("L0",-1,0x3)` + +## Logging to file + +By default the trace output is sent to the epics error log (which by default echoes to console too), but it can instead be sent to file e.g. + +`asynSetTraceFile("L0",-1,"temp.log")` + +## Getting (VERY VERBOSE) output from stream device. + +In your `st.cmd`, or at the EPICS prompt from a console session, use: + +`var streamDebug 1` + +This will turn on debug logging from stream device which is very verbose. + +## Seeing all channel access put requests to the IOC + +In your `st.cmd`, or at the EPICS prompt from a console session, use: + +`var caPutLogToConsole 1` + +This will log all details of `caput` etc to the ioc to console using the `caPutLog` mechanism. + diff --git a/doc/iocs/troubleshooting/Dependency-Walker.md b/doc/iocs/troubleshooting/Dependency-Walker.md new file mode 100644 index 000000000..f2154cbb6 --- /dev/null +++ b/doc/iocs/troubleshooting/Dependency-Walker.md @@ -0,0 +1,8 @@ +# DLL Dependency Walker + +To find DLL dependencies of an IOC do the following: +* Open EPICS terminal (`cmd` with `config_env.bat` run) +* cd to iocBoot area of IOC (where its st.cmd is) +* type `depends` - this will set PATHS for this IOC and launch a GUI window +* Now use File -> open and browse to the IOC exe file in the bin directory a couple of levels above +* Now wait... diff --git a/doc/iocs/troubleshooting/IOC-And-Device-Trouble-Shooting.md b/doc/iocs/troubleshooting/IOC-And-Device-Trouble-Shooting.md new file mode 100644 index 000000000..f9c53e35d --- /dev/null +++ b/doc/iocs/troubleshooting/IOC-And-Device-Trouble-Shooting.md @@ -0,0 +1,317 @@ +# General troubleshooting + +**After diagnosing the problem** If you needed to go down to the instrument please correct/update any connection information to [Specific Device IOC page](/Specific-IOCs). + +### My IOC does not show up in the IOC list in IBEX + +To update the list of IOCs in IBEX (which is stored in the database) you will need to run `make iocstartups` from `C:\Instrument\Apps\EPICS\` - this will concatenate all the `config.xml` files in the `ioc` directory for each IOC, then filter it and push it to the database. + +### My IOC fails to make with "Permission Denied" + +This is likely not to do with permissions but that the file is in use elsewhere. Try running stop_ibex_server.bat and if that doesn't work find out what else may be using the file ([LockHunter](https://lockhunter.com/) is quite useful). + +### My IOC fails to build with `epics/base/master/configure/RULES.Db:201: *** target pattern contains no '%'` + +This could be caused by the IOC attempting to copy in invalid db files during the build check in `XXIOC-01App/Db/Makefile` that any dbs listed there exist. + +### I am having an Issue with a Specific IOC + +Some IOC have important details which may help see if it is listed in [Specific Device IOC](/Specific-IOCs). + +### checking stream device/asyn serial port settings + +At the IOC command prompt, type: +``` +dbior("drvAsyn",2) +``` + +### ioc fails to connect to serial COM port +Errors like +``` +2022/03/29 17:19:08.570 L0 -1 autoConnect could not connect: \\.\COM106 Can't open: Access is denied. +``` +*access is denied* probably means another process (an IOC or terminal emulator like hyperterm) is already using the port. After you stop the offending process, it may be a few seconds before you can connect. + +### It doesn't work What Should I Do? + +1. Connect over hyperterminal (see *Connect over hyperterminal*) +1. Check command set is correct (look at documentation) +1. Start the IOC and check it is not in SIM mode +1. Set the IOC mask (see *What is Passing between the IOC and the *Stream* Device*) +1. If you don't think records are processing correctly, try setting the `TPRO` (trace processing) field of the relevant records to a non-zero value. This will print a message to the IOC log whenever the record processes. + +### Any write to the PV fails + +If any caput command to a pv fails with a message similar to +``` +Old : TE:NDW1801:ICEFRDGE_01:MIMIC:SKIP:SP SKIP0 +New : CA.Client.Exception..................................TE:NDW1801:ICEFRDGE_01:MIMIC:SKIP.:SP. .SKIP0. +......... + Warning: "Channel write request failed" + Context: "op=1, channel=TE:NDW1801:ICEFRDGE_01:MIMIC:SKIP:SP, type=DBR_STRING, count=1, ctx="TE:NDW1801:ICEFRDGE_01:MIMIC:SKIP:SP"" + Source File: ../oldChannelNotify.cpp line 160 + Current Time: Tue Nov 19 2019 09:56:21.490881300 +...............................................................… +``` +Then you may want to check if said PV is disabled somewhere in your db file through its DISP field by another PV. + +### Connect over hyperterminal + +Start hyper-terminal set up connection to device. Find a command that is returns something or that changes something on the display. Run that command look for the change. Try +1. Reading +1. Writing + +If it doesn't work check: + +1. Protocol (RS232, RS422, RS482) +1. Baud Rate +1. Com Port +1. Stop Bits +1. Parity +1. Line ending/Termination characters. +1. Flow control + +If this doesn't work check the moxa logging (see data over the moxa connection). If this doesn't work there is a device that can be plugged into a serial port to intercept all traffic. + +### It won't work in the IOC initially, but with no changes works after connecting to hyperterminal + +If the IOC doesn't work on startup, but does after using hyperterminal, at a command prompt run a `mode com1` on the appropriate com port (example is for COM1) before and after running hyperterminal. If there is a change that isn't duplicated by starting the IOC, then there is possibly a setting missing in asyn. + +### It did work and then freezes ## + +This may be due to a low control issue e.g. `Xon`/`Xoff` has been incorrectly specified and some binary character +happens to match `Xoff`. You should see asyn still sending character when you enable trace, but non getting through. If you run the `dbior` command at level 2 you may see `waiting as seen Xoff`. + +### The device is sensitive to input commands + +Some devices has sensitivity to the rate at which you poll them. Typically the devices manual will detail appropriate command spacing, and the ability to use `wait [time in ms];` after `in` or `out` commands in the protocol file is available. However, it is important to ensure your records and their types and error handling have been correctly defined. For instance in the event of a type error, since this is handled outside of the protocol file, it will bypass any structures you have in place to slow polling rates. + +### What is Passing between the IOC and the *Stream* Device + +It is possible to put stream into a debug mode where everything sent and received is written to the console. To do this simply add to you st.cmd file (defined on your asyn port) : + +``` +asynSetTraceMask("L0",-1,0x9) +asynSetTraceIOMask("L0",-1,0x2) +``` + +where is the port name you used in the asyn setup e.g. `drvAsynSerialPortConfigure(...)` + +This will include the terminator character, if you don't see it it is not being sent or received. +If no reply is given this will include a message "No reply from device in XXXms" + +See [Asyn trace mask](ASYN-Trace-Masks-(Debugging-IOC,-ASYN)) for more details on specifying trace masks + +To stop printing these commands to the log use: + +``` +asynSetTraceMask("L0",-1,0x0) +asynSetTraceIOMask("L0",-1,0x0) +``` + +### Is the MOXA seeing anything? + +It is sometime useful to see if the moxa is seeing any traffic. If you are in the cabin you can look at the flashing lights (remember to take away 4 to convert from com number to port number). There is one light for send and one for receive both should flash. + +If you want to do it remotely then you can by using the moxa web page. The address is on the [nagios page (standard log on)](https://varanus.isis.cclrc.ac.uk/nagios/). Look at Hosts -> MOXA_ the IP address is at top of the page in the middle. Enter the IP in a web browser and enter password from password page. +Then Monitor -> Asyn shows received and sent byte count. + +### Environment Variable not getting set from MASTER_RELEASE + +Variables are transferred from `...EPICS\configure\MASTER_RELEASE` to `...EPICS\ioc\master\\iocBoot\\envPaths` when the ioc is made. You will have to delete the file to get the newest macros in and the paths have to exist. + +### I get a `UDF` alarm on by `bi` record that's getting its value from a `calcout` + +This was observed whilst programming the GEMORC IOC. We had records like so: + +``` +record(calc, "$(P)CALC") +{ + field(INPA, "$(P)LONGIN1") + field(INPB, "$(P)LONGIN2") + field(CALC, "A+B==0") + field(OUT, "$(P)BI") +} + +record(bi, "$(P)BI") +{ + field(ZNAM, "No") + field(ONAM, "Yes") +} +``` + +The result was a `UDF` alarm on the `bi` because the `RVAL` wasn't being updated in line with the `VAL`. We tried a lot of things to sort this. In the end we changed the `OUT` field of the `CALC` to a `FLNK` and added the following field to the `bi`: `field(INP, "$(P)CALC")`. We don't know why this solution worked over the many others we tried. + +### IOC Crashes on debug the exception in "Unhandled exception at ... : Stack cookie instrumentation code detected a stack-based buffer overrun." + +This is caused when something bad happens to the stack. There are many underlying causes to this, in my case it was that the IOC had loaded a dll which used an incorrect version of visual studio in its build see [here](https://docs.microsoft.com/en-us/cpp/c-runtime-library/potential-errors-passing-crt-objects-across-dll-boundaries?view=msvc-160). The dlls that are loaded (with their paths) can be seen in debug -> Windows -> Modules in VS. To check if it is this error delete the offending dll and try the process again. To fix the error convert the IOC to use build dependencies instead of the master release list, instructions are in [Reducing Build Dependencies](../compiling/Reducing-Build-Dependencies) + +In some cases you may be using a vendor DLL and have no control over which version of VS it is built with. If this is the case you may get an access violation when freeing memory allocated by the DLL. The easiest way round this if there is no way to rebuild the DLL is just to take the memory hit of not freeing the resource. + +### Log shows `currentTime::getCurrentTime(): XXX sec time discontinuity detected` + +The log for all IOCs shows the message: + +``` +currentTime::getCurrentTime(): XXX sec time discontinuity detected +``` + +It is caused by the time going backwards due to a correction; the underlying cause is unknown. However, this seems to disappear once the host machine is restarted. + +### My IOC is not starting/reacting at all when launching from console/ProcServ + +If your IOC running in procserv is unresponsive (does not start up, `ctrl + x` from console does nothing) but you can still start the IOC manually from the IOCBoot directory, it is possible another process is using the port allocated to this IOC by procserv. +1. You can find the port procserv is trying to use for a given IOC under `EPICS\iocstartup\procserv.bat` +1. In a command window, type `netstat -ano`. This gives you a list of ports currently in use and the IDs of processes using them +1. Search that list for the port from step 1. This will give you the ID of the process using it. +1. Find the process by it's ID in task manager. + +### Asyn reports an `asynStatus` value of greater than 5 + +The `asynStatus` enum only contains values 0 to 5. But these are extended in `asyn/asynPortDriver/paramErrors.h` to include a few more useful statuses. + +## Autosave + +### Lost autosave values (especially on Galils) + +This is based on ticket: https://github.com/ISISComputingGroup/IBEX/issues/2180 + +Possible symptoms include: + +- Autosaved values mysteriously changing when the IOC is restarted + - This includes Galils not retaining their position +- Errors of the form `[DATE] dbFindRecord for 'MY_PV.FIELD' failed` +- Autosave files containing just a header and `` tag + +This has been observed primarily on Galils since they create custom monitor sets but it is feasible the issue could be seen elsewhere. The Galils pass macros to their monitor sets. If no macros are passed (e.g. if `GALILADDR0n` is not set) then no monitor will be created and no values will be saved. This will mean the device (e.g. the motor) will start with its default values. If the macro is reintroduced then those default values will become the new autosave values. The previous values can be recovered by restoring a previous autosave file (e.g. copy `GALIL_02_settings.sav_170309-144116` to `GALIL_02_settings.sav` and restart the IOC). + +Galils also will not autosave any settings or values if they are set to `DEVSIM` or `RECSIM` mode. + +### comparing current to previous autosaved values + +When an IOC starts, it copies the last autosave values to a date stamped file such as `GALIL_02_settings.sav_170309-144116` You can compare the current running IOC values to any of these files by using the `asVerify` command that is built as part of the `autosave` module and should be in your path after a `config_env` e.g. +``` +asVerify C:\Instrument\Var\autosave\GALIL_02\GALIL_02_settings.sav_170309-144116 +``` +There is no command to apply an autosave file to the current PVS, this is because it may be a bit dangerous as boot time autosave can apply values without forcing a process of a PV, whereas using a `caput` could have different results. Thus it is better to stop ioc, update autosave file, start ioc as described for galil above. + +## Motors + +### Limit switches not active at the limit position + +https://github.com/ISISComputingGroup/IBEX/issues/2174 + +In the motor record (`C:\Instrument\Apps\EPICS\support\motor\master\motorApp\MotorSrc\motorRecord.cc`) the limit flag is only activated if the motor's limit switch bit is active and motor's command direction (CDIR) is correct. Correct in this context means that the motor is moving out of range. That is, the motor is moving in a positive direction past an upper limit or in a negative direction past the lower limit. The command direction is only set when a command (e.g. home, move) is sent. If the IOC is restarted, the value isn't saved (and cannot be auto saved) so the limit flag behaviour will depend on the initial value of the command direction. + +In summary, being at the limit position is insufficient to cause the limit flag to be active. This is expected behaviour, though can be slightly unintuitive at first. + +### Readbacks from device keep twitching + +This may be because of variances in the signal returned from the motor. This is true, for example, on the jaws set on Merlin. It also causes the motor status label on the bumpstrip to keep switching. Note that leaving it in this state can increase dramatically the amount of information sent to the archiver. + +To fix, use caput for the relevant PV in the motor record: + +- `IN:[INST]:MOT:MTR0n0m.MDEL`: Sets the motor deadband. This will stop the position readback from oscillating +- `IN:[INST]:MOT:MTR0n0m.ADEL`: Sets the archiver deadband. This will stop values being written to the archiver unless the value changes by more than the deadband value +- `IN:[INST]:MOT:MTR0n0m_EDEL_SP`: Sets the encoder deadband. No readbacks will be updated unless the encoder varies by this amount. + +### McLennan OPI controls all red + +- Does the panel on the front of the controller display a letter? + - `r` + - Can happen owing to a communication issue when the IOC starts if the baud rate/stop bits aren't set correctly + - `t` + - Tracking abort. Usually happens if the encoder resolution is incorrect. Is the `ERES` macro fraction the right way around? + - `n` + - Observed once on Merlin after the McLennan had been idle for a long time. Only happened after homing the motor. Restarting the controller (with the IOC stopped) fixed the issue + - No + - Try restarting the IOC, then move the motor through a significant operation (a home or jog to limit). Sometimes restarting the IOC is enough, sometimes the move causes the underlying issue to reveal itself (see `n` above) + +### which process owns a serial port + +From an administrator (i.e. gamekeeper) prompt, run the `sysinternals` handle command like: + +``` +handle.exe -a | findstr "Serial pid:" +``` + +This will print a list of PIDs and any matches to a serial port allocated, take the pid number above the relevant \Device\Serial line + +{#ioc_troubleshooting_logging} +## Logging + +### I Changed the logging setting but it uses old settings + +To sync up changes in logging from the DB file to the logging you must: + +1. Restart the IOC so the logging info fields are pushed to the database +1. If any new PVs are being archived or settings have been changed the ARINST needs restarting +1. The ARACCESS needs restarting so it picks up the configuration changes from the database + +### There was an error which I have fixed but I missed a log file + +To regenerate log files set the date in the `c:\Logs\LOG_last_active_time` to the time you want to start generating files from, if this is more than delta days ago (2nd line of the log file) increase this number too. Then restart the ARACCESS + +### I want to Generate a Log file from some PVs Now + +This can be done between two dates see `...EPICS\ISIS\inst_servers\master\ArchiverAccess\log_file_generator.py` as an example. + +### Value logs from IOC not produced/ARACCESS not creating log files/ENGINX Stress Rig not writing log files + +Certain IOCs can be made to generate log files using the [ARACCESS component](/system_components/Logging-from-the-archive), if these are no longer being produced then check: + +- ARCCESS console (restart it) This should show a little blurb and should have start and stop lines, e.g. + ``` + [2020-09-24 09:20:21] @@@ @@@ @@@ @@@ @@@ + [2020-09-24 09:20:21] @@@ Received a sigChild for process 25052. The process was killed by signal 9 + [2020-09-24 09:20:21] @@@ Current time: 2020-09-24 09:20:21 + [2020-09-24 09:20:21] @@@ Child process is shutting down, a new one will be restarted shortly + [2020-09-24 09:20:21] @@@ ^R or ^X restarts the child, ^Q quits the server + [2020-09-24 09:20:22] @@@ Restarting child "ARACCESS" + [2020-09-24 09:20:22] @@@ (as C:\Windows\system32\cmd.exe) + [2020-09-24 09:20:22] @@@ The PID of new child "ARACCESS" is: 20124 + [2020-09-24 09:20:22] @@@ @@@ @@@ @@@ @@@ + [2020-09-24 09:20:23] [1600935623.82] INFO: Creating a new connection pool: DBSVR_127.0.0.1_archive_report + [2020-09-24 09:20:24] [1600935624.74] INFO: Creating a new connection pool: DBSVR_127.0.0.1_iocdb_iocdb + [2020-09-24 09:20:25] [1600935625.00] INFO: Reading config for ioc: INSTRON_01 + [2020-09-24 09:20:25] [1600935625.00] INFO: Logging configuration (pvs as read from the archive) + [2020-09-24 09:20:25] - file (log on end): C:\logs\INSTRON_01\INSTRON_01_{start_time}.dat + [2020-09-24 09:20:25] - file (continuous): C:\logs\INSTRON_01\INSTRON_01_{start_time}_continuous.dat + [2020-09-24 09:20:25] - trigger pv: IN:ENGINX:INSTRON_01:LOG:RECORD:SP.VAL + [2020-09-24 09:20:25] - trigger pv: Logging from pv IN:ENGINX:INSTRON_01:LOG:SCAN.VAL with a default on error of 1s + [2020-09-24 09:20:25] - file headers: [u'Cross Sectional Area = {1:.6f}', u'Gauge Length for Strain = {2:.6f}', u'RB Number = {0}', u''] + [2020-09-24 09:20:25] - pvs in fileheader ['IN:ENGINX:ED:RBNUMBER.VAL', 'IN:ENGINX:INSTRON_01:STRESS:AREA.VAL', 'IN:ENGINX:INSTRON_01:STRAIN:LENGTH.VAL'] + [2020-09-24 09:20:25] - table headers: ['Date/time', u'Run Number', u'Position (mm)', u'Load (MPa)', u'Strain (%)'] + [2020-09-24 09:20:25] - table line: {time} {2} {0:.6f} {1:.6f} {3:.6f} + [2020-09-24 09:20:25] - pvs in table line ['IN:ENGINX:INSTRON_01:POS.VAL', 'IN:ENGINX:INSTRON_01:STRESS.VAL', 'IN:ENGINX:DAE:RUNNUMBER.VAL', 'IN:ENGINX:INSTRON_01:STRAIN.VAL'] + [2020-09-24 09:20:25] [1600935625.01] INFO: Last active: 2020-09-24T09:19:31 (13469489) + ``` + - If this doesn't show both start and stop lines which that it is set to auto start in the default block component +- Start and stop the logging and see if it says it is creating a file, e.g. `INFO: Writing log file '...'` is this in the expected place +- Check the PVs it says it is logging and monitoring do they exist? +- Next check the ARINST level is it up (check `http://localhost:4812/groups`) + - if not restart it, this means that there is no data stored for the run +- Next step is to check that the data is getting into the database (probably plot on log plotter) + +- Write tip on Wiki for importing a database dump file into local database. Commands and output: + + +### Reading historical PV values from local database _after_ it has been backed up and truncated + +This can be done by _importing_ the backed-up database (usually on a network drive) into a local developer's copy, then reading/plotting from there. +Example command from a request to read chopper values on MERLIN: + +``` +mysql.exe -u root -p < [insert network back-up path here]\ndxMERLIN\ibex_database_backup_2024_01_30\ibex_db_sqldump_2024_01_30.sql +Enter password: ************ +``` + +This [script](https://github.com/ISISComputingGroup/EPICS/blob/master/utils/query_ioclog.bat) can be used to extract data into CSV file(s) for e.g. diagnostic purposes. + +The local database can then be truncated to remove the import once work is complete. + +### `EPICS_CA_MAX_ARRAY_BYTES` + +The `EPICS_CA_MAX_ARRAY_BYTES` constant is used throughout ibex to specify the size of the largest waveform (which is usually produced by the dae). Large objects are need especially for (Live View)[/specific_iocs/dae/DAE-Live-View]. Currently the largest know version of this is on POLREF who want to see their linear detector with a time of flight 2D view. This has the size of: + + 5001 (time channels) * 640 * 8 ~ 20M diff --git a/doc/iocs/troubleshooting/IOC-Start-Example.md b/doc/iocs/troubleshooting/IOC-Start-Example.md new file mode 100644 index 000000000..16d9b662f --- /dev/null +++ b/doc/iocs/troubleshooting/IOC-Start-Example.md @@ -0,0 +1,1183 @@ +# IOC Startup Example + +The following gives an example of a normal start of SIMPLE IOC running in proc serve with the instrument running. The following splits out all the bits: + +``` +[2017-08-16 13:53:52] @@@ Restarting child "SIMPLE" +[2017-08-16 13:53:52] @@@ (as C:\windows\system32\cmd.exe) +[2017-08-16 13:53:52] @@@ The PID of new child "SIMPLE" is: 7280 +[2017-08-16 13:53:52] @@@ @@@ @@@ @@@ @@@ +``` +header which shows what has started + +``` +[2017-08-16 13:53:54] #!../../bin/windows-x64/simple +[2017-08-16 13:53:54] ## You may have to change simple to something else +[2017-08-16 13:53:54] ## everywhere it appears in this file +[2017-08-16 13:53:54] < envPaths +``` +start of st.cmd file + +``` +[2017-08-16 13:53:54] epicsEnvSet("IOC","iocsimple") +[2017-08-16 13:53:54] epicsEnvSet("TOP","C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master") +[2017-08-16 13:53:54] epicsEnvSet("ACCESSSECURITY","C:/Instrument/Apps/EPICS/support/AccessSecurity/master") +... ... +[2017-08-16 13:53:54] epicsEnvSet("IOCSTARTUP","C:/Instrument/Apps/EPICS/iocstartup") +[2017-08-16 13:53:54] epicsEnvSet("ICPBINARYDIR","C:/Instrument/Apps/EPICS/ICP_Binaries") +``` + +macros which are defined in `EPICS\configure\MASTER_RELEASE` and point to support modules that may be used by the IOC + +``` +[2017-08-16 13:53:54] cd C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master +[2017-08-16 13:53:54] ## Register all support components +[2017-08-16 13:53:54] dbLoadDatabase "dbd/simple.dbd" +[2017-08-16 13:53:54] simple_registerRecordDeviceDriver pdbbase +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/init.cmd +``` +More st.cmd and then load initial init which sets up configuration info: + +``` +[2017-08-16 13:53:54] icpconfigLoad +[2017-08-16 13:53:54] icpconfigLoad: ioc "SIMPLE" group "SIMPLE" options 0x0 host "NDW1798" +[2017-08-16 13:53:54] icpconfigLoad: config base (ICPCONFIGBASE) is "C:/Instrument/Settings/config" +[2017-08-16 13:53:54] icpconfigLoad: config root (ICPCONFIGROOT) is "C:/Instrument/Settings/config/NDW1798/configurations" +[2017-08-16 13:53:54] icpconfigLoad: * $(SIMULATE)="0" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFSIM)="#" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTSIM)=" " ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(SIMSFX)="" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(DISABLE)="0" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFDISABLE)="#" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTDISABLE)=" " ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(DEVSIM)="0" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFDEVSIM)="#" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTDEVSIM)=" " ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(RECSIM)="0" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFRECSIM)="#" ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTRECSIM)=" " ({initial default}) +[2017-08-16 13:53:54] icpconfigLoad: * $(ICPCONFIGDIR)="C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron" ({initial default}) +``` +defaults +``` +[2017-08-16 13:53:54] icpconfigLoad: last configuration was "Instron" (C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron) +[2017-08-16 13:53:54] icpconfigLoad: configuration "Instron" +[2017-08-16 13:53:54] icpconfigLoad: loading 0 component(s) for "/configurations/Instron" +[2017-08-16 13:53:54] icpconfigLoad: Loading default macros for "/configurations/Instron" +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC sim level "/configurations/Instron" +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC macros for "/configurations/Instron" +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC PVs for "/configurations/Instron" +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC PV sets for "/configurations/Instron" +[2017-08-16 13:53:54] Cannot open directory: C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron/files +[2017-08-16 13:53:54] icpconfigLoad: Found 0 files for "/configurations/Instron" +``` +Last configuration loaded +``` +[2017-08-16 13:53:54] icpconfigLoad: loading old macro file "C:/Instrument/Settings/config/NDW1798/configurations/globals.txt" +[2017-08-16 13:53:54] icpconfigLoad: * $(GALILNUMCRATES)="1" +``` +`globals.txt` macros. + +``` +[2017-08-16 13:53:54] # define a macro with the same name as the IOC name and a blank value. This means we can use a line like +[2017-08-16 13:53:54] # $(IFIOC_GALIL_01=#) to have something that is only executed in GALIL_01 IOC + +[2017-08-16 13:53:54] # and you can pass to db file via A=1,$(IFIOC)= ,A=2 + +[2017-08-16 13:53:54] # used when loading motorUtils and motors moving + +[2017-08-16 13:53:54] epicsEnvSet("IFIOC", "IFIOC_SIMPLE") + +[2017-08-16 13:53:54] epicsEnvSet("IFIOC_SIMPLE", " ") + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/asyn.cmd + +[2017-08-16 13:53:54] ## defaults for asyn drivers + +[2017-08-16 13:53:54] ## Global default is to trace only errors + +[2017-08-16 13:53:54] ## 0=none,0x1=err,0x2=IO_device,0x4=IO_filter,0x8=IO_driver,0x10=flow,0x20=warning + +[2017-08-16 13:53:54] asynSetTraceMask("", -1, 0x1) + +[2017-08-16 13:53:54] asyn.cmd line 5: Command asynSetTraceMask not found. + +[2017-08-16 13:53:54] ## Global default to send trace output to errlog (missing 3rd arg passes NULL as that parameter) + +[2017-08-16 13:53:54] asynSetTraceFile("", -1) + +[2017-08-16 13:53:54] asyn.cmd line 8: Command asynSetTraceFile not found. + +[2017-08-16 13:53:54] ## Global default is to print escaped ascii for any IO_* trace specified above + +[2017-08-16 13:53:54] ## 0=none,1=ascii,2=esc,4=hex + +[2017-08-16 13:53:54] asynSetTraceIOMask("", -1, 0x2) + +[2017-08-16 13:53:54] asyn.cmd line 12: Command asynSetTraceIOMask not found. +``` +async defaults + +``` +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/ioctesting.cmd + +[2017-08-16 13:53:54] # If in test dev sim mode set up values + +[2017-08-16 13:53:54] epicsEnvSet TESTDEVSIM "#" + +[2017-08-16 13:53:54] # epicsEnvSet "DEVSIM" "1" + +[2017-08-16 13:53:54] # epicsEnvSet "RECSIM" "0" + +[2017-08-16 13:53:54] # stringiftest("DEVSIM", "yes", 2) + +[2017-08-16 13:53:54] # stringiftest("RECSIM", "", 2) + +[2017-08-16 13:53:54] # < C:/Instrument/Var/tmp/test_config.txt + +[2017-08-16 13:53:54] # If in test rec sim mode + +[2017-08-16 13:53:54] epicsEnvSet TESTRECSIM "#" + +[2017-08-16 13:53:54] # epicsEnvSet "DEVSIM" "0" + +[2017-08-16 13:53:54] # epicsEnvSet "RECSIM" "1" + +[2017-08-16 13:53:54] # stringiftest("DEVSIM", "", 2) + +[2017-08-16 13:53:54] # stringiftest("RECSIM", "yes", 2) + +[2017-08-16 13:53:54] # < C:/Instrument/Var/tmp/test_config.txt + +[2017-08-16 13:53:54] ## Create the emulator port macro and set it to either the one given or a spare port: + +[2017-08-16 13:53:54] # freeIPPort(SPARE_PORT) + +[2017-08-16 13:53:54] # epicsEnvSet "EMULATOR_PORT" + +[2017-08-16 13:53:54] # epicsEnvShow("EMULATOR_PORT") +``` +Set up macros for ioc test framework (including loading macros from the temp file if needed) + +``` +[2017-08-16 13:53:54] ## Load record instances + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/dbload.cmd + +[2017-08-16 13:53:54] #dbLoadRecords("$(TIMESTAMPRECORD)/db/timestamp.db","P=$(MYPVPREFIX)$(IOCNAME):") + +[2017-08-16 13:53:54] dbLoadRecords("C:/Instrument/Apps/EPICS/support/devIocStats/master/db/iocAdminSoft.db","IOC=TE:NDW1798:CS:IOC:SIMPLE:DEVIOS") + +[2017-08-16 13:53:54] #dbLoadRecords("$(ASUBFUNCTIONS)/db/iocExit.db","P=$(MYPVPREFIX),Q=CS:IOC:$(IOCNAME):,IOCNAME=$(IOCNAME)") + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/autosave.cmd + +[2017-08-16 13:53:54] ## start autosave.cmd + +[2017-08-16 13:53:54] save_restoreSet_Debug(0) + +[2017-08-16 13:53:54] # status-PV prefix, so save_restore can find its status PV's. + +[2017-08-16 13:53:54] #epicsEnvSet("ASPREFIX","$(MYPVPREFIX)CS:IOC:$(IOCNAME):AS:") + +[2017-08-16 13:53:54] epicsEnvSet("ASPREFIX","TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] save_restoreSet_status_prefix("TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] # Ok to save/restore save sets with missing values (no CA connection to PV)? + +[2017-08-16 13:53:54] save_restoreSet_IncompleteSetsOk(1) + +[2017-08-16 13:53:54] # Save dated backup files? + +[2017-08-16 13:53:54] save_restoreSet_DatedBackupFiles(1) + +[2017-08-16 13:53:54] # Number of sequenced backup files to write + +[2017-08-16 13:53:54] save_restoreSet_NumSeqFiles(1) + +[2017-08-16 13:53:54] # Time interval between sequenced backups + +[2017-08-16 13:53:54] save_restoreSet_SeqPeriodInSeconds(300) + +[2017-08-16 13:53:54] # specify where save files should be - use a different directory if in simulation mode + +[2017-08-16 13:53:54] mkdir("C:/Instrument/Var/autosave/SIMPLE") + +[2017-08-16 13:53:54] set_savefile_path("C:/Instrument/Var/autosave", "SIMPLE") + +[2017-08-16 13:53:54] ## specify what save files should be restored. Note these files must be + +[2017-08-16 13:53:54] ## in the directory specified in set_savefile_path(), or, if that function + +[2017-08-16 13:53:54] ## has not been called, from the directory current when iocInit is invoked + +[2017-08-16 13:53:54] ## example: set_pass0_restoreFile("autosave_geiger.sav") + +[2017-08-16 13:53:54] ## restore positions at pass 0 so not passed to hardware + +[2017-08-16 13:53:54] ## + +[2017-08-16 13:53:54] ## Use epicsEnvSet("AUTOSAVEREQ","#") if you do not have any req files to load + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_positions.sav") + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_settings.sav") + +[2017-08-16 13:53:54] # restore settings at pass 1 so passed to hardware + +[2017-08-16 13:53:54] set_pass1_restoreFile("SIMPLE_settings.sav") + +[2017-08-16 13:53:54] # these values are obtained from info fields in DB files via makeAutosaveFiles() / makeAutosaveFileFromDbInfo() + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_info_positions.sav") + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_info_settings.sav") + +[2017-08-16 13:53:54] set_pass1_restoreFile("SIMPLE_info_settings.sav") + +[2017-08-16 13:53:54] #save_restoreSet_CAReconnect(1) + +[2017-08-16 13:53:54] # specify directories in which to to search for included request files + +[2017-08-16 13:53:54] set_requestfile_path("C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master/iocBoot/iocsimple", "") + +[2017-08-16 13:53:54] # Autosave status PVs - P must be same as used in save_restoreSet_status_prefix() above + +[2017-08-16 13:53:54] dbLoadRecords("C:/Instrument/Apps/EPICS/support/autosave/master/asApp/Db/save_restoreStatus.db", "P=TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] ## end autosave.cmd +``` +setup autosave. + +``` +[2017-08-16 13:53:54] dbLoadRecords("db/simple.db","P=TE:NDW1798:SIMPLE:") +``` +IOC db load of records + +``` +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/preiocinit.cmd + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # write process variable and IOC startup information to database + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] pvdump + +[2017-08-16 13:53:54] pvdump: ioc name is "SIMPLE" pid 11856 + +[2017-08-16 13:53:54] pvdump: MySQL write of 129 PVs with 33 info entries, plus 307 macros took 0.092 seconds + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # set up IOC access security + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] asSetFilename("C:/Instrument/Apps/EPICS/support/AccessSecurity/master/default.acf") + +[2017-08-16 13:53:54] asSetSubstitutions("P=TE:NDW1798:,ACF_IH1=localhost,ACF_IH2=localhost,ACF_IH3=localhost,ACF_IH4=localhost") + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # log to server address EPICS_IOC_LOG_INET listening on port EPICS_IOC_LOG_PORT + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] epicsEnvSet("EPICS_IOC_LOG_INET", "localhost") + +[2017-08-16 13:53:54] iocLogInit() + +[2017-08-16 13:53:54] log client: connected to log server at "127.0.0.1:7004" + +[2017-08-16 13:53:54] cd C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master/iocBoot/iocsimple + +[2017-08-16 13:53:54] iocInit + +[2017-08-16 13:53:54] sevr=info Starting iocInit + +[2017-08-16 13:53:54] ############################################################################ + +[2017-08-16 13:53:54] ## EPICS R3.15.5 + +[2017-08-16 13:53:54] ## EPICS Base built Jul 10 2017 + +[2017-08-16 13:53:54] ############################################################################ + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_positions.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_positions.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_info_positions.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_positions.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_positions.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_positions.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_settings.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav' at initHookState 7 (after record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_settings.sav' at initHookState 7 (after record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] icpconfigLoad: setPVValuesStatic setting 0 pvs (pre iocInit) + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:55] cas warning: Configured TCP port was unavailable. + +[2017-08-16 13:53:55] cas warning: Using dynamically assigned TCP port 60745, + +[2017-08-16 13:53:55] cas warning: but now two or more servers share the same UDP port. + +[2017-08-16 13:53:55] cas warning: Depending on your IP kernel this server may not be + +[2017-08-16 13:53:55] cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST) + +[2017-08-16 13:53:55] sevr=info iocRun: All initialization complete + +[2017-08-16 13:53:55] icpconfigLoad: setPVValues setting 0 pvs (post iocInit) + +[2017-08-16 13:53:55] < C:/Instrument/Apps/EPICS/iocstartup/postiocinit.cmd + +[2017-08-16 13:53:55] # enable caPutLog to logger on localhost + +[2017-08-16 13:53:55] # pass 0 to log value changes, pass 1 to log all puts, 2 logs all puts with no filtering on same PV + +[2017-08-16 13:53:55] caPutLogInit "localhost" "1" + +[2017-08-16 13:53:55] log client: connected to log server at "127.0.0.1:7011" + +[2017-08-16 13:53:55] sevr=info caPutLog: successfully initialized + +[2017-08-16 13:53:55] # Handle autosave 'commands' contained in loaded databases. + +[2017-08-16 13:53:55] # file naming needs to agree with autosave.cmd + +[2017-08-16 13:53:55] makeAutosaveFileFromDbInfo("SIMPLE_info_settings","autosaveFields") + +[2017-08-16 13:53:55] makeAutosaveFileFromDbInfo("SIMPLE_info_positions","autosaveFields_pass0") + +[2017-08-16 13:53:55] create_monitor_set("SIMPLE_info_positions.req", 5, "") + +[2017-08-16 13:53:55] create_monitor_set("SIMPLE_info_settings.req", 30, "") + +[2017-08-16 13:53:55] sevr=info SIMPLE_info_positions.sav: 63 of 63 PV's connected + +[2017-08-16 13:53:55] sevr=info + +[2017-08-16 13:53:56] sevr=info SIMPLE_info_settings.sav: 1 of 1 PV's connected + +[2017-08-16 13:53:56] sevr=info +``` + +Initialise and run + +# Full Log +``` +[2017-08-16 13:53:52] @@@ Restarting child "SIMPLE" +[2017-08-16 13:53:52] @@@ (as C:\windows\system32\cmd.exe) +[2017-08-16 13:53:52] @@@ The PID of new child "SIMPLE" is: 7280 +[2017-08-16 13:53:52] @@@ @@@ @@@ @@@ @@@ +[2017-08-16 13:53:54] #!../../bin/windows-x64/simple + +[2017-08-16 13:53:54] ## You may have to change simple to something else + +[2017-08-16 13:53:54] ## everywhere it appears in this file + +[2017-08-16 13:53:54] < envPaths + +[2017-08-16 13:53:54] epicsEnvSet("IOC","iocsimple") + +[2017-08-16 13:53:54] epicsEnvSet("TOP","C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master") + +[2017-08-16 13:53:54] epicsEnvSet("ACCESSSECURITY","C:/Instrument/Apps/EPICS/support/AccessSecurity/master") + +[2017-08-16 13:53:54] epicsEnvSet("AGILENT33220A","C:/Instrument/Apps/EPICS/support/agilent33220A/master") + +[2017-08-16 13:53:54] epicsEnvSet("AGILENT3631A","C:/Instrument/Apps/EPICS/support/agilent3631A/master") + +[2017-08-16 13:53:54] epicsEnvSet("AMINT2L","C:/Instrument/Apps/EPICS/support/amint2l/master") + +[2017-08-16 13:53:54] epicsEnvSet("AREA_DETECTOR","C:/Instrument/Apps/EPICS/support/areaDetector/master") + +[2017-08-16 13:53:54] epicsEnvSet("ASUBFUNCTIONS","C:/Instrument/Apps/EPICS/support/asubFunctions/master") + +[2017-08-16 13:53:54] epicsEnvSet("ASYN","C:/Instrument/Apps/EPICS/support/asyn/master") + +[2017-08-16 13:53:54] epicsEnvSet("AUTOSAVE","C:/Instrument/Apps/EPICS/support/autosave/master") + +[2017-08-16 13:53:54] epicsEnvSet("AXIS","C:/Instrument/Apps/EPICS/support/axis/master") + +[2017-08-16 13:53:54] epicsEnvSet("BARNDOORS","C:/Instrument/Apps/EPICS/support/barndoors/master") + +[2017-08-16 13:53:54] epicsEnvSet("BOOST","C:/Instrument/Apps/EPICS/libraries/master/boost") + +[2017-08-16 13:53:54] epicsEnvSet("BUSY","C:/Instrument/Apps/EPICS/support/busy/master") + +[2017-08-16 13:53:54] epicsEnvSet("CALC","C:/Instrument/Apps/EPICS/support/calc/master") + +[2017-08-16 13:53:54] epicsEnvSet("CAPUTLOG","C:/Instrument/Apps/EPICS/support/caPutLog/master") + +[2017-08-16 13:53:54] epicsEnvSet("CCD100","C:/Instrument/Apps/EPICS/support/CCD100/master") + +[2017-08-16 13:53:54] epicsEnvSet("COMMON","C:/Instrument/Apps/EPICS/ISIS/Common/master") + +[2017-08-16 13:53:54] epicsEnvSet("CRYVALVE","C:/Instrument/Apps/EPICS/support/cryValve/master") + +[2017-08-16 13:53:54] epicsEnvSet("CSM","C:/Instrument/Apps/EPICS/support/csm/master") + +[2017-08-16 13:53:54] epicsEnvSet("CURL","C:/Instrument/Apps/EPICS/support/curl/master") + +[2017-08-16 13:53:54] epicsEnvSet("DANFYSIK8000","C:/Instrument/Apps/EPICS/support/danfysikMps8000") + +[2017-08-16 13:53:54] epicsEnvSet("DAQMXBASE","C:/Instrument/Apps/EPICS/support/DAQmxBase/master") + +[2017-08-16 13:53:54] epicsEnvSet("DEVIOCSTATS","C:/Instrument/Apps/EPICS/support/devIocStats/master") + +[2017-08-16 13:53:54] epicsEnvSet("ECLAB","C:/Instrument/Apps/EPICS/support/ECLab/master") + +[2017-08-16 13:53:54] epicsEnvSet("EEMCU","C:/Instrument/Apps/EPICS/support/MCAG_Base_Project/master/epics/epicsIOC") + +[2017-08-16 13:53:54] epicsEnvSet("EFSW","C:/Instrument/Apps/EPICS/support/efsw/master") + +[2017-08-16 13:53:54] epicsEnvSet("EUROTHERM2K","C:/Instrument/Apps/EPICS/support/eurotherm2k/master") + +[2017-08-16 13:53:54] epicsEnvSet("FILELIST","C:/Instrument/Apps/EPICS/support/FileList/master") + +[2017-08-16 13:53:54] epicsEnvSet("FILESERVER","C:/Instrument/Apps/EPICS/support/FileServer/master") + +[2017-08-16 13:53:54] epicsEnvSet("FINS","C:/Instrument/Apps/EPICS/support/FINS/master") + +[2017-08-16 13:53:54] epicsEnvSet("FLATBUFFERS","C:/Instrument/Apps/EPICS/support/flatbuffers/master") + +[2017-08-16 13:53:54] epicsEnvSet("GALIL","C:/Instrument/Apps/EPICS/support/galil/master") + +[2017-08-16 13:53:54] epicsEnvSet("HAMEG8123","C:/Instrument/Apps/EPICS/ISIS/Hameg_8123/master") + +[2017-08-16 13:53:54] epicsEnvSet("HIDEWINDOW","C:/Instrument/Apps/EPICS/support/HideWindow/master") + +[2017-08-16 13:53:54] epicsEnvSet("HTMLTIDY","C:/Instrument/Apps/EPICS/support/htmltidy/master") + +[2017-08-16 13:53:54] epicsEnvSet("HVCAEN","C:/Instrument/Apps/EPICS/support/HVCAENx527/master") + +[2017-08-16 13:53:54] epicsEnvSet("ICPCONFIG","C:/Instrument/Apps/EPICS/support/icpconfig/master") + +[2017-08-16 13:53:54] epicsEnvSet("INSTRON","C:/Instrument/Apps/EPICS/support/instron/master") + +[2017-08-16 13:53:54] epicsEnvSet("IP","C:/Instrument/Apps/EPICS/support/ip/master") + +[2017-08-16 13:53:54] epicsEnvSet("IPAC","C:/Instrument/Apps/EPICS/support/ipac/master") + +[2017-08-16 13:53:54] epicsEnvSet("ISISDAE","C:/Instrument/Apps/EPICS/support/isisdae/master") + +[2017-08-16 13:53:54] epicsEnvSet("JAWS","C:/Instrument/Apps/EPICS/support/jaws/master") + +[2017-08-16 13:53:54] epicsEnvSet("JULABO","C:/Instrument/Apps/EPICS/support/julabo/master") + +[2017-08-16 13:53:54] epicsEnvSet("KHLY2400","C:/Instrument/Apps/EPICS/support/Keithley_2400/master") + +[2017-08-16 13:53:54] epicsEnvSet("KEPCO","C:/Instrument/Apps/EPICS/support/kepco/master") + +[2017-08-16 13:53:54] epicsEnvSet("LKSH336","C:/Instrument/Apps/EPICS/support/lakeshore/master/lakeshore336") + +[2017-08-16 13:53:54] epicsEnvSet("LIBICONV","C:/Instrument/Apps/EPICS/support/libiconv/master") + +[2017-08-16 13:53:54] epicsEnvSet("LIBJSON","C:/Instrument/Apps/EPICS/support/libjson/master") + +[2017-08-16 13:53:54] epicsEnvSet("LIBRDKAFKA","C:/Instrument/Apps/EPICS/support/librdkafka/master") + +[2017-08-16 13:53:54] epicsEnvSet("LIBXML2","C:/Instrument/Apps/EPICS/support/libxml2/master") + +[2017-08-16 13:53:54] epicsEnvSet("LIBXSLT","C:/Instrument/Apps/EPICS/support/libxslt/master") + +[2017-08-16 13:53:54] epicsEnvSet("LINKAM95","C:/Instrument/Apps/EPICS/support/linkam95/master") + +[2017-08-16 13:53:54] epicsEnvSet("LVDCOM","C:/Instrument/Apps/EPICS/ISIS/lvDCOM/master") + +[2017-08-16 13:53:54] epicsEnvSet("MCA","C:/Instrument/Apps/EPICS/support/mca/master") + +[2017-08-16 13:53:54] epicsEnvSet("MAGNET3D","C:/Instrument/Apps/EPICS/ISIS/magnet3D/master") + +[2017-08-16 13:53:54] epicsEnvSet("MERCURY_ITC","C:/Instrument/Apps/EPICS/ISIS/MercuryiTC/master") + +[2017-08-16 13:53:54] epicsEnvSet("MK2CHOPR","C:/Instrument/Apps/EPICS/support/mk2chopper/master") + +[2017-08-16 13:53:54] epicsEnvSet("MODBUS","C:/Instrument/Apps/EPICS/support/modbus/master") + +[2017-08-16 13:53:54] epicsEnvSet("MOTIONSETPOINTS","C:/Instrument/Apps/EPICS/support/motionSetPoints/master") + +[2017-08-16 13:53:54] epicsEnvSet("MOTOR","C:/Instrument/Apps/EPICS/support/motor/master") + +[2017-08-16 13:53:54] epicsEnvSet("MYSQL","C:/Instrument/Apps/EPICS/support/MySQL/master") + +[2017-08-16 13:53:54] epicsEnvSet("NEOCERA","C:/Instrument/Apps/EPICS/support/neocera/master") + +[2017-08-16 13:53:54] epicsEnvSet("NETSHRVAR","C:/Instrument/Apps/EPICS/support/NetShrVar/master") + +[2017-08-16 13:53:54] epicsEnvSet("NANODAC","C:/Instrument/Apps/EPICS/support/nanodac/master") + +[2017-08-16 13:53:54] epicsEnvSet("NULLHTTPD","C:/Instrument/Apps/EPICS/support/nullhttpd/master") + +[2017-08-16 13:53:54] epicsEnvSet("OPENSSL","C:/Instrument/Apps/EPICS/support/OpenSSL/master") + +[2017-08-16 13:53:54] epicsEnvSet("OPTICS","C:/Instrument/Apps/EPICS/support/optics/master") + +[2017-08-16 13:53:54] epicsEnvSet("PCRE","C:/Instrument/Apps/EPICS/support/pcre/master") + +[2017-08-16 13:53:54] epicsEnvSet("PDR2000","C:/Instrument/Apps/EPICS/support/pdr2000/master") + +[2017-08-16 13:53:54] epicsEnvSet("PIXELMAN","C:/Instrument/Apps/EPICS/support/pixelman/master") + +[2017-08-16 13:53:54] epicsEnvSet("PROCSERVCONTROL","C:/Instrument/Apps/EPICS/support/procServControl/master") + +[2017-08-16 13:53:54] epicsEnvSet("PUGIXML","C:/Instrument/Apps/EPICS/support/pugixml/master") + +[2017-08-16 13:53:54] epicsEnvSet("PVCOMPLETE","C:/Instrument/Apps/EPICS/support/pvcomplete/master") + +[2017-08-16 13:53:54] epicsEnvSet("PVDUMP","C:/Instrument/Apps/EPICS/support/pvdump/master") + +[2017-08-16 13:53:54] epicsEnvSet("READASCII","C:/Instrument/Apps/EPICS/support/ReadASCII/master") + +[2017-08-16 13:53:54] epicsEnvSet("ROTSC","C:/Instrument/Apps/EPICS/support/rotating_sample_changer/master") + +[2017-08-16 13:53:54] epicsEnvSet("RUNCONTROL","C:/Instrument/Apps/EPICS/support/RunControl/master") + +[2017-08-16 13:53:54] epicsEnvSet("RANDOM","C:/Instrument/Apps/EPICS/support/random/master") + +[2017-08-16 13:53:54] epicsEnvSet("SAMPLECHANGER","C:/Instrument/Apps/EPICS/support/sampleChanger/master") + +[2017-08-16 13:53:54] epicsEnvSet("SLACKING","C:/Instrument/Apps/EPICS/libraries/master/slacking") + +[2017-08-16 13:53:54] epicsEnvSet("SNCSEQ","C:/Instrument/Apps/EPICS/support/seq/master") + +[2017-08-16 13:53:54] epicsEnvSet("SKFCHOPPER","C:/Instrument/Apps/EPICS/support/SKFChopper/master") + +[2017-08-16 13:53:54] epicsEnvSet("SPRLG","C:/Instrument/Apps/EPICS/support/superlogics/master") + +[2017-08-16 13:53:54] epicsEnvSet("SQLITE","C:/Instrument/Apps/EPICS/support/sqlite/master") + +[2017-08-16 13:53:54] epicsEnvSet("SSCAN","C:/Instrument/Apps/EPICS/support/sscan/master") + +[2017-08-16 13:53:54] epicsEnvSet("STD","C:/Instrument/Apps/EPICS/support/std/master") + +[2017-08-16 13:53:54] epicsEnvSet("STPS350","C:/Instrument/Apps/EPICS/ISIS/Stanford_PS350/master") + +[2017-08-16 13:53:54] epicsEnvSet("STSR400","C:/Instrument/Apps/EPICS/ISIS/Stanford_SR400/master") + +[2017-08-16 13:53:54] epicsEnvSet("STREAMDEVICE","C:/Instrument/Apps/EPICS/support/StreamDevice/master") + +[2017-08-16 13:53:54] epicsEnvSet("TDKLAMBDAGENESYS","C:/Instrument/Apps/EPICS/support/TDKLambdaGenesys/master") + +[2017-08-16 13:53:54] epicsEnvSet("TEKDMM40X0","C:/Instrument/Apps/EPICS/support/Tektronix_DMM_40X0/master") + +[2017-08-16 13:53:54] epicsEnvSet("TEKAFG3XXX","C:/Instrument/Apps/EPICS/support/Tektronix_AFG3XXX/master") + +[2017-08-16 13:53:54] epicsEnvSet("TEKMSO4104B","C:/Instrument/Apps/EPICS/support/Tektronix_MSO_4104B/master") + +[2017-08-16 13:53:54] epicsEnvSet("TINYXML","C:/Instrument/Apps/EPICS/support/TinyXML/master") + +[2017-08-16 13:53:54] epicsEnvSet("TPG300","C:/Instrument/Apps/EPICS/support/TPG/master") + +[2017-08-16 13:53:54] epicsEnvSet("TPG","C:/Instrument/Apps/EPICS/support/TPG/master") + +[2017-08-16 13:53:54] epicsEnvSet("TTIEX355P","C:/Instrument/Apps/EPICS/support/ttiEX355P/master") + +[2017-08-16 13:53:54] epicsEnvSet("UTILITIES","C:/Instrument/Apps/EPICS/support/utilities/master") + +[2017-08-16 13:53:54] epicsEnvSet("VISADRV","C:/Instrument/Apps/EPICS/support/VISAdrv/master") + +[2017-08-16 13:53:54] epicsEnvSet("WEBGET","C:/Instrument/Apps/EPICS/support/webget/master") + +[2017-08-16 13:53:54] epicsEnvSet("ZLIB","C:/Instrument/Apps/EPICS/support/zlib/master") + +[2017-08-16 13:53:54] epicsEnvSet("EV4_BASE","C:/Instrument/Apps/EPICS/support/EPICS_V4/master") + +[2017-08-16 13:53:54] epicsEnvSet("PVDATABASE","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvDatabaseCPP") + +[2017-08-16 13:53:54] epicsEnvSet("PVASRV","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvaSrv") + +[2017-08-16 13:53:54] epicsEnvSet("PVACLIENT","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvaClientCPP") + +[2017-08-16 13:53:54] epicsEnvSet("PVACCESS","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvAccessCPP") + +[2017-08-16 13:53:54] epicsEnvSet("NORMATIVETYPES","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/normativeTypesCPP") + +[2017-08-16 13:53:54] epicsEnvSet("PVDATA","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvDataCPP") + +[2017-08-16 13:53:54] epicsEnvSet("PVCOMMON","C:/Instrument/Apps/EPICS/support/EPICS_V4/master/pvCommonCPP") + +[2017-08-16 13:53:54] epicsEnvSet("MK3CHOPPER","C:/Instrument/Apps/EPICS/support/mk3chopper/master") + +[2017-08-16 13:53:54] epicsEnvSet("ONCRPC","C:/Instrument/Apps/EPICS/support/oncrpc/master") + +[2017-08-16 13:53:54] epicsEnvSet("EPICS_BASE","C:/Instrument/Apps/EPICS/base/master") + +[2017-08-16 13:53:54] epicsEnvSet("EPICS_ROOT","C:/Instrument/Apps/EPICS") + +[2017-08-16 13:53:54] epicsEnvSet("SUPPORT","C:/Instrument/Apps/EPICS/support") + +[2017-08-16 13:53:54] epicsEnvSet("ISISSUPPORT","C:/Instrument/Apps/EPICS/ISIS") + +[2017-08-16 13:53:54] epicsEnvSet("IOCSTARTUP","C:/Instrument/Apps/EPICS/iocstartup") + +[2017-08-16 13:53:54] epicsEnvSet("ICPBINARYDIR","C:/Instrument/Apps/EPICS/ICP_Binaries") + +[2017-08-16 13:53:54] cd C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master + +[2017-08-16 13:53:54] ## Register all support components + +[2017-08-16 13:53:54] dbLoadDatabase "dbd/simple.dbd" + +[2017-08-16 13:53:54] simple_registerRecordDeviceDriver pdbbase + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/init.cmd + +[2017-08-16 13:53:54] icpconfigLoad + +[2017-08-16 13:53:54] icpconfigLoad: ioc "SIMPLE" group "SIMPLE" options 0x0 host "NDW1798" + +[2017-08-16 13:53:54] icpconfigLoad: config base (ICPCONFIGBASE) is "C:/Instrument/Settings/config" + +[2017-08-16 13:53:54] icpconfigLoad: config root (ICPCONFIGROOT) is "C:/Instrument/Settings/config/NDW1798/configurations" + +[2017-08-16 13:53:54] icpconfigLoad: * $(SIMULATE)="0" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFSIM)="#" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTSIM)=" " ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(SIMSFX)="" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(DISABLE)="0" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFDISABLE)="#" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTDISABLE)=" " ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(DEVSIM)="0" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFDEVSIM)="#" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTDEVSIM)=" " ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(RECSIM)="0" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFRECSIM)="#" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(IFNOTRECSIM)=" " ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: * $(ICPCONFIGDIR)="C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron" ({initial default}) + +[2017-08-16 13:53:54] icpconfigLoad: last configuration was "Instron" (C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron) + +[2017-08-16 13:53:54] icpconfigLoad: configuration "Instron" + +[2017-08-16 13:53:54] icpconfigLoad: loading 0 component(s) for "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: Loading default macros for "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC sim level "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC macros for "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC PVs for "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: Loading IOC PV sets for "/configurations/Instron" + +[2017-08-16 13:53:54] Cannot open directory: C:/Instrument/Settings/config/NDW1798/configurations/configurations/Instron/files + +[2017-08-16 13:53:54] icpconfigLoad: Found 0 files for "/configurations/Instron" + +[2017-08-16 13:53:54] icpconfigLoad: loading old macro file "C:/Instrument/Settings/config/NDW1798/configurations/globals.txt" + +[2017-08-16 13:53:54] icpconfigLoad: * $(GALILNUMCRATES)="1" (C:/Instrument/Settings/config/NDW1798/configurations/globals.txt) + +[2017-08-16 13:53:54] icpconfigLoad: * $(GALIL_01__GALILADDR01)="130.246.51.169" (C:/Instrument/Settings/config/NDW1798/configurations/globals.txt) + +[2017-08-16 13:53:54] icpconfigLoad: * $(GALIL_02__GALILADDR02)="None" (C:/Instrument/Settings/config/NDW1798/configurations/globals.txt) + +[2017-08-16 13:53:54] icpconfigLoad: * $(GALIL_03__GALILADDR03)="None" (C:/Instrument/Settings/config/NDW1798/configurations/globals.txt) + +[2017-08-16 13:53:54] icpconfigLoad: loaded 4 macros from old macro file "C:/Instrument/Settings/config/NDW1798/configurations/globals.txt" (0 ioc, 0 group) + +[2017-08-16 13:53:54] # define a macro with the same name as the IOC name and a blank value. This means we can use a line like + +[2017-08-16 13:53:54] # $(IFIOC_GALIL_01=#) to have something that is only executed in GALIL_01 IOC + +[2017-08-16 13:53:54] # and you can pass to db file via A=1,$(IFIOC)= ,A=2 + +[2017-08-16 13:53:54] # used when loading motorUtils and motors moving + +[2017-08-16 13:53:54] epicsEnvSet("IFIOC", "IFIOC_SIMPLE") + +[2017-08-16 13:53:54] epicsEnvSet("IFIOC_SIMPLE", " ") + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/asyn.cmd + +[2017-08-16 13:53:54] ## defaults for asyn drivers + +[2017-08-16 13:53:54] ## Global default is to trace only errors + +[2017-08-16 13:53:54] ## 0=none,0x1=err,0x2=IO_device,0x4=IO_filter,0x8=IO_driver,0x10=flow,0x20=warning + +[2017-08-16 13:53:54] asynSetTraceMask("", -1, 0x1) + +[2017-08-16 13:53:54] asyn.cmd line 5: Command asynSetTraceMask not found. + +[2017-08-16 13:53:54] ## Global default to send trace output to errlog (missing 3rd arg passes NULL as that parameter) + +[2017-08-16 13:53:54] asynSetTraceFile("", -1) + +[2017-08-16 13:53:54] asyn.cmd line 8: Command asynSetTraceFile not found. + +[2017-08-16 13:53:54] ## Global default is to print escaped ascii for any IO_* trace specified above + +[2017-08-16 13:53:54] ## 0=none,1=ascii,2=esc,4=hex + +[2017-08-16 13:53:54] asynSetTraceIOMask("", -1, 0x2) + +[2017-08-16 13:53:54] asyn.cmd line 12: Command asynSetTraceIOMask not found. + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/ioctesting.cmd + +[2017-08-16 13:53:54] # If in test dev sim mode set up values + +[2017-08-16 13:53:54] epicsEnvSet TESTDEVSIM "#" + +[2017-08-16 13:53:54] # epicsEnvSet "DEVSIM" "1" + +[2017-08-16 13:53:54] # epicsEnvSet "RECSIM" "0" + +[2017-08-16 13:53:54] # stringiftest("DEVSIM", "yes", 2) + +[2017-08-16 13:53:54] # stringiftest("RECSIM", "", 2) + +[2017-08-16 13:53:54] # < C:/Instrument/Var/tmp/test_config.txt + +[2017-08-16 13:53:54] # If in test rec sim mode + +[2017-08-16 13:53:54] epicsEnvSet TESTRECSIM "#" + +[2017-08-16 13:53:54] # epicsEnvSet "DEVSIM" "0" + +[2017-08-16 13:53:54] # epicsEnvSet "RECSIM" "1" + +[2017-08-16 13:53:54] # stringiftest("DEVSIM", "", 2) + +[2017-08-16 13:53:54] # stringiftest("RECSIM", "yes", 2) + +[2017-08-16 13:53:54] # < C:/Instrument/Var/tmp/test_config.txt + +[2017-08-16 13:53:54] ## Create the emulator port macro and set it to either the one given or a spare port: + +[2017-08-16 13:53:54] # freeIPPort(SPARE_PORT) + +[2017-08-16 13:53:54] # epicsEnvSet "EMULATOR_PORT" + +[2017-08-16 13:53:54] # epicsEnvShow("EMULATOR_PORT") + +[2017-08-16 13:53:54] ## Load record instances + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/dbload.cmd + +[2017-08-16 13:53:54] #dbLoadRecords("$(TIMESTAMPRECORD)/db/timestamp.db","P=$(MYPVPREFIX)$(IOCNAME):") + +[2017-08-16 13:53:54] dbLoadRecords("C:/Instrument/Apps/EPICS/support/devIocStats/master/db/iocAdminSoft.db","IOC=TE:NDW1798:CS:IOC:SIMPLE:DEVIOS") + +[2017-08-16 13:53:54] #dbLoadRecords("$(ASUBFUNCTIONS)/db/iocExit.db","P=$(MYPVPREFIX),Q=CS:IOC:$(IOCNAME):,IOCNAME=$(IOCNAME)") + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/autosave.cmd + +[2017-08-16 13:53:54] ## start autosave.cmd + +[2017-08-16 13:53:54] save_restoreSet_Debug(0) + +[2017-08-16 13:53:54] # status-PV prefix, so save_restore can find its status PV's. + +[2017-08-16 13:53:54] #epicsEnvSet("ASPREFIX","$(MYPVPREFIX)CS:IOC:$(IOCNAME):AS:") + +[2017-08-16 13:53:54] epicsEnvSet("ASPREFIX","TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] save_restoreSet_status_prefix("TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] # Ok to save/restore save sets with missing values (no CA connection to PV)? + +[2017-08-16 13:53:54] save_restoreSet_IncompleteSetsOk(1) + +[2017-08-16 13:53:54] # Save dated backup files? + +[2017-08-16 13:53:54] save_restoreSet_DatedBackupFiles(1) + +[2017-08-16 13:53:54] # Number of sequenced backup files to write + +[2017-08-16 13:53:54] save_restoreSet_NumSeqFiles(1) + +[2017-08-16 13:53:54] # Time interval between sequenced backups + +[2017-08-16 13:53:54] save_restoreSet_SeqPeriodInSeconds(300) + +[2017-08-16 13:53:54] # specify where save files should be - use a different directory if in simulation mode + +[2017-08-16 13:53:54] mkdir("C:/Instrument/Var/autosave/SIMPLE") + +[2017-08-16 13:53:54] set_savefile_path("C:/Instrument/Var/autosave", "SIMPLE") + +[2017-08-16 13:53:54] ## specify what save files should be restored. Note these files must be + +[2017-08-16 13:53:54] ## in the directory specified in set_savefile_path(), or, if that function + +[2017-08-16 13:53:54] ## has not been called, from the directory current when iocInit is invoked + +[2017-08-16 13:53:54] ## example: set_pass0_restoreFile("autosave_geiger.sav") + +[2017-08-16 13:53:54] ## restore positions at pass 0 so not passed to hardware + +[2017-08-16 13:53:54] ## + +[2017-08-16 13:53:54] ## Use epicsEnvSet("AUTOSAVEREQ","#") if you do not have any req files to load + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_positions.sav") + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_settings.sav") + +[2017-08-16 13:53:54] # restore settings at pass 1 so passed to hardware + +[2017-08-16 13:53:54] set_pass1_restoreFile("SIMPLE_settings.sav") + +[2017-08-16 13:53:54] # these values are obtained from info fields in DB files via makeAutosaveFiles() / makeAutosaveFileFromDbInfo() + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_info_positions.sav") + +[2017-08-16 13:53:54] set_pass0_restoreFile("SIMPLE_info_settings.sav") + +[2017-08-16 13:53:54] set_pass1_restoreFile("SIMPLE_info_settings.sav") + +[2017-08-16 13:53:54] #save_restoreSet_CAReconnect(1) + +[2017-08-16 13:53:54] # specify directories in which to to search for included request files + +[2017-08-16 13:53:54] set_requestfile_path("C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master/iocBoot/iocsimple", "") + +[2017-08-16 13:53:54] # Autosave status PVs - P must be same as used in save_restoreSet_status_prefix() above + +[2017-08-16 13:53:54] dbLoadRecords("C:/Instrument/Apps/EPICS/support/autosave/master/asApp/Db/save_restoreStatus.db", "P=TE:NDW1798:AS:SIMPLE:") + +[2017-08-16 13:53:54] ## end autosave.cmd + +[2017-08-16 13:53:54] dbLoadRecords("db/simple.db","P=TE:NDW1798:SIMPLE:") + +[2017-08-16 13:53:54] < C:/Instrument/Apps/EPICS/iocstartup/preiocinit.cmd + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # write process variable and IOC startup information to database + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] pvdump + +[2017-08-16 13:53:54] pvdump: ioc name is "SIMPLE" pid 11856 + +[2017-08-16 13:53:54] pvdump: MySQL write of 129 PVs with 33 info entries, plus 307 macros took 0.092 seconds + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # set up IOC access security + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] asSetFilename("C:/Instrument/Apps/EPICS/support/AccessSecurity/master/default.acf") + +[2017-08-16 13:53:54] asSetSubstitutions("P=TE:NDW1798:,ACF_IH1=localhost,ACF_IH2=localhost,ACF_IH3=localhost,ACF_IH4=localhost") + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] # log to server address EPICS_IOC_LOG_INET listening on port EPICS_IOC_LOG_PORT + +[2017-08-16 13:53:54] # + +[2017-08-16 13:53:54] epicsEnvSet("EPICS_IOC_LOG_INET", "localhost") + +[2017-08-16 13:53:54] iocLogInit() + +[2017-08-16 13:53:54] log client: connected to log server at "127.0.0.1:7004" + +[2017-08-16 13:53:54] cd C:/Instrument/Apps/EPICS/ISIS/SimpleIoc/master/iocBoot/iocsimple + +[2017-08-16 13:53:54] iocInit + +[2017-08-16 13:53:54] sevr=info Starting iocInit + +[2017-08-16 13:53:54] ############################################################################ + +[2017-08-16 13:53:54] ## EPICS R3.15.5 + +[2017-08-16 13:53:54] ## EPICS Base built Jul 10 2017 + +[2017-08-16 13:53:54] ############################################################################ + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_positions.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_positions.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_positions.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_info_positions.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_positions.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_positions.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_positions.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_settings.sav' at initHookState 6 (before record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info reboot_restore: entry for file 'SIMPLE_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav' at initHookState 7 (after record/device init) *** + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.savB'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't figure out which seq file is most recent, + +[2017-08-16 13:53:54] sevr=info save_restore: so I'm just going to start with 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Trying backup file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0' + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open file 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. + +[2017-08-16 13:53:54] sevr=info save_restore: Can't find a file to restore from...sevr=info save_restore: ...last tried 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_settings.sav0'. I give up. + +[2017-08-16 13:53:54] save_restore: ********************************** + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] sevr=info save_restore: Can't open save file.sevr=info reboot_restore: entry for file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] sevr=info reboot_restore: Found filename 'SIMPLE_info_settings.sav' in restoreFileList. + +[2017-08-16 13:53:54] sevr=info *** restoring from 'C:/Instrument/Var/autosave/SIMPLE/SIMPLE_info_settings.sav' at initHookState 7 (after record/device init) *** + +[2017-08-16 13:53:54] sevr=info reboot_restore: done with file 'SIMPLE_info_settings.sav' + +[2017-08-16 13:53:54] + +[2017-08-16 13:53:54] icpconfigLoad: setPVValuesStatic setting 0 pvs (pre iocInit) + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:54] sevr=info Duplicated host 'localhost' in HAG 'instmachine' + +[2017-08-16 13:53:55] cas warning: Configured TCP port was unavailable. + +[2017-08-16 13:53:55] cas warning: Using dynamically assigned TCP port 60745, + +[2017-08-16 13:53:55] cas warning: but now two or more servers share the same UDP port. + +[2017-08-16 13:53:55] cas warning: Depending on your IP kernel this server may not be + +[2017-08-16 13:53:55] cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST) + +[2017-08-16 13:53:55] sevr=info iocRun: All initialization complete + +[2017-08-16 13:53:55] icpconfigLoad: setPVValues setting 0 pvs (post iocInit) + +[2017-08-16 13:53:55] < C:/Instrument/Apps/EPICS/iocstartup/postiocinit.cmd + +[2017-08-16 13:53:55] # enable caPutLog to logger on localhost + +[2017-08-16 13:53:55] # pass 0 to log value changes, pass 1 to log all puts, 2 logs all puts with no filtering on same PV + +[2017-08-16 13:53:55] caPutLogInit "localhost" "1" + +[2017-08-16 13:53:55] log client: connected to log server at "127.0.0.1:7011" + +[2017-08-16 13:53:55] sevr=info caPutLog: successfully initialized + +[2017-08-16 13:53:55] # Handle autosave 'commands' contained in loaded databases. + +[2017-08-16 13:53:55] # file naming needs to agree with autosave.cmd + +[2017-08-16 13:53:55] makeAutosaveFileFromDbInfo("SIMPLE_info_settings","autosaveFields") + +[2017-08-16 13:53:55] makeAutosaveFileFromDbInfo("SIMPLE_info_positions","autosaveFields_pass0") + +[2017-08-16 13:53:55] create_monitor_set("SIMPLE_info_positions.req", 5, "") + +[2017-08-16 13:53:55] create_monitor_set("SIMPLE_info_settings.req", 30, "") + +[2017-08-16 13:53:55] sevr=info SIMPLE_info_positions.sav: 63 of 63 PV's connected + +[2017-08-16 13:53:55] sevr=info + +[2017-08-16 13:53:56] sevr=info SIMPLE_info_settings.sav: 1 of 1 PV's connected + +[2017-08-16 13:53:56] sevr=info +``` \ No newline at end of file diff --git "a/doc/iocs/troubleshooting/IOC-crash-or-unexpected-termination-\342\200\220-Find-function-from-windows-module-offset.md" "b/doc/iocs/troubleshooting/IOC-crash-or-unexpected-termination-\342\200\220-Find-function-from-windows-module-offset.md" new file mode 100644 index 000000000..be169c34d --- /dev/null +++ "b/doc/iocs/troubleshooting/IOC-crash-or-unexpected-termination-\342\200\220-Find-function-from-windows-module-offset.md" @@ -0,0 +1,20 @@ +# IOC Crash - find function from offset + +This is following a bit on from https://devblogs.microsoft.com/oldnewthing/20160302-00/?p=93103 +``` +Faulting application name: JULABO-IOC-01.exe, version: 0.0.0.0, time stamp: 0x65a785c3 Faulting module name: asyn.dll, version: 0.0.0.0, time stamp: 0x65a75e35 Exception code: 0xc0000005 Fault offset: 0x00000000000066c0 Faulting process ID: 0x1b0c0 Faulting application start time: 0x01dac65ace0ce665 Faulting application path: C:\Instrument\Apps\EPICS\ioc\master\JULABO\bin\windows-x64\JULABO-IOC-01.exe Faulting module path: C:\INSTRU~1\Apps\EPICS\support\asyn\master\bin\windows-x64\asyn.dll Report ID: 772b96c7-21cc-4753-9db9-cea1b0655858 Faulting package full name: Faulting package-relative application ID: +``` +so we want to find out what is at offset `0x66c0` in `asyn.dll` + +* vnc onto instrument +* create cmd window and cd to iocBoot for ioc +* run `dllpath.bat` +* now run `C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe` +* file -> open executable and browse to ioc exe in bin +* look at debug ->modules window and note load address of DLL of interest e.g. `00007ffeb6da0000` +* if offset was `0x66c0` then type `u 0x00007ffeb6da0000 + 0x66c0 L1` and you'll get something like +``` +0:000> u 0x00007ffeb6da0000 + 0x66c0 l1 +asyn!findDpCommon [C:\Instrument\Apps\EPICS\support\asyn\master\asyn\asynDriver\asynManager.c @ 2620] [inlined in asyn!getTraceMask [C:\Instrument\Apps\EPICS\support\asyn\master\asyn\asynDriver\asynManager.c @ 2620]]: +00007ffe`b6da66c0 488b41e0 mov rax,qword ptr [rcx-20h] +``` diff --git a/doc/iocs/troubleshooting/IOC-flow-control-settings.md b/doc/iocs/troubleshooting/IOC-flow-control-settings.md new file mode 100644 index 000000000..dfc1f180d --- /dev/null +++ b/doc/iocs/troubleshooting/IOC-flow-control-settings.md @@ -0,0 +1,89 @@ +# IOC Flow Control Settings + +| IOC | Software
flow control | Hardware
flow control | Reference document
(VI/manual) | Ticket | +|:------------:|:---------------------:|:---------------------:|:------------------------------:|:------:| +| AG3631A | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| AG33220A | TCP | TCP | IOC | | +| AG53220A | TCP | TCP | VI | | +| AMINT2L | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| BKHOFF | TCP | TCP | IOC | | +| CCD100 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| CONEXAGP | On | Off | IOC | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| CONTROLSVCS | N/A | N/A | | | +| COORD | N/A | N/A | | | +| COUETTE | LVDCOM | LVDCOM | IOC | | +| CRYVALVE | Off | Off | VI and docs | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| CYBAMAN | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| DELFTARDUSTEP | LVDCOM | LVDCOM | IOC | | +| DELFTBPMAG | Unknown | | | | +| DELFTDCMAG | Unknown | Unknown | | | +| DELFTSHEAR | LVDCOM | LVDCOM | IOC | | +| DFKPS | Off | Off | VIs | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| ECLAB | N/A | N/A | | | +| EGXCOLIM | LVDCOM | LVDCOM | IOC | | +| EUROTHRM | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| FERMCHOP | Off | Off | Manual and IOC | | +| FINS | N/A | N/A | IOC | | +| FZJDDFCH | TCP | TCP | IOC | | +| GALIL (Ethernet) | TCP | TCP | IOC | | +| GALIL (Serial) | Off | On | IOC | | +| GAMRY | Off | Off | IOC | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| GEMORC | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| HAMEG8123 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| HIFIMAG | LVDCOM | LVDCOM | IOC | | +| HLG | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| HVCAEN | N/A | N/A | IOC | | +| IEG | Off | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| ILM200 | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| INHIBITR | N/A | N/A | IOC | | +| INSTETC | N/A | N/A | IOC | | +| INSTRON | TCP | TCP | IOC | | +| IPS | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| ISISDAE | N/A | N/A | IOC | | +| ITC503 | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| JULABO | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| KEPCO | On | Off | VI | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| KHLY2400 | **Macro** | **Macro** | VI/Manual | | +| KHLY2700 | **Macro** | **Macro** | VI/Manual | | +| KYNCTM3K | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| LINKAM95 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| LINMOT | Off | Off | Manual and VI | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| LKSH218 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| LKSH336 | TCP | TCP | IOC | | +| LKSH460 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| MCLEN | On | Off | IOC | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| MERCURY_ITC | LVDCOM | LVDCOM | IOC | | +| MK2CHOPR | Off | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| MK3CHOPR | Remote procedure calls - not serial | | | | +| NANODAC | TCP | TCP | IOC | | +| NEOCERA | On | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| PDR2000 | Off | Off | VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| PIMOT | On | Off | IOC | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| PSCTRL | N/A | N/A | | | +| RKNDIO | Off | Off | IOC | | +| RKNPS | Off | Off | Danfysik 8k VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| ROTSC | Off | Off | Manual and VIs
(HRPD and POLARIS) | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| RUNCTRL | N/A | N/A | | | +| SAMPOS | LVDCOM | LVDCOM | IOC | | +| SCHNDR | On | Off | IOC | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| SCIMAG3D | LVDCOM | LVDCOM | IOC | | +| SDTEST | Macro | Macro | IOC | | +| SECI2IBEX | LVDCOM | LVDCOM | IOC | | +| SKFCHOPPER | IP | IP | IOC | | +| SKFMB350 | Off | Off | VI (manual?) | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| SM300 | Off | Off | VI | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| SMC100 | On | Off | Manual | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| SP2XX | Off | Off | Manual and VI | | +| SPINFLIPPER306015 | Off | Off | Manual | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| SPRLG | Off | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| STPS350 | LVDCOM | LVDCOM | IOC | | +| STSR400 | LVDCOM | LVDCOM | IOC | | +| TDK_LAMBDA_GENESYS | Off | Off | Manual | [3139](https://github.com/ISISComputingGroup/IBEX/issues/3139) | +| TEKAFG3XXX | VXI-11 | VXI-11 | IOC | | +| TEKDMM40X0 | Off | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| TEKMSO4104B | VXI-11 | VXI-11 | IOC | | +| TPG26x | Off | Off |Manual and VI| [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| TPG300 | Off | Off |Manual and VI| [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| TRITON | IP | IP | IOC | | +| TTIEX355P | Off | Off | Manual and VI | [3180](https://github.com/ISISComputingGroup/IBEX/issues/3180) | +| **IOC** | **Software
flow control** | **Hardware
flow control** | **Reference document
(VI/manual)** | **Ticket** | \ No newline at end of file diff --git a/doc/iocs/troubleshooting/Log-file-location.md b/doc/iocs/troubleshooting/Log-file-location.md new file mode 100644 index 000000000..4578d575e --- /dev/null +++ b/doc/iocs/troubleshooting/Log-file-location.md @@ -0,0 +1,10 @@ +# Log file locations + +Various log file locations: + +log file | location | what it shows +------ | ------- | ------------- +genie_python | `..\Instrument\Var\logs\genie_python\genie-YYYY-MM-DD.log` | Every command that genie python executes (log is written before command is executed) +ISIS DAE IOC Log | `..\Instrument\Var\logs\ioc\ISISDAE_01-YYYMMDD.log` | Shows what ISIS DAE IOC, this is controlling the isisicp +ISIS ICP Log | `c\Data\Export only\logs\icp\log\icp-YYYY-MM-DD-Ddd.log` | Shows what the isis icp data collection program is doing +ISIS ICP Putlog (Put log) | `C:\data\[instrument name][run number]_ICPputlog.txt` | Shows caputs to PV's including their new and old values \ No newline at end of file diff --git a/doc/iocs/troubleshooting/Remote-Debugging-from-Visual-Studio.md b/doc/iocs/troubleshooting/Remote-Debugging-from-Visual-Studio.md new file mode 100644 index 000000000..b1bc539af --- /dev/null +++ b/doc/iocs/troubleshooting/Remote-Debugging-from-Visual-Studio.md @@ -0,0 +1,17 @@ +# Remote Debugging from Visual Studio + +Remote debugging allows you to debug in visual studio c and c# applications running on another machine. To do this: + +1. On the remote machine run the visual studio remote debugging (x64) tool for your version of visual studio. It can be downloaded from [microsoft](https://docs.microsoft.com/en-gb/visualstudio/debugger/remote-debugging) but may be there already. You can also run the one at `ISIS_Experiment_Controls_Public\VS_Remote_Debuggers\VS2017\x64` +1. Set the tool to allow no authentication (Tools->Options No Authentication) +1. Set the environment variable on your local machine + `_NT_SYMBOL_PATH=srv*\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\Releases\Symbols;srv*\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\EPICS\Symbols;srv*\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\ISISICP\Symbols;srv*c:\MyServerSymbols*https://msdl.microsoft.com/download/symbols` +1. Run VS on your local machine +1. Select Debug -> Attach to Process +1. Set Connection Type Remote(no authentication) and use the server name in connection target +1. Select the process from the list + 1. If you need to catch an error in ioc startup, start the ioc with `runIOC.bat aaa` this will cause it to start up with running st.cmd. + 1. Once you are connected and have break points setup then type `< st.cmd` to boot the ioc +1. Open the Modules windows (Debug -> Windows -> Modules), find a module you are interested in right click and do load symbols. I am a bit hazy on how this works Freddie knows more. If it works for you please put how you did it here. +1. You should now be a able to open code and put breakpoints in as usual. + diff --git a/doc/iocs/troubleshooting/Serial-Bruteforce-script.md b/doc/iocs/troubleshooting/Serial-Bruteforce-script.md new file mode 100644 index 000000000..74cdcd5b6 --- /dev/null +++ b/doc/iocs/troubleshooting/Serial-Bruteforce-script.md @@ -0,0 +1,35 @@ +# Serial Bruteforce Script + +The following is a Python script that can be run against devices with unknown comms settings. It will attempt to get a reply out of a device using all available comms settings. + +Note that the script may only get a single character from the device, even if the device responds with many characters. It is likely that several serial settings will reply - further analysis of the device will be required to determine the proper set (e.g. many devices will still give replies even with an incorrect parity or stop bits setting). + +You will need to run `python -m pip install pyserial` before running this script. + +```python +import serial + + +COMMAND = b"1OS\r" +PORT = "COM1" + +def try_send(baud, stop, bits, parity, xonxoff): + try: + with serial.Serial(PORT, stopbits=stop, bytesize=bits, baudrate=baud, parity=parity, xonxoff=xonxoff, timeout=0.5) as p: + print(f"trying {baud} {stop} {bits} {parity} {xonxoff}") + p.write(COMMAND) + print(p.read()) + except serial.SerialException as e: + pass + except Exception as e: + print(e) + + +def force(): + for baud in [9600, 19200, 38400, 57600, 115200, 4800, 2400]: + for stop in [serial.STOPBITS_ONE, serial.STOPBITS_ONE_POINT_FIVE, serial.STOPBITS_TWO]: + for bits in [serial.FIVEBITS, serial.SIXBITS, serial.SEVENBITS, serial.EIGHTBITS]: + for parity in [serial.PARITY_NONE, serial.PARITY_EVEN, serial.PARITY_ODD, serial.PARITY_MARK, serial.PARITY_SPACE]: + for xonxoff in [True, False]: + try_send(baud, stop, bits, parity, xonxoff) +``` \ No newline at end of file diff --git a/doc/iocs/troubleshooting/Serial-Port-Debug.md b/doc/iocs/troubleshooting/Serial-Port-Debug.md new file mode 100644 index 000000000..b7faf4fa6 --- /dev/null +++ b/doc/iocs/troubleshooting/Serial-Port-Debug.md @@ -0,0 +1,33 @@ +# Serial Port Debug + +This is a temporary page for fault finding with changes from [ticket 4435](https://github.com/ISISComputingGroup/IBEX/issues/4435) + +symptoms: com port stuck, no reads work, dbior hangs. + +Until this PR is deployed you will first need to update `asyn.dll` for the IOC concerned +* stop IOC concerned +* copy `asyn.dll` from Ticket4435 build (`asyn_ovio` jenkins job) to same directory as `.exe` for IOC (so it will load in preference to existing `asyn.dll`) +* start IOC +* when done, remove new `asyn.dll` from the IOC `bin/windows-x64` directory + +After restarting with new asyn you should be able to type +``` +dbior drvAsyn 5 +``` +without it hanging. The new ASIO may even fix the issue, but should at least give you more output. If the IOC is not fixed, can you try adding: +``` +epicsEnvSet("WAITFORBYTES",1) +``` +to the st.cmd, this will enable another sort of read mode to test. + +It would also be worth adding +``` +asynSetOption("L0", -1, "eventmask", "0x11b") +``` +to startup, this will print all com events except writes – to see writes too use: +``` +asynSetOption("L0", -1, "eventmask", "0x11f") +``` +You can also stop IOC and try using the [`testSerialPort` command](TestSerialPort) + +It is also possible to use the new "escape" and "purge" options with asynSetOption at runtime to manipulate the various things about the send and receive buffers and line status. \ No newline at end of file diff --git a/doc/iocs/troubleshooting/TestSerialPort.md b/doc/iocs/troubleshooting/TestSerialPort.md new file mode 100644 index 000000000..ca38cbf9f --- /dev/null +++ b/doc/iocs/troubleshooting/TestSerialPort.md @@ -0,0 +1,34 @@ +# TestSerialPort + +TestSerialPort.exe is built as part of asyn and should be in your path after a config_env.bat + +For full list of options run +``` +TestSerialPort.exe --help +``` +general syntax is +``` +testSerialPort [options] COMPORT [outputString] [outputEos] [inputEos] +``` +Note that is COMPORT doesn't start with the letters `COM` it is considered an IP address and a connection will be made there as per `drvAsynIPPortCOnfigure` + +Also asyn only allows up to 2 characters for `inputEos`/`outputEos` (`streamDevice` itself can do more) + +To write a string with \\r\\n terminator and print reply +``` +testserialport COM5 "stuff_to_write" "\r\n" +``` +To write string, see reply and also log all serial events (as per WIN32 SetCommMask()) +``` +testserialport COM5 "stuff_to_write" "\r\n" --eventmask=0x1ff +``` +To see serial port status (need to have stopped IOC) +``` +testserialport COM5 --noread --report=5 +``` +The command uses escaped characters as per `printf/epicsStrnRawFromEscaped` e.g. for Eurotherm the Stream device `\\x05` (hex) would be written as `\\005` (octal) +``` +testserialport COM7 "\0040011PV" "\005" "\003" --eventmask=0x1ff +``` + +This program is built using VS2017 and so will not be on the instruments as standard. To use it on a deployment built with VS2010 you will to copy a static VS2017 build of `testserialport.exe` from jenkins. \ No newline at end of file diff --git a/doc/logo.svg b/doc/logo.svg new file mode 100644 index 000000000..a218177b9 --- /dev/null +++ b/doc/logo.svg @@ -0,0 +1,9 @@ + + ibex-logo + + + + diff --git a/doc/overview/First-Time-Build.md b/doc/overview/First-Time-Build.md new file mode 100644 index 000000000..ae6672340 --- /dev/null +++ b/doc/overview/First-Time-Build.md @@ -0,0 +1,270 @@ +# First-time install + +*You can get more information on the development workflow [here](/processes/git_and_github/Git-workflow)*. + +If any of the websites listed here do not work, contact another developer to get an alternate solution. + +## Things to know as a developer + +See [here](/processes/dev_processes/Things-to-know-as-a-developer) + +## Install Perl + +Download and install [Strawberry Perl](http://strawberryperl.com/) + +## Install Visual Studio + +See [Install Visual Studio](/iocs/compiling/Install-Visual-Studio) + +## Install Java JDK + +Install **OpenJDK 21 hotspot** from https://adoptium.net/?variant=openjdk21&jvmVariant=hotspot (the MSI installer is fine, tick all the boxes when it asks you which components to install) + +```{important} +**Do not install an Oracle JDK**. It has an unsuitable license. +``` + +You may wish to install some optional java components [as detailed here](#developer_upgrade_java). + +## Install Maven + +Create `C:\Tools\` + +Install [Maven](https://maven.apache.org/download.cgi) and follow the 'Windows tips' in [these instructions ](https://maven.apache.org/install.html) to set 'Environment Variables' for the SDK with JAVA_HOME and maven in PATH +It is recommended to install Maven into `C:\Tools\` + +_System Properties - Environment Variables - Path - New -`C:\Tools\apache-maven-###\bin`_ + +The Windows Tips from the above link says you should add maven to the PATH in the user variables. If it does not recognise mvn -v afterwards, then try to add it to the list of variables in PATH in System variables. + +## Install Git +Install Git [Getting-started-with-Git-and-GitHub](/processes/git_and_github/Getting-started-with-Git-and-GitHub) + +## Install Moxa NPort Driver manager +Install `\\isis.cclrc.ac.uk\inst$\kits$\CompGroup\Utilities\MOXA Nport Software\drvmgr*.exe` - this is required on the instruments for networked serial ports. It's required on dev machines for the database server (more specifically the moxa mappings page in the GUI) + +## Copy EPICS build + +>_EPICS contains lots of code that requires compilation before use, so we generally copy a pre-built version of this to save time and then build modules as changes are made._ + +1. Create `C:\Instrument\Apps\` +1. Navigate to `C:\Instrument\Apps\` and check whether the EPICS directory already exists. If so, remove the EPICS directory before continuing. +1. Follow instructions on the [Quick instructions section of the developer server build page](/iocs/compiling/Developer-Server-Build) to copy a built version of EPICS to `\instrument\apps` + +To easily add `EPICSTerm.bat` to the start menu after copying EPICS, run `C:\Instrument\Apps\EPICS\add_epicsterm_to_start_menu.bat` + +## Install the uktena python distribution + +See [Building and installing the uktena python distribution](/system_components/python/Building-and-installing-uktena) + +## Ruff git hook and convenience scripts + +In order to use this please clone the reusable workflows repo into dev +```git clone https://github.com/ISISComputingGroup/reusable-workflows.git``` + +We use `ruff` for [formatting & linting](/system_components/python/Python-conventions) and `pyright` for static analysis. + +We have a central configuration file for python code formatting & linting. + +If you wish (you probably do) to install a convenient alias which automatically uses an appropriate `ruff` config for each repository, see [here](https://github.com/ISISComputingGroup/reusable-workflows/blob/f40d42e26c9bb10779c1bd845be760d3b19f7e98/README.md#local-configuration-convenience-script) + +If you wish to set up a git hook to automatically check this on each commit, see [here](https://github.com/ISISComputingGroup/reusable-workflows/blob/f40d42e26c9bb10779c1bd845be760d3b19f7e98/README.md#local-configuration-git-hook). You probably do want to set this up, but be careful with external repositories as it may not adhere to their formatting/linting standards. You can bypass the pre-commit hook if needed with `git commit --no-verify` + +```{note} +This configuration is not mandatory as the build server will run linting and formatting checks on python repositories anyway, but running the checks locally is faster and more dev-friendly than waiting for the build servers. +``` + +## Install MySQL + +See [Installing and upgrading MySQL](/deployment/deploy/Installing-and-Upgrading-MySQL) + +```{note} +If installing for the first time, a new installation of MySQL may try to start up IBEX servers, which may lead to errors if Python isn't installed yet. + +Due to a change in admin provisioning on computers, you may need to use the admin runner available in one of the IBEX Utilities referred to later more directly. Alter `run_all` in the admin runner file to print the command to be run at the prompt and wait for an input to continue rather than simply run the command for you via the `AdminRunner` class. +``` + +## Building the GUI + +Please see [Building the GUI](/client/getting_started/Building-the-GUI). + +## Setting up the configurations & scripting directory + +```{note} +For all commands in this section... +* On a developer machine use your own username rather than "spudulike" +* Replace `` with your machine name, e.g. `NDXIRISTEST1`, `NDLT123` +* Replace `` with your lower case machine name, e.g. `init_ndxtest1.py`, `init_ndlt123.py` + +If you already have a 'NDXXXX' folder, rename it to 'NDXXXX.old' and continue with the following steps) +``` + +1. Create the following folder structure: `C:\Instrument\Settings\config` + +2. Navigate to the config folder + +3. Enable git credential store and set a username - replace `` with real computer name +``` +git config --global core.autocrlf true +git config --global credential.helper wincred +git config --global user.name "spudulike" +git config --global user.email "spudulike@.isis.cclrc.ac.uk" +``` + +4. Enable default recursive check +``` +git config --global push.recurseSubmodules check +``` + +5. Set up branch for this machine + +5a) Not A New Instrument +i.e. if this machine has had IBEX installed before and the config branch already exists, run through the following commands: +``` +cd C:\Instrument\Settings\config +git clone http://control-svcs.isis.cclrc.ac.uk/gitroot/instconfigs/inst.git NDXXXX +cd NDXXXX +git checkout NDXXXX +``` + +5b) New Instrument +i.e. if no config branch exists for this machine name, run through the following: + +* Via a git client clone the repository from 'http://spudulike@control-svcs.isis.cclrc.ac.uk/gitroot/instconfigs/inst.git' to a directory with your machine name, like so: +``` +git clone http://spudulike@control-svcs.isis.cclrc.ac.uk/gitroot/instconfigs/inst.git NDXXXX +``` + (when prompted, give spudulike password) + +* Confirm that you now have the config file structure in place (components, configurations and synoptics directories) + +* Create a branch from master with the machine name (if on an instrument) or your fedid if on a dev machine: + +``` +cd / +git checkout -b +rename "C:\Instrument\Settings\config\NDXXXX\Python\init_inst_name.py" to "C:\Instrument\Settings\config\NDXXXX\Python\init_ndxxxx.py" (note lowercase on init_ndxxxx.py) +git add Python/init_ndxxxx.py +git rm Python/init_inst_name.py +git commit -m "create initial python" +git push --set-upstream origin +``` + +```{note} +The init_inst_name python file should have underscores rather than dashes if the machine name contains dashes. So `NDWTEST-BLAH` would have the init file `init_ndwtest_blah.py` +``` + +Any configs created through IBEX will now be stored on this branch (they will only be pushed remotely if you do a manual push first e.g. the last line above) + +These repositories can be web browsed via [http://control-svcs.isis.cclrc.ac.uk/git/](http://control-svcs.isis.cclrc.ac.uk/git/) + +6. Check it is possible to access the configurations of another developer or of an instrument by fetching the correct branch and switching to it, like so: +``` +git fetch +git checkout NDXALF +``` + +7. Switch back to your new branch: +``` +git checkout +``` + +```{note} +The developer branch has been created to store useful configurations that may be shared amongst all developers. +``` + +## Setting up a calibrations directory + +If the `C:\Instrument\Settings\config\common` directory already exists, cd into it and do a `git pull` on master. Else run the following command from a Git-enabled command prompt (or modify target to run with Git Bash): + +``` +git clone https://gitlab.stfc.ac.uk/isisexperimentcontrols/common.git C:/Instrument/Settings/config/common + +``` + +The purpose and function of the calibration files are described [here](/system_components/configurations/Calibration-Files). + +{#first_time_install_configure_dae} +## Configure DAE for simulation mode on developer's computer / Register ISISICP + +* Make sure **ISISDAE-IOC-01.exe** and **ISISICP.exe** processes are not running +* run `create_icp_binaries.bat` in `c:\Instrument\Apps\EPICS` to get the latest version +* Open an administrator command prompt (right click on command prompt in start menu and click "run as administrator") +* `cd` to `c:\Instrument\Apps\EPICS\ICP_Binaries\isisdae` +* Type: +``` + register_programs.cmd +``` +Unfortunately the `/RegServer` registration process doesn't report either success or failure. If, on later starting the ISISDAE IOC, you see lots of errors of the form "CoCreateInstanceEx (ISISICP) : Class not registered" then it means the /RegServer flag did not work. Try registering it again in case you were not Administrator when you tried it the first time. If you get messages about missing method/functions etc. it may mean a previous isisicp.exe registered successfully, but the newer one didn't - just try again as administrator + +If you use Visual Studio 2017 (or older versions) and got "vcruntime140_1.dll was not found" error, you need to install Microsoft Visual C++ Redistributable for Visual Studio 2022. + +You can do this via this link: + +https://visualstudio.microsoft.com/downloads/ +(Scroll down, expand "Other Tools, Frameworks and Redistributables" and install Microsoft Visual C++ Redistributable for Visual Studio 2022(x64)) + + +### Set up the CS-Studio archiver +Before doing this ensure that the `build.bat` started in a previous step (to build EPICS) has successfully completed. +In `C:\Instrument\Apps\EPICS\CSS\master` run `setup_css.bat` +this will create directories for the archive engine. in `.\css-win.x86_64` + +### Getting DAE ready to start a run (so you are in SETUP rather than processing) + +You need to create a configuration so ISISDAE-IOC-01 will start: + +- run `start_ibex_server` (in `C:\Instrument\Apps\EPICS`) +- start the GUI +- configuration -> edit current configuration -> give it a name and save as. It should now prompt you to switch to the configuration, after you do this the `Config:` line on the bottom of the gui screen should show this name and the instrument should be in `SETUP` +- now go into experiment setup in DAE view and +- - in time channels select "specify parameters" (not tcb file), in time regime 1 put from 10.0 to 19900.0 step 10.0 with mode `DT=C`, and in time regime 2 use 10.0 to 19900.0 step 1.0 mode `DT=C` (time regime 2 is only used if an event mode wiring table is selected). If there are no boxes to fill in, this means the configuration has not switched to the one you saved above and the ISISDAE-IOC-01 process is not running +- - in data acquisition tab use the dropdown for wiring, detector and spectra to choose a file with the name "ibextest" in it e.g. wiring_ibextest.dat for wiring, detector_ibextest for detector, spectra_ibextest for spectra etc. You need to have all three boxes (wiring,detector,spectra) filled for things to work, also these files are interdependent so make sure they are all "*_ibextest.dat" ones for now. Note that either wiring_ibextest.dat or wiring_event_ibextest.dat can be chosen as the wiring table to select histogram or event mode. +- - Press the Apply button +- Now go back to main DAE view and press the "BEGIN RUN" button - you should get the instrument going into RUNNING +- Press Abort and you should go back into SETUP +- You can also open a python scripting window (scripting on left) and use g.begin() and g.abort() to do the same thing + +## Utilities + +Git clone (usually in c:\Instrument\Dev) the following utilities: + +``` +git clone https://github.com/ISISComputingGroup/IBEX_device_generator.git +git clone https://github.com/ISISComputingGroup/ibex_utils.git +git clone https://github.com/ISISComputingGroup/ConfigChecker.git +``` + +## Optional Extras + +The following are not strictly required for general IBEX development. They will be useful if you are on the project for > 1 year but otherwise probably not worth installing. + +### Explorer context menu shortcut for EPICS terminal + +To add a shortcut to open an EPICS terminal using the context menu in Windows' file explorer, Run Command Prompt as Admin, navigate to EPICS in the Command Prompt, and type `add_epics_context_menu.reg` (from EPICS top) to run it as admin. This will add a registry key that will enable you to right click in an empty space in file explorer and open the directory in an EPICS terminal. + +### VNC + +If you are supporting instruments it may be useful to download a VNC client. We have not settled on one that we all use but we have used: + + - VNC Viewer (just the client) which is available [here](https://www.realvnc.com/en/connect/download/viewer/) + +(we have used tightVNC (just the client) in past which is available [here](http://www.tightvnc.com/)) + +### NI DAQ + +It is recommended that developers only install this if they know that they will at some point be using a DAQ mx. If you do not do this step, you will be unable to run certain IOCs (e.g. RIKEN power supplies, muon separator), and consequently some of their tests will fail. + +Some IOCs depends on DAQMX binaries from national instruments. Go to http://sine.ni.com/psp/app/doc/p/id/psp-268 or if not go here https://www.ni.com/en-gb/support/downloads/drivers/download.ni-daqmx.html#311818 +and download the latest DAQMX drivers. When installing, ensure you check the box to install DAQMX. + +### Beckhoff XAR + +If you will be developing or reviewing code involved with Beckhoff PLCs you will need to install TwinCAT XAR. This can be found on the public share in `installers`. More information for simulating Beckhoffs can be found [here](/specific_iocs/motors/beckhoff/Beckhoff-testing). + +```{note} +XAR is the runtime that but for testing you will need the full setup which is the installer on the share +``` + +### Install WiX toolset (We use version 3 and not the latest one - only install if required) +Download and install latest stable version from https://wixtoolset.org/docs/wix3/ \ No newline at end of file diff --git a/doc/overview/High-Level-Architectural-Design.md b/doc/overview/High-Level-Architectural-Design.md new file mode 100644 index 000000000..1ec066e5e --- /dev/null +++ b/doc/overview/High-Level-Architectural-Design.md @@ -0,0 +1,51 @@ +# High level architectural design + +This document describes the high level design for IBEX. Eventually there should be outlines of the architectural design for each part of the high level design. + +### High Level Design + +![High Level Architecture](high_level_architecture.png) + +Notes on the figure: + +* CA - Channel Access +* External Gateway - This exposes the PVs running on the NDX server to the rest of the ISIS network, as read-only +* JDBC - Java Database Connectivity +* JMS - Java Messaging Service +* Soft IOC - These IOCs do not talk directly to hardware + +### Clients + +Clients can run on the NDX machine or elsewhere on the network. The external gateway defines the rules of which network addresses are allowed to read or write from PVs. At ISIS all machines in the blockhouse are allowed to write to PVs, and other machines on the ISIS network have read-only access to PVs. + +### NDX Server + +This is the Windows PC running the control software for the instrument (e.g. NDXLARMOR). This machine provides the PVs for the instrument externally, for example on Larmor all the PVs prefixed with ``IN:LARMOR:``. + +### Instrument Hardware + +Coming soon... + +### Low Level Design Documents + +Below is a list of the lower level architectural design documents. + +* [Alarms](/system_components/Alarms) +* [BlockServer](/system_components/blockserver/BlockServer-Structure) +* [Database Schemas](/system_components/mysql/Database-Schemas) +* [Client](/client/design_docs/Client-Architectural-Design) +* [Script Server Back End Design Document](/system_components/nicos/design_docs/Script-Server-Back-End-Design-Document) +* [Script Server Front End Design Document](/system_components/nicos/design_docs/Script-Server-Front-End-Design-Document) +* [Configuration & Component Architecture](/client/design_docs/Configuration-and-Component-Architecture) +* [Reflectometers](/specific_iocs/reflectometry/design_docs/Reflectometers) +* [Muon Active Compensation](/specific_iocs/magnets/Muon-Active-Compensation) +* [In situ DSC](/specific_iocs/temp_controllers/In-Situ-DSC) + +### Notes + +The figures are made in the freely available [yEd](https://www.yworks.com/products/yed), and are stored in the images folder when you check out the Wiki. + +### Nexus Files and Data + +Nexus files are created on run end and hold the spectra data and the block values. The block values are present if logging is enabled on the block. Once the logging is enabled the values get written to the archive database. There is a thread which pulls this data from the MySQL into the SQLite database. On end of run the nexus file is created fro the SQLite database. + diff --git a/doc/overview/Links-and-Resources.md b/doc/overview/Links-and-Resources.md new file mode 100644 index 000000000..92d8f2d6a --- /dev/null +++ b/doc/overview/Links-and-Resources.md @@ -0,0 +1,59 @@ +# Links and Resources + +## Training material + +- [IBEX Training Material](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FTraining&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}) + +- There is a settings branch called NDXTRAIN with the setup for the training PCs (excluding machine specific files) + +- [IBEX scripting and Configuration Workshop slide](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IBEX%20Scripting%20and%20Configurations%20Workshop.pptx) + +- [IBEX general intro slides (work in progress)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IBEX_Update.pptx) + +- [Sharepoint discussion site](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions) + +- [Instrument sharepoint site](http://www.facilities.rl.ac.uk/isis/computing/instruments/Instrument%20Documents/Forms/AllItems.aspx) contains passwords and instrument definition + +- [EPICS Channel access explained](http://www.aps.anl.gov/epics/docs/CAproto.html) +- [EPICS Channel access explained 2 (presentation)](https://epics.anl.gov/docs/APS2014/08-Database-Principles.pdf): nice bit on `NMS`, `MS`, `MSS`, `MSI` + +- [HDF5 Viewer for Nexus files](https://www.hdfgroup.org/downloads/hdfview#obtain) + +- A bit old but still a good overview of EPICS http://www.aps.anl.gov/epics/docs/GSWE.php + +- More recent EPICS training material http://www.aps.anl.gov/epics/docs/training.php in particular http://www.aps.anl.gov/epics/docs/USPAS2014.php + +## Links + +- [IBEX wiki](https://github.com/ISISComputingGroup/IBEX/wiki) +- [IBEX User Manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki) +- [IBEX Project Board for Issues](https://github.com/ISISComputingGroup/IBEX/projects/1) +- [Stream Protocol Page](http://epics.web.psi.ch/software/streamdevice/doc/) +- [genie_python reference](http://shadow.nd.rl.ac.uk/genie_python/sphinx/genie_python.html) +- [nagios](https://varanus.nd.rl.ac.uk/nagios/) +- [EPICS WIKI](https://wiki-ext.aps.anl.gov/epics/index.php/RRM_3-14) +- [Current program version, servers and settings](http://beamlog.nd.rl.ac.uk/inst_summary.xml) +- [Instrument Overview page](http://dataweb.isis.rl.ac.uk/ibexdataweb/overview/ibexoverview.html) +- [Instrument Dashboards](http://dataweb.isis.rl.ac.uk/Dashboards/default.htm) + +{#all_troubleshooting_links} +## Troubleshooting + +A non-exhaustive collection of troubleshooting pages. + +* [Jenkins](/processes/continuous_integration/Jenkins-Trouble-Shooting) +* [Blockserver](/system_components/blockserver/Blockserver-Trouble-Shooting) +* [IOC and device](/iocs/troubleshooting/IOC-And-Device-Trouble-Shooting) +* [Motors](/specific_iocs/motors/Motors-Trouble-Shooting) +* [DAE](/specific_iocs/dae/DAE-Trouble-Shooting) +* [Experimental Runs](/specific_iocs/dae/Experimental-Runs) - This includes how log plotter can be used to look at old data. +* [Eclipse](/client/eclipse/Common-Eclipse-Issues) +* [GUI](/client/GUI-Troubleshooting) +* [Archive Engine](/system_components/archive/CSS-Troubleshooting) +* [Database](/system_components/mysql/Database-Troubleshooting) +* [Web Dashboard](#webdashboard_troubleshooting) +* [genie_python](/scripting/genie_python/genie_python-Troubleshooting) +* [Computer](/systems/inst_control/Computer-Troubleshooting) (e.g. the NDX) +* [ICAT and TopCAT](/systems/external/ICAT-Troubleshooting) +* [Mini-inst](/system_components/startup/Mini-inst-Troubleshooting) +* [Where are the log files](/iocs/troubleshooting/Log-file-location) diff --git a/doc/overview/New-Starters.md b/doc/overview/New-Starters.md new file mode 100644 index 000000000..00eb70906 --- /dev/null +++ b/doc/overview/New-Starters.md @@ -0,0 +1,146 @@ +# New Starter's Guide + +## Welcome to IBEX! + +1. [Using the Developer Wiki](#using-the-developer-wiki) + 1. [Searching the Codebase](#searching-the-codebase) +2. [Getting Set Up](#getting-set-up) + 1. [Using Git](#using-git) + 2. [Creating Useful Shortcuts](#creating-useful-shortcuts) + 3. [Office Admin Links](#office-admin-links) +3. [Familiarising with IBEX](#familiarising-with-ibex) + 1. [Introduction](#introduction) + 2. [Components and Technical Concepts](#components-and-technical-concepts) +4. [EPICS Introduction](#epics-introduction) + 1. [Further EPICS](#epics-introduction) + 2. [Input/Output Controller (IOC) Resources](#input-output-controller-resources) +5. [Control System Studio](#control-system-studio) + +This page is designed to be used as a comprehensive reference to getting started in the Instrument Controls team. Written by new-ish starters, for the newest starters. It's a collection of useful resources which will speed up your onboarding and get you developing. + +## Using the Developer Wiki + +There are many how-to guides in the Developer's manual. Being able to use the wiki is as much a skill as learning to use any of our tools. If you're stuck, this can be really useful - try searching for different keywords from your ticket. + +### Searching the Codebase + +Another mode of attack is searching through our codebase for particular code snippets, which is equally useful! You don't need to reinvent the wheel - you can search to see how code has been already written for a similar purpose, find examples of how unfamiliar concepts are used, or see our coding conventions for certain processes. This is also a good way to locate files if you don't know where they live, but know some keywords they may contain! + +You can do this by searching in [ISISComputingGroup](https://github.com/ISISComputingGroup/) and clicking on the 'Code' tab (as seen above). + +## Getting Set Up + +First off you're going to need all the developer tools we use, a copy of the existing codebase, and the Eclipse GUI for IBEX*. + +- [Installing the Backend System](First-Time-Build) + +Don't forget to come back afterwards! + +### Using Git + +We use Git for version control of the codebase. For IBEX-flavoured Git resources, see: + +- [Our Github resources](../processes/Git-and-GitHub) +- [IBEX Git workflow](../processes/git_and_github/Git-workflow) + +If you're not familiar with it, learn it! + +- [Github's own Git tutorials](https://try.github.io/) +- [Friendly git sandbox](https://learngitbranching.js.org/) - the 'remote' tutorials in particular +- [Nice intro to Git & version control](https://swcarpentry.github.io/git-novice/01-basics/index.html) + +### Creating Useful Shortcuts + +There's a couple files/commands you'll find yourself using a *lot*. It's handy to create desktop shortcuts for these frequently used links: + +- `EPICSTerm` (The EPICS terminal window where IOCs are built and run) +- `start_ibex_server` +- `stop_ibex_server` + +These can be found in `C:\Instrument\Apps\EPICS`. + +### Office Admin Links + +As a new starter you should add your name and picture to the training [slides](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FTraining&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View=%7BF2C33C51-70E6-4343-B937-2C59A2568306%7D). + +- [ISIS Sharepoint](https://stfc365.sharepoint.com/sites/isis-hub/) - Shared ISIS documents, manuals, meeting minutes etc +- [Oracle Login](https://portal.ssc.rcuk.ac.uk/) - Book leave, create timesheets +- [Flexi Time](https://flexiral.stfc.ac.uk/FCDWeb/) - If you're on Flexi time and need to view/edit your work history +- [Health & Safety Training](https://lmsweb.stfc.ac.uk/moodle/login/index.php) + +Here's some useful bits and bobs for how our team operates and the conventions we use in our work. + +- [IBEX Processes](../Processes) - Logistics of the day-to-day +- [IBEX Project Board](https://github.com/orgs/ISISComputingGroup/projects) - Where our tickets live +- [Tickets and their Workflow](../processes/dev_processes/Tickets-and-their-Workflow) +- [Useful Tools](../Tools) + +## Familiarising with IBEX + +### Introduction + +While that's installing, get to know the IBEX components. Get to know EPICS. It's the software environment that IBEX uses to control the ISIS instruments. + +Ironically, to get a well-explained overview of what IBEX is, and how to interact with it as a dev, the [IBEX User Manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki) is a really nice place to start. In particular, see: + +- [IBEX Key Concepts](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Key-Concepts-in-IBEX) +- [IBEX GUI Features](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/IBEX-GUI-Features) +- [How To Do Things In IBEX](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/How-To-Do-Things-In-IBEX) + +There are some training materials available that we use to train scientists on IBEX, which is also a nice way to get started: + +- [IBEX training](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FTraining&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View=%7BF2C33C51-70E6-4343-B937-2C59A2568306%7D) + +### Components and Technical Concepts + +Some pages to read over: + +- [System Components](../System-components) +- [Useful tools](../Tools) + +Pages to skim over. Read in detail only when needed: + +- [Settings and Configurations](../system_components/Settings-and-Configurations) +- [Data Generation and Storage](../systems/inst_control/Data-Generation-and-Storage-on-Instrument-PCs-(NDX's)) +- [Information for User Scientists](https://github.com/ISISComputingGroup/IBEX/wiki) + +Every section after this line contains reference information that you do not necessarily have to read before you do your first ticket, but it is where you should take a look when you find out you need to learn new things in order to do your work. + +## EPICS Introduction + +There is much to learn about EPICS and it can be difficult to know what you're looking for. You do not have to read these resources until you get to writing your first IOC. Here are some overviews: + +- Old but still good overview - [https://epics.anl.gov/docs/GSWE.php](https://epics.anl.gov/docs/GSWE.php) +- More recent overview [http://www.aps.anl.gov/epics/docs/training.php](http://www.aps.anl.gov/epics/docs/training.php) in particular [http://www.aps.anl.gov/epics/docs/USPAS2014.php](http://www.aps.anl.gov/epics/docs/USPAS2014.php) +- [Powerpoint EPICS overview](https://epics.anl.gov/docs/USPAS2014/1-Monday/EPICS_Intro.pdf) +- [Training Course](https://epics.anl.gov/docs/USPAS2014.php) +- [Database Principles (what's a record? And more)](https://epics.anl.gov/docs/USPAS2014/2-Tuesday/Database-1.pdf) +- [Database Principles II (with examples)](https://epics.anl.gov/docs/USPAS2014/2-Tuesday/Database-2.pdf) + +### Further EPICS + +- [State Notation Language 1](https://epics.anl.gov/docs/USPAS2014/2-Tuesday/SNL_1_EPICSAutomation.pdf) [State Notation Language 2](https://epics.anl.gov/docs/USPAS2014/2-Tuesday/SNL_2_Sequencer.pdf) +- [ASYN/Stream Device overview](https://epics.anl.gov/docs/USPAS2014/2-Tuesday/SNL_2_Sequencer.pdf) +- [YouTube Videos covering the epics.gov.anl slideshows](https://epics.anl.gov/docs/APS2015.php) + +### Input Output Controller Resources + +When building an IOC, emulator, or writing tests, much can be learned by looking at existing IOCs: these can be found in `C:\Instrument\Apps\EPICS\ioc\master`. + +However, there are also several places online to find good sources for learning and referencing. + +- [Creating & Basics of IOCs](../IOCs) - Our guide to creating IOCs and how they work. Lots of useful tips! +- [Protocol Files and Record Types](http://epics.web.psi.ch/software/streamdevice/doc/protocol.html) - Excellent reference for when you're mixing up your %d's and %f's. Bookmark this one! +- [Sequencer - State Notation in IOCs](http://www-csr.bessy.de/control/SoftDist/sequencer/Tutorial.html#pv-names-using-program-parameters) - All about implementing a state machine into an IOC, if needed. + +## Control System Studio + +We use a number of parts of Control System Studio (CSS) in our GUI (Alarms, Databrowser etc.). Most of these resources require little developer interaction apart from creating Operator Interfaces (OPIs) - GUIs, essentially - for the instrument controllers. It can be useful to create a desktop shortcut of this too. + +Here's an overview and a get started guide. + +- [Creating OPIs](/client/opis/OPI-Creation) - An introduction for how we use CSS and to set it up on your computer +- [Overview of CS-Studio](https://epics.anl.gov/docs/USPAS2014/1-Monday/CSS_1_Overview.pdf) +- [Intro to using CS-Studio](http://www.aps.anl.gov/epics/docs/USPAS2014/1-Monday/CSS_2_First_Steps.pdf) + +You can find CSS at `C:\Instrument\Apps\EPICS\CSS\master`. \ No newline at end of file diff --git a/doc/overview/deployment_architecture.png b/doc/overview/deployment_architecture.png new file mode 100644 index 000000000..2e6b813ff Binary files /dev/null and b/doc/overview/deployment_architecture.png differ diff --git a/doc/overview/high_level_architecture.graphml b/doc/overview/high_level_architecture.graphml new file mode 100644 index 000000000..323b9d3af --- /dev/null +++ b/doc/overview/high_level_architecture.graphml @@ -0,0 +1,1602 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + 2 + + + + + + + + + + NDX Server (Windows PC) + + + + + + + + + + + + + + + + + Database +Server + + + + + + + + + + + + + + + + + + + + 2 + + + + + + + + + + Clients (External or on NDX Server) + + + + + + + + + + + + + + + + + IBEX Client + + + + + + + + + + + + + + + + + EPICS +(caget/caput) + + + + + + + + + + + + + + + + + Channel Access (Local Host) + + + + + + + + + + + + + + + + + genie_python +(standalone) + + + + + + + + + + + + + + + + + Block Server + + + + + + + + + + + + + + + + + + + + Instrument Hardware + + + + + + + + + + 2 + + + + + + + + + + + + + + + + DAE + + + + + + + + + + + + + + + + + Hardware +Component + + + + + + + + + + + + + + + + + MOXA + + + + + + + + + + + + + + + + + Hardware +Component + + + + + + + + + + + + + + + + + MOXA + + + + + + + + + + + + + + + + + Hardware +Component + + + + + + + + + + + + + + + + + Hardware +Component + + + + + + + + + + + + + + + + + + + ICP + + + + + + + + + + + + + + + + + Archive DB +(MySQL) + + + + + + + + + + + + + + + + + + + + Alarm Server + + + + + + + + + + + + + + + + + Beam Data + + + + + + + + + + + + + + + + + Script Server +(Nicos and +Proxy) + + + + + + + + + + + + + + + + + Other +Instrument + IOCs + + + + + + + + + + + + + + + + + Channel Access (ISIS Network) + + + + + + + + + + + + + + + + + Channel Access (External Gateway) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + LabVIEW +VI + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + IOC + + + + + + + + + + + + + + + + + Exp_data DB +(MySQL) + + + + + + + + + + + + + + + + + + + + Alarm DB +(MySQL) + + + + + + + + + + + + + + + + + + + + IOC DB +(MySQL) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Soft IOC + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Soft IOC + + + + + + + + + + + + + + + + + msg_log DB +(MySQL) + + + + + + + + + + + + + + + + + + + + Message Log +Server + + + + + + + + + + + + + + + + + Archive +Engine +Server + + + + + + + + + + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + + + + CA/JSON over CA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + CA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + JSON over CA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + JMS + + + + + + + + + + + + + + + + + + JDBC + + + + + + + + + + + + + + + + + + JDBC + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + JDBC + + + + + + + + + + + + + + + + + + + + + + JMS + + + + + + + + + + + + + + + <?xml version="1.0" encoding="utf-8"?> +<svg version="1.1" + xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" + x="0px" y="0px" width="41px" height="48px" viewBox="-0.875 -0.887 41 48" enable-background="new -0.875 -0.887 41 48" + xml:space="preserve"> +<defs> +</defs> +<linearGradient id="SVGID_1_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-979.1445" x2="682.0508" y2="-979.1445" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_1_)" d="M19.625,36.763C8.787,36.763,0,34.888,0,32.575v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,34.888,30.464,36.763,19.625,36.763z"/> +<linearGradient id="SVGID_2_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-973.1445" x2="682.0508" y2="-973.1445" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_2_)" d="M19.625,36.763c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.927-18.396,3.927 + c-9.481,0-17.396-1.959-18.396-3.927l-1.229,2C0,34.888,8.787,36.763,19.625,36.763z"/> +<path fill="#3C89C9" d="M19.625,26.468c10.16,0,19.625,2.775,19.625,2.775c-0.375,2.721-5.367,5.438-19.554,5.438 + c-12.125,0-18.467-2.484-19.541-4.918C-0.127,29.125,9.465,26.468,19.625,26.468z"/> +<linearGradient id="SVGID_3_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-965.6948" x2="682.0508" y2="-965.6948" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_3_)" d="M19.625,23.313C8.787,23.313,0,21.438,0,19.125v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,21.438,30.464,23.313,19.625,23.313z"/> +<linearGradient id="SVGID_4_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-959.6948" x2="682.0508" y2="-959.6948" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_4_)" d="M19.625,23.313c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.926-18.396,3.926 + c-9.481,0-17.396-1.959-18.396-3.926l-1.229,2C0,21.438,8.787,23.313,19.625,23.313z"/> +<path fill="#3C89C9" d="M19.476,13.019c10.161,0,19.625,2.775,19.625,2.775c-0.375,2.721-5.367,5.438-19.555,5.438 + c-12.125,0-18.467-2.485-19.541-4.918C-0.277,15.674,9.316,13.019,19.476,13.019z"/> +<linearGradient id="SVGID_5_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-952.4946" x2="682.0508" y2="-952.4946" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#3C89C9"/> + <stop offset="0.1482" style="stop-color:#60A6DD"/> + <stop offset="0.3113" style="stop-color:#81C1F0"/> + <stop offset="0.4476" style="stop-color:#95D1FB"/> + <stop offset="0.5394" style="stop-color:#9CD7FF"/> + <stop offset="0.636" style="stop-color:#98D4FD"/> + <stop offset="0.7293" style="stop-color:#8DCAF6"/> + <stop offset="0.8214" style="stop-color:#79BBEB"/> + <stop offset="0.912" style="stop-color:#5EA5DC"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_5_)" d="M19.625,10.113C8.787,10.113,0,8.238,0,5.925v10c0,2.313,8.787,4.188,19.625,4.188 + c10.839,0,19.625-1.875,19.625-4.188v-10C39.25,8.238,30.464,10.113,19.625,10.113z"/> +<linearGradient id="SVGID_6_" gradientUnits="userSpaceOnUse" x1="642.8008" y1="-946.4946" x2="682.0508" y2="-946.4946" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="0.0039" style="stop-color:#9DD7FF"/> + <stop offset="0.2273" style="stop-color:#BDE5FF"/> + <stop offset="0.4138" style="stop-color:#D1EEFF"/> + <stop offset="0.5394" style="stop-color:#D9F1FF"/> + <stop offset="0.6155" style="stop-color:#D5EFFE"/> + <stop offset="0.6891" style="stop-color:#C9E7FA"/> + <stop offset="0.7617" style="stop-color:#B6DAF3"/> + <stop offset="0.8337" style="stop-color:#9AC8EA"/> + <stop offset="0.9052" style="stop-color:#77B0DD"/> + <stop offset="0.9754" style="stop-color:#4D94CF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<path fill="url(#SVGID_6_)" d="M19.625,10.113c10.839,0,19.625-1.875,19.625-4.188l-1.229-2c0,2.168-8.235,3.926-18.396,3.926 + c-9.481,0-17.396-1.959-18.396-3.926L0,5.925C0,8.238,8.787,10.113,19.625,10.113z"/> +<linearGradient id="SVGID_7_" gradientUnits="userSpaceOnUse" x1="644.0293" y1="-943.4014" x2="680.8223" y2="-943.4014" gradientTransform="matrix(1 0 0 -1 -642.8008 -939.4756)"> + <stop offset="0" style="stop-color:#9CD7FF"/> + <stop offset="1" style="stop-color:#3C89C9"/> +</linearGradient> +<ellipse fill="url(#SVGID_7_)" cx="19.625" cy="3.926" rx="18.396" ry="3.926"/> +<path opacity="0.24" fill="#FFFFFF" enable-background="new " d="M31.04,45.982c0,0-4.354,0.664-7.29,0.781 + c-3.125,0.125-8.952,0-8.952,0l-2.384-10.292l0.044-2.108l-1.251-1.154L9.789,23.024l-0.082-0.119L9.5,20.529l-1.65-1.254 + L5.329,8.793c0,0,4.213,0.903,7.234,1.07s8.375,0.25,8.375,0.25l3,9.875l-0.25,1.313l1.063,2.168l2.312,9.645l-0.521,1.416 + l1.46,1.834L31.04,45.982z"/> +</svg> + + + + diff --git a/doc/overview/high_level_architecture.png b/doc/overview/high_level_architecture.png new file mode 100644 index 000000000..8c32ed9ea Binary files /dev/null and b/doc/overview/high_level_architecture.png differ diff --git a/doc/processes/Admin.md b/doc/processes/Admin.md new file mode 100644 index 000000000..21c3ef69d --- /dev/null +++ b/doc/processes/Admin.md @@ -0,0 +1,8 @@ +# Admin + +```{toctree} +:glob: +:maxdepth: 1 + +admin/* +``` \ No newline at end of file diff --git a/doc/processes/Continuous-Integration.md b/doc/processes/Continuous-Integration.md new file mode 100644 index 000000000..b83bb8661 --- /dev/null +++ b/doc/processes/Continuous-Integration.md @@ -0,0 +1,8 @@ +# Continuous Integration + +```{toctree} +:glob: +:maxdepth: 1 + +continuous_integration/* +``` \ No newline at end of file diff --git a/doc/processes/Decision-Log.md b/doc/processes/Decision-Log.md new file mode 100644 index 000000000..a4b976f96 --- /dev/null +++ b/doc/processes/Decision-Log.md @@ -0,0 +1,91 @@ +# Decision Log + +A place to record decisions: + +1. Decide format for calibration files. Decided to update LabVIEW and add a header with comments. This is because updating Seci is not a huge amount of work (1/2 day to do all instruments) and we don't make want to make a halfway house that has to be changed in the future. We don't want to go as far as allowing manufacturer file directly in EPICs because this is hard; this option can happen in the future. + +2. We will not (in general) put `@init` handlers on records. This is because it is hard to do for any records which contain intermediate logic. + +3. Flow control macros will be available as a settable macro for devices on which it can be configured. The reason is that if it is set on the device we don't want to have to change the st.cmd to set the option. The consequences of this is that it allows users to turn flow control on which can cause devices to stall if the device has it switched off. The default should be not to turn it on. Since it is a configurable item it should be in the lists macros that users can set otherwise it is a support call to change this. + +4. Manual config upgrades should no be added to the upgrade script not added as a text note to the dev release notes. This means that they should be included in the code so can be reviewed. We may need to version these steps so that we don't get manual upgrade steps overflowing releases. This is to prevent them getting lost. + +5. `Google test` was chosen for the cxx testing framework over `cxx test` because it is actively developed, unlike cxx test which is a branch that was created for Mantid but no longer used, and it builds on windows. + +6. Decided that if a motor is moving when it goes uncommunicative it should remain as moving. This is because this is the current behaviour so will be expected and it is not clear if it is moving or stopped if you can not communicate. In either case, there should be communication alarm on the PVs. + +7. Decided to split genie commands into two sets basic and extended. This keeps the simplest commands in a place where users can find them and means they will not have a large choice. Extended commands are for experts and are in a separate module. This will be accessed via `g.ext` users can import this in the usual way if desired. The cons of this is that we are splitting up where advanced users can find commands but this is acceptable because they are advanced users. The other problem is deciding which commands are extended and which are not. + +8. `Genie_python` should be able to deal with PVs but the names of the PVs are not guaranteed they are advanced users only. Commands utilising PVs should have comments stating that it is an extended command and that the PV name may not be stable and that it is better to use a block. Ideally these commands should be placed in the advanced package. The downside is once we let this out then it will be used; however some scripts already do similar things it is better we support officially. + +9. An ENGIN-X scientist asked for access to the exp_details database to create reports detailing RB numbers and users on the instrument. They were given read-only access but warned that we didn't guarantee the schema or the lifetime of the data and it would be better to go through the BusApps API. + +10. Create upper case aliases of all blocks and use these in genie_python. This avoids the extra look up of current block names and the communication problem we have seen. The downside is there are more aliases created by and used in the block server and gateway but because there are usually fewer than 100 blocks this performance downside should not matter. The added advantage is that the blockserver is removed as a dependency in `caget`/`caset`. + +11. Setting a device to remote mode should **not** be done on IOC start. There should be a clear indication on the OPI that it is in remote mode (where possible) and a button to set it into remote mode. There should also be an alarm on the PV but not at IOC level when the device is in local mode. It was thought that instrument scientists did not know that IOCs were restarting (e.g. when the updated a config for collimator) and so would be surprised at the change in the device. + - Advised by Kathryn, present John, Alistair + - 2019/05/16 + +12. If a setpoint on a device has a range set by the device it should be set up by using DRVH and DRVL. This is in preference to not capping it and letting the device ignore wrong values. The last choice is to add another record on top which intercepts the value and records an error and sends the capped value. The reason is that we would like to capture the range of the device without adding the complication of another record. This is also a similar behaviour to SECI. Finally, we could, in the future, check the value set in a `cset` against these values. + - Present, John, Tom, Aaron, David and Freddie + - 2019/07/05 + +13. It was decided that as part of the motion system on ZOOM a tablet or some other handheld device will be used [see] (https://github.com/ISISComputingGroup/IBEX/issues/2937). This is required as scientists would like to be able to move axes whilst they are inside the light curtain. All safety issues on this will be handled by a deadman's handle built into the device. In further discussions [here](https://github.com/ISISComputingGroup/IBEX/issues/4238) it was decided that the trying to move the IBEX GUI to something touch friendly would be hard and instead a new cut-down GUI should be produced in Phoebus. + - Present: Thomas, Dom, Kathryn + +14. It was decided that as part of the parameter limits [ticket](https://github.com/ISISComputingGroup/IBEX/issues/4168) for the script generator we will pass all parameters as strings from java to python and handle them in the python code by providing a decorator to pass casting operations to e.g. `@cast_parameters_to(temperature=float, field=str, uamps=float)`. This will allow them to define their own casting operations to deal with funny edge cases in a nice way. It also reduces the amount of edge cases we may find with py4j handling the difference between python and java class. + - Present: James, Tom, Alistair + - 09/12/2019 + +15. We should move towards using virtual environments for our python projects see [here](/system_components/python/Python-dependencies) for more detail. + - Present: Dom, Tom, Freddie, John + - 19/05/2020 + +16. Genie_python should be kept 2/3 compatible until the block server is converted to python 3 in case it uses it. After this the decision should be revisited to see if we can go purely 3. + - Present: Dom, Freddie, John + - 27/05/2020 + +17. Helium level meters and other such items should be put on the domain "HA" (Hall) with no instrument identifier. So that the pv would be "HA:HLM:LEVEL1" to achieve this the IOC should run with a blank `PVPREFIX` and the IOC should have `HA` in its IOC prefix. This is to make it the same as central services which runs with a blank prefix because it runs IOCs in multiple domains. We don't need the machine identifies for hall based services because it is not really anything to do with the machine unlike the instrument pvs. + - Present: Alex, John, Kathryn, Freddie, Dom + - 08/07/2020 + +18. Motion setpoints based on bare motors will no longer be supported in IBEX. This is to make the code simpler, as we move towards many multiple axes in [#4573](https://github.com/ISISComputingGroup/IBEX/issues/4573) and motion setpoints more robust to motor failure. All motors should support the axes record so this should not be a problem. For more esoteric motors which need to be used on short notice it is fairly easy to either add a motor axes on top of the motor record by adding something on the form`< $(SM300CONFIG)/axes.cmd` or moving the axis to the required positions in a script. + - Present: Dom, John, Kathryn, Tom + - 14/08/2020 + +19. Where to put the script generator release download for scientists. Decision is to put it in github, upload it to the release page, then create a page to link to it with install instructions (either in git hub or github pages). This was chosen because it is likely to be the quickest way of getting something uploaded for people to download, and we need it before cycle. Other choice were our external web server or an STFC sit; both of these would have been slow. + - Present: John, Alistair, Kevin, Bish, Thomas + - 2020/08/25 + +20. How to manage the database architecture to avoid instrument disk filling up. A decision was made that on most instruments the data being logged by the Instrument Archive is not critical to an experiment and so it is ok if it stops logging given a network outage on the instrument (the exception here is on [Enginx](https://github.com/ISISComputingGroup/IBEX/issues/5817)). Therefore we will move the MySQL instance that the [Instrument Archive](https://github.com/ISISComputingGroup/IBEX/issues/5819) is pushing to onto a [central server](https://github.com/ISISComputingGroup/IBEX/issues/5818). This will also be done for the [IOC log messages](https://github.com/ISISComputingGroup/IBEX/issues/5820) as they are replicated on logs on the instrument anyway. In the future it would be good to have a small cache on the instrument for some of this data to avoid issues when network is lost. + + - Present: Chris M-S, David, Dom, Freddie, John, James, Kathryn, Tom W + - 2020/10/14 + +21. Should we still support a phone app with the MCR news for ISIS staff to use? Decision was made that it's a lot of development effort to support a mobile app for all platforms and the only benefits it gives over a slack/teams channel is that it doesn't require a user login. Seeing as everyone on ISIS will have a teams account this doesn't really apply. We are going to use the teams channel instead and if it would be helpful to use graphs this can be implemented using a python service or similar. + +22. In one system test we are testing the restarting of each IOC to test that we do not hit a limit as we did with the 256 limit in procservcontrol. Some IOCs are failing to restart, this may be because there is no hardware to talk to or the PVs we are looking at are autosave pvs and they are not available in the IOC. We have decided to start these IOCs in recsim to try and avoid this as this test is not to test the IOC itself but to test procservcontrol of the IOC. We can then at a later date put more work into the IOCs to make them start and go into alarm correctly. + +23. How to manage the [`InstrumentScripts`](https://github.com/ISISNeutronMuon/InstrumentScripts) repository to make sure the code is kept up to date on all instruments but at the same time be open to Instrument Scientists modifying it. Decision was made [here](https://github.com/ISISComputingGroup/IBEX/issues/5858) to do the following: +* When doing a release we will tag a release of Instrument Scripts and release it to all instruments, whether they currently use the library or not +* We will write an automated service to tell us if instruments have local changes in Instrument Scripts (which should be located in C:\Instrument\scripts), or if they are not on the release tag +* If instruments do have local changes we will discuss with them getting the changes into master for the next release +* We will block pushing to master, instead requiring people to make PRs which we will review and check for unexpected side effects to the best of our ability. Other instrument scientists are welcome to contribute to this review process + + - Present: Bish, Dom, John, Thomas L and Tom W + - 13/11/2020 + +24. How to manage script definitions for the script generator. See ticket https://github.com/ISISComputingGroup/IBEX/issues/5754 for newly decided workflow and actions and tickets to be carried out + +- Present: @Kevin-Woods-Tessella @John-Holt-Tessella @DominicOram @FreddieAkeroyd @ThomasLohnert @Tom-Willemsen +- Date: 01/12/2020 + +25. We have decided to hide the complexity of using the script generator parameters file from script generator users by removing references to parameter files and tying the two files produced directly to generating scripts. This was captured in [#6492](https://github.com/ISISComputingGroup/IBEX/issues/6492). + +- Present: James, Kathryn and Dom +- Date: 12/05/2021 + +26. We decided to put the emulator and ioc tests into the support submodule for each IOC. As well as a number of other repository structure decisions (see https://github.com/ISISComputingGroup/IBEX/issues/6131) + +- Present: James, Kathryn, Dom, Freddie, Jack Harper, Chris MS, David and Sam J +- Date: 03/06/2021 \ No newline at end of file diff --git a/doc/processes/Dev-Processes.md b/doc/processes/Dev-Processes.md new file mode 100644 index 000000000..09edd13d8 --- /dev/null +++ b/doc/processes/Dev-Processes.md @@ -0,0 +1,8 @@ +# Developer Processes + +```{toctree} +:glob: +:maxdepth: 1 + +dev_processes/* +``` \ No newline at end of file diff --git a/doc/processes/Git-and-GitHub.md b/doc/processes/Git-and-GitHub.md new file mode 100644 index 000000000..2417aadd2 --- /dev/null +++ b/doc/processes/Git-and-GitHub.md @@ -0,0 +1,9 @@ +# Git and Github + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +git_and_github/* +``` \ No newline at end of file diff --git a/doc/processes/Instrument-Details.md b/doc/processes/Instrument-Details.md new file mode 100644 index 000000000..3bae430d3 --- /dev/null +++ b/doc/processes/Instrument-Details.md @@ -0,0 +1,14 @@ +# Instrument Migration Notes + +```{warning} +These instrument details pages provide historical context about the devices on a specific instrument, +at the time they were migrated to IBEX. **These pages may not be up to date with the latest instrument +configuration.** +``` + +```{toctree} +:glob: +:maxdepth: 1 + +instrument_details/* +``` diff --git a/doc/processes/Meetings-with-Scientists.md b/doc/processes/Meetings-with-Scientists.md new file mode 100644 index 000000000..4935ad273 --- /dev/null +++ b/doc/processes/Meetings-with-Scientists.md @@ -0,0 +1,8 @@ +# Meetings - with scientists + +```{toctree} +:glob: +:maxdepth: 1 + +meetings_with_scientists/* +``` \ No newline at end of file diff --git a/doc/processes/Meetings.md b/doc/processes/Meetings.md new file mode 100644 index 000000000..aa28b65b1 --- /dev/null +++ b/doc/processes/Meetings.md @@ -0,0 +1,8 @@ +# Meetings - internal + +```{toctree} +:glob: +:maxdepth: 1 + +meetings/* +``` \ No newline at end of file diff --git a/doc/processes/Pm.md b/doc/processes/Pm.md new file mode 100644 index 000000000..0412178c9 --- /dev/null +++ b/doc/processes/Pm.md @@ -0,0 +1,8 @@ +# Project Management + +```{toctree} +:glob: +:maxdepth: 1 + +pm/* +``` \ No newline at end of file diff --git a/doc/processes/Retrospective-Notes.md b/doc/processes/Retrospective-Notes.md new file mode 100644 index 000000000..c84602512 --- /dev/null +++ b/doc/processes/Retrospective-Notes.md @@ -0,0 +1,12 @@ +# Retrospective Notes + +Notes from sprint retrospective meetings. + +```{toctree} +:reversed: +:glob: +:maxdepth: 1 +:titlesonly: + +retrospective-notes/* +``` diff --git a/doc/processes/admin/Data-Protection.md b/doc/processes/admin/Data-Protection.md new file mode 100644 index 000000000..7c66fad33 --- /dev/null +++ b/doc/processes/admin/Data-Protection.md @@ -0,0 +1,64 @@ +# Data Protection + +Data protection is concerned with the fair and proper use of information about people. + +All individuals have a right to privacy, which means that all organisations must manage personal information with appropriate care. In the UK, the protection of data is enshrined in law - specifically the Data Protection Act 2018 (DPA-2018) 2018 and the General Data Protection Regulation (GDPR) which, although it is an EU regulation, also forms part of UK law. The DPA-2018 and GDPR apply to the ‘processing of personal data’, and will catch most businesses and organisations, whatever their size. + +For more information on DPA-2018 and GDPR, visit the web-site of the [Information Commissioner's Office](https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/). + +## What is ‘personal data’? +Personal data is information about living persons who: + 1. can be identified or who are identifiable, directly from the information in question; or + 1. who can be indirectly identified from that information by combining it with other information. + +Personal data means information about any living individual. This could be anyone, including a customer, client, employee, partner, member, supporter, business contact, public official or member of the public. The data need not be ‘private’ information – information which is public knowledge or is about someone’s professional life can be personal data. + +## What is ‘processing’? +The DPA-2018 and GDPR apply to the processing of _personal data_, whether by automated or manual means. + +Almost anything you do with data counts as processing; including collecting, recording, storing, using, analysing, combining, disclosing or deleting it. Furthermore, the law expects that any organisation (or individual) that processes personal data does so in a way that is clear, open, honest and fair with people from the start. + +In short, if processing involves personal data, the DPA-2018 & GDPR apply. Otherwise, they don't. + +## IBEX and Personal Data + +IBEX processes a small amount of personal data. It displays information from the Experiment Details database (display is a form of disclosure) and makes information available to the web dashboard (from where the information is displayed (i.e. disclosed) to a much wider group of people). + +### Which features of IBEX display personal information? + * Experiment Details View + * IBEX Journal Viewer + +### Experiment Details View +The Experiment Details View displays the following fields: + 1. `Users` - contains the names of scientists performing experiment at ISIS. Names are clearly personal data. + 1. `RB Number` - why is the `RB Number` personal data? Because it is the key to the Experiment Details database. If you can get access to the Experiment Details database, you can use the `RB Number` to find out more about the PI and other user scientists. + +### IBEX Journal Viewer +The IBEX Journal Viewer (i.e. the journal viewer built into IBEX) displays the following fields: + 1. `Users` + 1. `User Institute` - could help identify a user, therefore, it's personal data. + 1. `Local Contact` - name of the instrument scientists, again it is obviously personal data. + 1. `RB Number`. + +### Do we display personal data on the web dashboard? +Yes, we do. In the following fields: + 1. `Users` + 1. `RB Number`. + +### Does this mean we can't display these fields in IBEX or on the web dashboard? +No, it does not mean that. DPA-2018 & GDPR permits personal data to be processed with the consent of the individual. When the PI (Principal Investigator) submits a proposal to ISIS, he/she is informed that certain fields will be made public, including his/her name, institute and the names of the other scientists involved in the proposal. By submitting a proposal, the PI has given consent. + +Similarly, ISIS publishes the names of instrument scientists (i.e. local contact) on its web pages. It is part of an instrument scientist's job that he/she is available for users (and potential users) to consult - hence they have consented for some personal information (e.g. name and a work e-mail address) to be used. + +Therefore, we are clear to use the `Users` and `RB Numbers` fields. + * Business Apps have confirmed that for proposals awarded beam time, the `proposal title`, `abstract`, and `experimenter names` (both the `PI` and `Co-Is`) are public data for the following types of proposals: Direct, Rapid, Dutch, Riken, and Indian Access. + * Business Apps also state that information relating to ICRD proposals is not published. ICRD = ISIS Collaborative R&D, meaning proposals made by industrial partners. Identifiable information from ICRD proposals should not be displayed on the web-dashboard (i.e. the scientists should suppress the display of the `Experiment Title` and leave the `Users` field blank). + +Does this mean we can display any fields containing personal data on the web dashboard? No, it does not. We should only display those fields that a PI has consented to display. In fact, on a precautionary basis, we should display no more information than is necessary. The `Users` and `RB Number` fields are sufficient. There is no need to display any more. + +### What about the `Experiment Title` field on the web dashboard? +The `Experiment Title` field cannot be used to identify an individual. We also give users the option to suppress the display of the `Experiment Title` field (via the DAE View). If a user or scientists wishes the +`Experiment Title` not to be displayed, they have the means to do so. + +### What about the Experiment Details View and IBEX Journal Viewer in IBEX? +For the same reasons as as above, we can display the `Users`, `User Institute`, `Local Contact` and `RB Numbers` fields in the Experiment Details View and IBEX Journal Viewer. IBEX is, of course, not accessible to external users (i.e. users outside the STFC firewall), therefore, any information displayed by IBEX is much less visible. Again, the precautionary principle should apply: we should display no more information than is necessary. diff --git a/doc/processes/admin/Obtaining-access-to-the-Instrument-Hall.md b/doc/processes/admin/Obtaining-access-to-the-Instrument-Hall.md new file mode 100644 index 000000000..2f7491f04 --- /dev/null +++ b/doc/processes/admin/Obtaining-access-to-the-Instrument-Hall.md @@ -0,0 +1,8 @@ +# Access to ISIS Instrument Halls + +To obtain access to the instrument hall, you need to have an SI10 form filled out and sent to the correct person and completed the radiation training. You can then enter the hall with a temporary dosimeter and temporary access given from MCR which lasts until the end of the month. + +To gain permanent access to the hall, you need to have a named dosimeter. To obtain a named dosimeter, you need to fill out a named dosimeter form which you can request from R5.5 reception. Once you have a named dosimeter, you can then gain permanent access to the hall from MCR. + +**N.B.** +Until you have obtained a named dosimeter, you will not be able to gain permanent access to the instrument hall. \ No newline at end of file diff --git a/doc/processes/admin/Remote-Working-vnc-cloud.png b/doc/processes/admin/Remote-Working-vnc-cloud.png new file mode 100644 index 000000000..af04d5081 Binary files /dev/null and b/doc/processes/admin/Remote-Working-vnc-cloud.png differ diff --git a/doc/processes/admin/Remote-Working.md b/doc/processes/admin/Remote-Working.md new file mode 100644 index 000000000..6672c9fce --- /dev/null +++ b/doc/processes/admin/Remote-Working.md @@ -0,0 +1,68 @@ +# Remote Working (VNC Cloud) + +## Proposed structure for VNC Cloud Groups: + +![](Remote-Working-vnc-cloud.png) + +## Proposed procedure for setting up VNC Cloud access: + +**Owner / Admin (Single Experiment Controls group member)** + +1. Enables "global" Two-Factor Authentication for _all_ users of the VNC Cloud system, in accordance with site security advice. +1. Creates "Machine Groups" in VNC portal all within the ISIS "Team", one per physical ISIS instrument e.g. "LMX" (See diagram above) +1. Creates "People Groups", one per instrument (e.g. "LMX Instrument Scientists" or perhaps "LMX Users" containing instrument scientists _and_ external users for simplicity) +1. Grants access to Machine Group to appropriate People Group +1. Invites Instrument Scientists (IS) to create a VNC account via VNC portal (invitation email) +1. Grants "Manager" privilege to IS +1. Adds IS to appropriate People Group(s) + +**Manager (IS or other ExptCtrl member)** + +1. Creates VNC Cloud account using link in invitation email from Owner/Admin +1. Sends emails to external users via VNC Cloud portal inviting them to create a VNC account +1. Grants "User" privilege to new users +1. Adds users to appropriate People (instrument) Group(s) (and removes when experiment over) +1. (Optional if willing & able) Installs VNC Server on the relevant machine(s) (e.g. NDCxxx & NDLxxx) via conventional VPN and RDP. (Details in "Deployment" section of VNC Cloud portal). More help in [VNC Article](https://help.realvnc.com/hc/en-us/articles/360002253198-Installing-and-Removing-VNC-Connect#windows-0-0) +1. Enables "Cloud Connectivity" in Server options. More help in [VNC Article](https://help.realvnc.com/hc/en-us/articles/360002249737-All-About-Cloud-Connections#enabling-cloud-connectivity-on-a-remote-computer-0-1) +1. Adds "local" computers (viewing, analysis, etc.) to Machine Group(s) (See diagram above) +1. Performs below steps to use VNC client + +**User (External facility user):** + +1. Creates VNC cloud account using link in invitation email from IS +1. Downloads, installs and runs VNC client. +1. Logs in and is presented with list of machines authorised to connect to +1. Connects to a machine, typically a general access cabin PC (NDCxxx) or analysis machine (NDLxxx) +_[at this point has same access as if physically present in instrument cabin]_ +1. Connects to instrument control computer (NDXxxx) via RDP (if session not already established) + +## Read/Write and Read-only + +The _Read Only_ and _Read/Write_ access is to be controlled by changing the privileges on the viewing machine's standard instrument user account (ISUA). + +This will be for the simplest case for when the viewing machine is running Windows and has (ISIS instrument) standard local user and admin accounts, although the principal still applies to a Linux analysis machine and other cabin machines. + +**Procedure:** + +1. Instrument scientist adds ISUA to access list in "Users and Permissions" section of VNC Server options +1. IS changes permissions in VNC server settings for ISUA to be either R or R/W to suit experiment +1. User runs local VNC client and logs in using their personal account +1. User sees list of available machines and connects to one using ISUA +1. Views (if R only and connection previously established by IS) **OR** +1. Connects (if R/W) to screen of instrument control computer via RDP if no active session, or "local" VNC client (as if in cabin in person) +1. Instrument scientist sets privileges of ISUA back to Read Only after experiment ends +1. (Optional) IS removes user from "Instrument People group" in VNC Cloud portal + +See [VNC article](https://help.realvnc.com/hc/en-us/articles/360002253618-Managing-Users-and-Session-Permissions-for-VNC-Server) for more information. + +**NB** + +When connecting using an account which has _Read Only_ access, the Users will have **_no control_** over the remote computer whatsoever, not even being able to connect to the control machine. This option is severely limited (by design) and so relies heavily on the IS to create and leave the RDP session to the control machine in a state which will provide sufficient information to their Users. + +The _Read / Write_ option on the other hand, offers **_full control_** of the remote computer and so the IS needs to consider carefully the implications of allowing Users to connect with this privilege level. + +## Troubleshooting + +If you can connect via cloud VNC but get a blank screen that you can do nothing with, this may be because no monitor is attached to the computer. The cloud VNC mechanism needs to have the vnc server program running in service mode (as a windows service), and this seems to need a screen of some sort. We run VNC server in user mode on the NDX and this is happy just having an active remote desktop session rather than a real screen, but user mode only allows point to point and not cloud connections. + +We have purchased some "screen dongles" that can be used instead of a real monitor, these attach to e.g. a display port adapter port on the PC. diff --git a/doc/processes/continuous_integration/Adding-a-new-Windows-machine-to-Jenkins.md b/doc/processes/continuous_integration/Adding-a-new-Windows-machine-to-Jenkins.md new file mode 100644 index 000000000..0ad265c27 --- /dev/null +++ b/doc/processes/continuous_integration/Adding-a-new-Windows-machine-to-Jenkins.md @@ -0,0 +1,54 @@ +# Adding a new Windows machine to Jenkins + +These are instructions for adding a new Windows machine as a node to be used by Jenkins. + +### Initial Preparation + +* First set up the machine so it can be used to build the back-end system manually by following these [instructions](/overview/First-Time-Build) + +* After you have registered the ISISICP go into `c:\Instrument\Apps\EPICS\ICP_Binaries\isisdae` and read `README_isisicp_reg.txt` in particular you will probably need to add the the account running jenkins to the windows group mentioned + +* Delete the EPICS subdirectory that was created in the previous step (maintaining C:\Instrument\Apps) + +### Adding to Jenkins + +* Go to ​https://epics-jenkins.isis.rl.ac.uk/computer/ and log in to Jenkins + +* Create a `New Node` with the Node Name as the computer name, select 'Permanent Agent' + +* Set a root directory of C:\Jenkins + +* Set a label of 'windows' + +* Select the `Launch agent by connecting it to the controller` + +* Save + +* Select the slave that has just been created and make a note of secret. For an initial test make a note of the curl and java commands from `Run from agent command line: (Windows)` + +* in a command window in c:\jenkins run the curl and java commands. These can be put into a bat file for interactive running. + +## setup as service + +We use https://github.com/jenkinsci/windows-slave-installer-module and https://github.com/winsw/winsw the relevant files are in +`\\isis\shares\ISIS_Experiment_Controls_Public\third_party_installers\latest_versions\builderserver` to copy to `c:\Jenkins` + +Copy `jenkins-agent.exe` and `jenkins-agent.xml` into same directory on the target machine e.g. `c:\jenkins` + +Edit `jenkins-agent.xml` and change https://epics-jenkins.isis.rl.ac.uk/computer/COMPUTER/jenkins-agent.jnlp and the SECRET field to the same as they are from the Jenkins' Node page, add `-workDir` argument of `c:\jenkins` +COMPUTER should be capitalised in same way as written on Jenkins. + +Open an admin cmd window and run `jenkins-agent.exe install` and then `servies.msc` + +Find the jenkins service in the Service Manager window on the machine, and change it to run as `isis\ibexbuilder` rather than local service account, you'll need to enter `ISISBuilder` password. + +Then run start service from the Service Manager window. + + +### Special Notes on Jenkins for GUI Tests + +The GUI test currently run on NDWRENO. The test behaviour is different when Jenkins is running as a service, so instead it is run as a command from a batch file. The batch file is located on the desktop for the user builder. On first start up this shows running in a console Window, but this is unintentionally hidden the the system tests. (I am not sure that this is true anymore - John) + +In case NDWRENO goes offline the command to run the slave is also shown at [http://epics-jenkins.isis.rl.ac.uk/computer/ndwreno/](http://epics-jenkins.isis.rl.ac.uk/computer/ndwreno/). + +System tests need the RCPTT plugin this is downloaded from here (http://www.eclipse.org/rcptt/download/), currently on 2.1.0. Once downloaded extract them to C:\Jenkins\RCPTT_Runner (set in runner.cmd). The current version appears to need java 1.8 and because the instrument add the latest JDK you need to install JDK 1.8. The version of java that jenkins slave uses is set in `jenkins-clas.xml` in `c:\jenkins`. \ No newline at end of file diff --git a/doc/processes/continuous_integration/Jenkins-Build-Server.md b/doc/processes/continuous_integration/Jenkins-Build-Server.md new file mode 100644 index 000000000..8baf3c7fd --- /dev/null +++ b/doc/processes/continuous_integration/Jenkins-Build-Server.md @@ -0,0 +1,75 @@ +# Jenkins Build Server + +There is a Jenkins Build Server located at [http://epics-jenkins.isis.rl.ac.uk/](http://epics-jenkins.isis.rl.ac.uk/). + +For information on configuring Jenkins see [Adding a new Windows machine to Jenkins](Adding-a-new-Windows-machine-to-Jenkins). + +## What is built + +### EPICS + +* Builds EPICS itself and the IOCs +* Runs the unit tests for the BlockServer and the DatabaseServer +* Runs a Python script as a series of unit tests to check the units in all the EPICS db files +* If the above was successful the compiled binaries are made available at `\\isis\inst$\Kits$\CompGroup\ICP\EPICS` + +### Client + +* The [`ibex_gui`](http://epics-jenkins.isis.rl.ac.uk/job/ibex_gui_pipeline/) build uses the `master` branch for the [IBEX GUI](https://github.com/ISISComputingGroup/ibex_gui) +* The `ibex_gui_build_PRs` builds all the PRs for the `ibex_gui` branch +* The Tycho build is run, which is slightly different to building within Eclipse. This can be run locally by running the `build/build.bat` file that is checked out with the client. +* The Tycho build also runs the unit tests +* Jenkins additionally runs CheckStyle and shows a fail build if there is an increase +* Jenkins builds will bundle the JRE with the client. The version of the JRE bundled is located at `\\isis\inst$\Kits$\CompGroup\ICP\ibex_client_jre`. This should be updated occasionally. +* If successful The builds are output to `\\isis\inst$\Kits$\CompGroup\ICP\Client`, including an installer. + +### genie_python + +genie_python gets built and put at `\\isis\inst$\Kits$\CompGroup\ICP\Client\genie_python`. There is a batch file to do the installation. + +## Setting up a Pipeline Build + +Pipeline builds are created by adding a Jenkins file to you root directory and then pointing Jenkins at your repository. + +### Jenkins File + +Defines all the steps there is much documentation on line. Start from a previous file which does something similar and edit. Here is an example [in the ibex gui](https://github.com/ISISComputingGroup/ibex_gui/blob/master/Jenkinsfile). In it the label is the label of the node on which it is to run. The labels are set by editing the node configuration in jenkins. + +Once this is set up create a new pipeline build in Jenkins. + +1. In jenkins click "New Item" +2. Name the project and select Pipeline +3. Under pipeline from SCM select `Git` + 1. Then add the repository url in and the credentials as ISISBuildServer + 1. Set the branch if needed + +This should now build. + + +## Setting up a Multi-branch pipeline + +Jenkins Multi-branch Pipeline project type enables you to implement different Jenkinsfile for different +branches of the same project. Additionally, we can make Jenkins to only create and execute pipeline +for a specific branch. + +1. Go to http://epics-jenkins.isis.rl.ac.uk/ and log in using your federal ID and password. +1. Click New Item on the top left corner of the dashboard. +1. Enter the name, select Multi-branch pipeline and click OK. +1. Add Display Name and Description (Not mandatory but would be useful) +1. Under `Branch Sources` tab click `Add source` and select `GitHub` + 1. Choose credentials User ISISBuilder + 1. Add Repository HTTPS URL (your git repo) + 1. Under `Behaviors` -> `Discover branches` select `Strategy` from the drop-down list to be + `Exclude branches that are also filed as PRs` + 1. Under `Behaviors` click `Add` and choose `Filter by name (with wildcards)` and fill the `Include` text field + with `Release_*` or whatever is the release prefix for your release branch and remove the rest of the options. + 1. Under `Property Strategy`, click `Add property` and choose `Suppress automatic SCM triggering`. +1. Under `Build Configuration` tab, `Script Path` should match the name of your Jenkinsfile +1. Click Save + +This will build now. To start the build again click `Scan Repository Now` + +## Other + +* [Jenkins trouble shooting](Jenkins-Trouble-Shooting) +* Jenkins builds use the office 365 connector plugin (https://github.com/jenkinsci/office-365-connector-plugin) to update the teams channel with build information. Examples of how this can be configured can be found in the Jenkins files of the top-level EPICS repository and the GUI repository. \ No newline at end of file diff --git a/doc/processes/continuous_integration/Jenkins-Trouble-Shooting.md b/doc/processes/continuous_integration/Jenkins-Trouble-Shooting.md new file mode 100644 index 000000000..87ba1cdff --- /dev/null +++ b/doc/processes/continuous_integration/Jenkins-Trouble-Shooting.md @@ -0,0 +1,80 @@ +# Jenkins Trouble Shooting + +## Ibex-Gui build failure + +### Lack of randomness? + +Build stops with: +``` + Failed to execute goal org.eclipse.tycho:tycho-surefire-plugin:0.20.0:test (default-test) on project uk.ac.stfc.isis.ibex.targets.tests: An unexpected error occurred (return code 13). See log for details. -> [Help 1] +``` +In log file +``` + java.lang.InternalError: Unexpected CryptoAPI failure generating seed +``` +This might be caused by not enough randomness in the system (not sure). It went away once I logged into the server (which appeared to be new) and re-ran the build. + +### General + +## No tests Run + +Look in console log. We found the lines: + +``` +RCPTT Runner version: 2.0 +An error has occurred. See the log file +C:\Jenkins\workspace\System_Tests\Results\runner-workspace\.metadata\.log. +``` + +In this log it couldn't read open a given jar file. This appeared to be a problem with the version of RCPPT. So ran manually on `reno` and fixed the problems. + +## Build Agent is OffLine + +Remote desktop to the machine in question and start the Jenkins slave service using services.msc. + +If the service fails to start have a look in `C:\Jenkins` at `jenkins-slave.wrapper.log` it shows what the command line is. Run the command line and see the error then fix it. If it is the path to java this is set in `jenkins-slave.xml`. If you have some time you might try and couple this to the system path so that we no longer need the path in here. + +## Java is out of date + +If the wrong java path is set then the slave will not start. Update the path to the correct version of java in `...\Jenkins\jenkins-slave.xml`. + +## Linux build node (`sl7cloud`) is offline + +If the node has rebooted and/or installed updates then it may have removed `jenkins` from its ssh authority. You need to edit `/etc/ssh/sshd_config` (e.g. `sudo vi /etc/ssh/sshd_config` and add `jenkins` to the end of the `AllowGroups` line. Then run `sudo service sshd restart` on `sl7cloud`. You can then go to the webpage for `sl7cloud` on Jenkins and ask it to `relaunch agent` on the node. + +To log onto the node you need to `ssh` to its real hostname - you can get this by clicking on the `sl7cloud` node in jenkins, going to `configure` and then you will see a `.nubes.stfc.ac.uk` hostname to use. Log in with fed id, authentication is via certificate. You should be able to sudo from your account. + +Log into the web page `https://cloud.stfc.ac.uk/` for an overview of all our VMs and `https://openstack.stfc.ac.uk/` for a more detailed management interface + +## Builds failing to check out + +If a jenkins job (often epics-static-clean on NDWVEGAS) is failing to run and near the start you see something like +``` +21:47:24 > git submodule foreach --recursive git reset --hard # timeout=10 +21:51:27 ERROR: Error fetching remote repo 'origin' +21:51:27 hudson.plugins.git.GitException: Failed to fetch from https://github.com/ISISComputingGroup/EPICS.git +21:51:27 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:909) +21:51:27 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1131) +21:51:27 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) +21:51:27 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125) +21:51:27 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93) +21:51:27 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80) +21:51:27 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) +21:51:27 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) +21:51:27 at java.util.concurrent.FutureTask.run(FutureTask.java:266) +21:51:27 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) +21:51:27 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) +21:51:27 at java.lang.Thread.run(Thread.java:748) +21:51:27 Caused by: hudson.plugins.git.GitException: Command "git submodule foreach --recursive git reset --hard" returned status code 128: +``` +This is usually caused by an `index.lock` file not having been removed correctly during a previous git operation. This file should only exist when git is running, if there are no git processes executing on that repository then there should be no `index.lock` file present. + +You can search the workspace for these files, but there is a now a program to do this for you (which also checks for running git processes). See `\\isis\shares\ISIS_Experiment_Controls\git_lock_clean` + +## [Office365connector] Matched status 'FAILURE' for webhook with name 'Office 365'. + +This is not an error with the `Office365connector`, it is reporting that the connector is being run because it had been configured to match a 'FAILURE' state and the build had failed. You will likely see this message at the end of most failed builds, the real reason for the build failure will be earlier in the jenkins log file. + +## Builds not Building on 1926 + +Connect to the computer and start the Jenkins Slave, the Beckhoff tests mean this has to be done manually on this system \ No newline at end of file diff --git a/doc/processes/continuous_integration/RHEL-jenkins-linux-dependency-update.md b/doc/processes/continuous_integration/RHEL-jenkins-linux-dependency-update.md new file mode 100644 index 000000000..c7a4ec69c --- /dev/null +++ b/doc/processes/continuous_integration/RHEL-jenkins-linux-dependency-update.md @@ -0,0 +1,19 @@ +# Updating the RHEL Jenkins Node + +## Update Java + +* download latest jdk17 for linux and copy to a tree in /opt/jdk17 +* edit `build.sh` in client gui build area to set correct `JAVA_HOME` path + +## Update Python + +* download python source +* on RHEL7 there is an issue with openssl versions. Open ssl 1.1 can be installed from yum, but will not be in the right directory for python. Make sure you install the openssl11-dev and openssl11-static packages as we will link statically to it +* create a `/usr/local/openssl11` with an include directory linked to /usr/include/openssl11 and a lib64 directory linked to /usr/lib64/openssl11 +* configure with +``` +env LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-Bstatic -lssl -lcrypto -Wl,-Bdynamic" ./configure --prefix=/usr/local/python --with-openssl=/usr/local/openssl11 +``` +Note that passing `--enable-optimizations` lead to a subsequent build failure +* now type `make` to build +* finally `sudo rm -fr /usr/local/python` and `sudo make insytall` diff --git a/doc/processes/continuous_integration/Running-Automated-System-Tests.md b/doc/processes/continuous_integration/Running-Automated-System-Tests.md new file mode 100644 index 000000000..63817802f --- /dev/null +++ b/doc/processes/continuous_integration/Running-Automated-System-Tests.md @@ -0,0 +1,3 @@ +# Running automated system tests + +Instructions for running automated system tests can be found at: https://github.com/ISISComputingGroup/system_tests/blob/master/README.md \ No newline at end of file diff --git a/doc/processes/continuous_integration/TestBuild-Jenkins-Build.md b/doc/processes/continuous_integration/TestBuild-Jenkins-Build.md new file mode 100644 index 000000000..534053a2a --- /dev/null +++ b/doc/processes/continuous_integration/TestBuild-Jenkins-Build.md @@ -0,0 +1,16 @@ +# TestBuild Jenkins Build + +The jenkins job `TestBuild` builds the branch `testbuild` in the top EPICS git repository and deploys it to a `TestBuild` directory in `kits$\EPICS`. The job is not run automatically on commit - you need to trigger it with `Build With Parameters` in Jenkins GUI at which point you could change e.g. the `EPICS_HOST_ARCH` from `windows-x64` (default) to `windows-x64-debug` if you wished. + +To specify your build do the following: +- create and push branches in each submodule where you want to test changes. You do not need to create pull requests, and can call the branches any name you wish. +- in your EPICS top level, if you already have a `testbuild` branch delete this with `git branch -d testbuild` +- check if a `testbuild` branch exists on GitHub https://github.com/ISISComputingGroup/EPICS/branches - if it does, check the other developer has finished and then delete this GitHub remote branch. If they are still using it, then it is probably based on a quite recent commit and you may be able to add your changes to this branch and both use the same jenkins build. +- If you deleted the remote branch, create a local `testbuild` branch with `git checkout -b testbuild` in your top level EPICS. If you are sharing `testbuild` with another developer, use `git checkout testbuild` which will get the current GitHub `testbuild` +- now edit `.gitmodules` and for each of the submodules you created a separate `my_branch_name` branch in add a corresponding `branch = my_branch_name` line to the relevant section of the file +- not commit your `.gitmodule` changes and push to `testbuild` on GitHub +- now start the jenkins build from the GUI + +The jenkins job builds "tip of branch" i.e. it will use latest master of all submodules and also latest commits to any `branch =` lines you added. So you do not need to "git add" any submodule directories, and if you push new changes to any branch you can just start a new jenkins build and they will be picked. + + diff --git a/doc/processes/dev_processes/Component-and-Activity-Stewards.md b/doc/processes/dev_processes/Component-and-Activity-Stewards.md new file mode 100644 index 000000000..34c925efc --- /dev/null +++ b/doc/processes/dev_processes/Component-and-Activity-Stewards.md @@ -0,0 +1,57 @@ +# Component & Activity Stewards + +This page lists the component & activity stewards. + +## What is a component / activity steward? +A Component or Activity Steward is the individual responsible for the development and quality of a component or activity. The role itself is [defined in more detail](#definition_component_steward) below but, in essence, it means that the named individual is responsible for the "well-being" of a component or activity. Knowing who the relevant steward is means you have a first point of contact for finding out about that particular component or activity. + +After a component/activity has been completed and stable for a cycle then the steward should retire for that activity/component. + +## List of Components/Activities & Stewards +The list of components/activities and stewards is not set in stone. It will change over time. + +Component/Activity | Steward | +--- | --- | +**Instrument Migrations** | | +MUSR Migration | LC | +OFFSPEC Migration | JH | +HIFI Migration | TBC | +CHRONUS Migration | LC & DK | +ARGUS Migration | LC & DK | +SXD Migration | LJ | +CHIPIR Migration | JH | +--- | --- | +**IBEX Core components** | | +Script Server (NICOS) | | +Graphing/Scripting | | +Script Generator | | +IBEX GUI | | +CSS & Phoebus | | +EPICS base and support modules | FA | +Delay tickets | | +Blockserver/Other server items | | +Linter | | +Squish | | +Motion Control | JH | +Reflectometry | JH | +Dataweb | | +--- | --- | +**Other items which impact significantly and directly on IBEX** | | +Detector systems monitoring | | +HIFI Cryomagnet | LC | +Systems (Hardware, OS, etc.) | CMS | +Lewis | | + +{#definition_component_steward} +## Definition of the role of Component / Activity Steward +The role of Component or Activity Steward is to be the individual responsible for the development and quality of a component or activity. An example of a component is the Blockserver, or a major part of the IBEX GUI. An example of an activity might be ensuring that an instrument is ready to migrate to IBEX (i.e. by ensuring that IBEX support the devices used on that instrument). + +Being responsible does not mean that you are the sole developer or the only person who ever works on the component. Rather it means that you are the individual responsible for the "well-being" of the component, on an on-going basis. In other words, making sure that the component fulfils and continues to fulfil its requirements, meets quality standards, has up-to-date and accurate documentation, that bugs get fixed, etc. It may also mean having to liaise with other members of the team, or with colleagues in other groups to make sure that components, services or hardware required by your component are identified, planned and in place at the right time. + +Similarly, for an activity, you are not expected to be the only person who contributes to that activity. Instead, you are the individual responsible for identifying the actions that need to be taken to complete the activity and helping the team plan the work required. + +In both cases, component or activity, you will be involved in creating or identifying the stories that guide the development of the component or activity and discussing the relative prioritisation of those stories (i.e. when the work should be scheduled). Depending on your role in the team, you might be responsible for several components or activities. + +In practice, it is likely that you will perform a large fraction of the planning, design, development, testing and documentation, but there is no reason why others should not contribute - after all, two (or even more) heads can be better than one. The benefit of involving other individuals is that they can propose new ideas, suggest alternative approaches, bring specific expertise or just generally help share the load. Indeed, it is important that more than one person has knowledge of a component; someone who can take over, if necessary, if you are absent or busy working on something else - you should actively seek to involve others in the development or implementation of your component or activity. + +The Component or Activity Steward does not act alone. You need to coordinate your activities with other members of the team, including the project manager, keeping them informed about overall progress, events and problems. For example, explaining to the team the timescales involved, whether specific events have critical timings or dependencies, whether special resources are required. If you encounter a problem, let people know - they might be able to help. It is important that others are kept informed about the state of your component or activity, so that they can take account of these matters for their own components or activities. diff --git a/doc/processes/dev_processes/Dependency-Updates.md b/doc/processes/dev_processes/Dependency-Updates.md new file mode 100644 index 000000000..46c117795 --- /dev/null +++ b/doc/processes/dev_processes/Dependency-Updates.md @@ -0,0 +1,228 @@ +# Dependency Updates + +After a release all of the dependencies of the system should be considered for update. This will ensure that we do not get too far out of date and any upgrade will, hopefully, be small and not require much effort. In general we do not want to be on the bleeding edge but at the last stable release (i.e. prefer LTS versions when they are available). The list of dependency will get shipped with the release notes and should be kept up-to-date with any notes. This page documents the process of updating. + +**Any dependencies which should not be updated should be listed with a reason [here](/processes/dev_processes/Unupdated-dependencies)** + +**When updating dependencies add the new dependencies to the upcoming release notes** + +## WebDashboard + +- [Update `tomcat` on external webserver](https://github.com/isiscomputinggroup/pvws-config?tab=readme-ov-file#updating). This is an external-facing process so **must** be kept up-to-date with latest security bug fixes. +- Update java JDK on external webserver. This is used to run tomcat, which is an external-facing process, so **must** be kept up-to-date with latest security bug fixes. +- Update [javascript dependencies](https://github.com/ISISComputingGroup/WebDashboard/blob/main/package.json) + +## GUI + +The GUI has multiple mechanisms for pulling in dependencies, all of which may potentially need updates. You should aim to do one dependency update at a time, testing each update both under the eclipse IDE and maven and committing before moving onto the next update. + +### `.jar` files + +We now only reference a single `.jar` file (`opal`) from the GUI. This is not currently being updated as the project has moved to nebula visualisation widgets and no new versions are being produced. + +### Target platform + +This is defined in the file `./base/uk.ac.stfc.isis.ibex.targetplatform/targetplatform.target` + +General update process: +- Look for newer versions of the repository. E.g. if the repository url is `http://some.website/releases/1.23`, try visiting `http://some.website/releases` to see whether the releases are listed. If so, point the target platform at the new releases. +- Some repositories are updated "in-place". Do upgrade these, simply delete and then re-add them to the target platform, when they are re-added they will pick up the latest versions. +- For maven dependencies referenced in target platform, look up latest version on maven central then update the version number in target platform to correspond. If the version numbers are hardcoded in `feature.xml` or `MANIFEST.MF` for individual plugins, update it there too. + +**Note: when updating the eclipse framework itself, you will need to download the same eclipse IDE with the same version number or else some jars may not be found. You also need to update `client.tycho.parent` - see below for details. Make sure you update the [gui build wiki page](/client/getting_started/Building-the-GUI) to ensure new starters get the correct version.** + +### Parent POM + +Tycho and eclipse framework versions are defined in the file `./base/uk.ac.stfc.isis.ibex.client.tycho.parent/pom.xml`. Note that the eclipse framework version *must* correspond with the version defined in the target platform, otherwise the maven build will fail with mismatched framework versions. + +### Client JRE + +The GUI builds copy a JRE from `\\isis\inst$\Kits$\CompGroup\ICP\ibex_client_jdk`. Copy the latest jdk to ICP, rename to follow the same format, and then update `ibex_gui\build\build.bat` to point at the new JRE location. + +### Pydev + +- git clone --recurse-submodules the latest version of [our fork](https://github.com/ISISComputingGroup/Pydev) and create a new dependency update branch off of master. +- git clone --recurse-submodules the latest version of [the upstream](https://github.com/fabioz/Pydev) on to a vendor branch and merge this branch into the dependency update branch. +- Run `mvn install` in the dependency update branch base directory. +- After a successful build, create a PR to merge the changes into master, and upload the updated repo to `\\shadow.isis.cclrc.ac.uk\ICP_P2W$`, this latest version should be named 'Pydev'. +- Remove and re-add the Pydev target platform dependency in the GUI + +## Python + +### Uktena python distribution + +- Check on Python.org for newer versions of python itself +- If a newer version is available, download the "windows installer". +- Select custom install, install python to a location of your choice (not `c:\instrument\apps\python3`). +- During the python installation process, **ensure that you tick the box asking whether you want to install TCL/TK support** in optional features. This is needed for independent matplotlib plots in standalone genie_python windows. +- Also during the python installation process, in advanced options screen select both `download debugging symbols` and `download debug binaries` +- Zip up the resulting directory and add it to the share at `\\isis\inst$\Kits$\CompGroup\ICP\genie_python_dependencies_python_3` as `python_clean_.zip` +- Edit `package_builder\common_build_python_3.bat` to use the new python version and test that all dependencies work correctly + +### Our packages + +We depend on a variety of python packages that we publish ourselves, either via pypi or via git. + +These may need to be updated to allow new python versions. For packages published on PyPI, a new pypi release will need to be made. + +These packages include: +- `genie` +- `ibex_bluesky_core` +- `lewis` +- `pcaspy` +- `CaChannel` +- `epicscorelibs_pcas` + +### External packages + +Check on PyPi for any package updates, then edit `requirements.txt` to install new versions where needed. Note that since we decided [all python projects should use virtual environments](/system_components/python/Python-dependencies) there will be a `pyproject.toml` or legacy `requirements.txt` file for all Python projects using the new import mechanism, ensure these are also updated. + +### ODE + +**We are not currently distributing ODE with `uktena`** - nothing needs to be done. + + + +Old instructions + +
+ +ODE is handled separately from other packages and is installed from a wheel on `\\isis\inst$\Kits$\CompGroup\ICP\genie_python_dependencies_python_3` if moving to a new python version i.e. 3.10 to 3.11 this wheel will need to be replaced. +* First check [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#ode) for a matching version of ODE, if one is not present one will need to be built, to do so: + * Make a note of the latest release tag from the [ODE Bitbucket](https://bitbucket.org/odedevs/ode/downloads/?tab=tags) + * open a cmd window and cd to e.g. `c:\devel` + * run `git clone https://bitbucket.org/odedevs/ode.git` + * If `0.16.4` is your version to build then `cd ode && git checkout 0.16.4` + * type `set "CMAKE=c:\Instrument\Apps\EPICS\ICP_Binaries\CMake\bin\cmake.exe"` + * type `mkdir ode-build && cd ode-build` + * run `"%CMAKE%" -G"Visual Studio 16 2022" -A x64 ..` + * type `start ode.sln` + * change to `Debug` to `Release` configuration at top, it should already say x64, and from menu build -> build solution + * now cd to the `bindings\python directory` + * open ode.pyx, change definition of `collide_callback` to add `noexcept` i.e. `cdef void collide_callback(void* data, dGeomID o1, dGeomID o2):` to `cdef void collide_callback(void* data, dGeomID o1, dGeomID o2) noexcept:` + * Navigate to `bindings/python` and open the `setup.py` + * Add `from wheel.bdist_wheel import bdist_wheel` to the imports at top. + * remove the whole try/except clause that calls `pkg-config` from `Popen`, instead just set the variables explicitly e.g. `ode_cflags = [r'-IC:\devel\ODE\include', r'-IC:\devel\ODE\ode-build\include']` and `ode_libs = [r'C:\devel\ODE\ode-build\Release\ode_double.lib']` + * in main() Update the version number to match the version of ode used, and set the name to `ode`. + * run `%python3% setup.py build_ext` and then `%python3% setup.py bdist_wheel` + * copy the wheel generated in `dist` to `\\isis\inst$\Kits$\CompGroup\ICP\genie_python_dependencies_python_3` + * copy `ode_double.dll` from `C:\devel\ODE\ode-build\Release` to the same place +* Edit `common_build_python.bat` in `package_builder` to point to the most recent wheel file. +* Test by running `python run_all_tests.py` in `inst_servers`, which contains collision avoidance monitor tests +
+ + +## System + +### MySQL +Download the latest stable MySQL version as a zip file and put in the share `\\isis\inst$\Kits$\CompGroup\ICP\MySQL` and remove the old one. + +In the upgrade script `https://github.com/ISISComputingGroup/ibex_utils/tree/master/installation_and_upgrade/ibex_install_utils` edit `install_tasks.py` to point at the new version. + +Update the one in your local machine by running `upgrade_mysql.bat`. + +In `create_icp_binaries.bat`, update the unpacked version of MySQL used during the build to the new version added on the share. + +In `c:\instrument\apps\epics\support\mysql\master\MySQLCppApp\src\mysql-connector-c++`, update the vendor submodule to a recent version, then rebuild `mysql`, `pvdump`, and then all IOCs, in that order. + +### Java +- Get the latest AdoptOpen JDK msi file from `https://adoptium.net/releases.html?jvmVariant=hotspot` and put it in `\\isis\shares\ISIS_Experiment_Controls_Public\third_party_installers\latest_versions`. + * Only upgrade major versions to another LTS version. +- Copy the older version onto the `\\isis\shares\ISIS_Experiment_Controls_Public\third_party_installers\old_versions`. +- Uninstall the older version from control Panel and install the new version. +- Test that running `start_ibex_server.bat` completes successfully, and ensure there are no obvious errors in the IOC log files for: + * ARBLOCK + * ARINST + * ALARM + * IOCLOG +- Check that CS Studio IDE is loaded correctly + +### Maven +- There are two versions of maven, this is to update the one following the format `maven-X.x.x` +- Download the latest maven from `https://maven.apache.org/download.cgi#` +- Delete the older version of the maven +- unzip and copy the content to `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\Binaries` +- Update `copy_in_maven.bat` in `\ibex_gui\build\` to point at the new maven version + + +### Make + +- Download the latest make sources from `https://ftp.gnu.org/gnu/make/` +- In the downloaded source tree, open a command prompt +- Run `config_env.bat` (this runs a correct `vcvarsall.bat` for the current system) +- Run `build_w32.bat` +- Copy the built binary in `./WinRel/gnumake.exe` to `C:\instrument\apps\epics\utils_win32\master\bin\make.exe` + +### git + +Git upgrade is handled by install scripts. + +## CS-Studio + +### GUI + +Our CS-Studio GUI dependencies are located on a share on shadow, a read only version `\\shadow.isis.cclrc.ac.uk\ICP_P2$\` (which is accessible via a webpage at `http://shadow.nd.rl.ac.uk/ICP_P2/` and a writable version `\\shadow.isis.cclrc.ac.uk\ICP_P2W$\`. + +CS-Studio requires a version of jdk11 to build that it gets from `C:\Program Files\AdoptOpenJDK` or `C:\Program Files\Eclipse Adoptium`. +- Install the latest jdk11, and ensure `isis_css_top\build.bat` points at the correct jdk location. +- You will need JavaFX binaries. These can be patched onto the AdoptOpenJDK/Eclipse Temurin installation. Download the Windows SDK from \\isis\inst$\Kits$\CompGroup\ICP\Java_utils\openjfx-19_windows-x64_bin-sdk\javafx-sdk-19 (originally from [gluon](https://gluonhq.com/products/javafx/)) and copy the bin, lib, and legal directories over the corresponding directories in the jdk. Note that the JavaFX version does not necessarily need to match your java installation, as long as the versions are compatible. For example we can use JavaFX 19 on a Java 11 installation. Please check that the license is still appropriate before you install. + +To update the CS-Studio components that the GUI uses: +- `git clone --recursive https://github.com/ISISComputingGroup/isis_css_top.git` +- Make relevant changes to the code, make sure submodules get pinned to new versions using same workflow as in EPICS top. +- Trigger a build on Jenkins +- After the build is complete go to the build server and copy the entire isis_css_top tree to the share (`\\shadow.isis.cclrc.ac.uk\ICP_P2W$`) rename the folder to `css_gui_dependencies_`. +- Updated the gui target platform to point at the new folder, i.e. `http://shadow.nd.rl.ac.uk/ICP_P2/css_gui_dependencies_/p2repo/` +- Reload the target platform and rebuild the gui. +- Test that your changes work correctly! +- if you cannot write to the `ICP_P2W$` share on shadow, your fed id account will need adding to the `icp` local group on shadow itself. This just requires somebody to run the `vigr` command on shadow and then possibly `service smb restart` too + +### Archive engines / alarm servers + +- The build uses the same process as above +- Once build is done, the products are in `org.csstudio.sns\repository\products` +- Copy the zip files for alarm server, alarm config tool, archive engine and archive config tool to `\\isis.cclrc.ac.uk\inst$\Kits$\CompGroup\ICP\Binaries\CSS`, renaming the directories to match the existing names if necessary (The existing ones should be backed up). +- The changes will be picked up after doing `create_icp_binaries` and then `css\master\setup_css.bat` +- Ensure scripts to launch Archivers and alarm server [1](https://github.com/ISISComputingGroup/EPICS-CSS/blob/master/AlarmServer/run_alarm_server.bat) [2](https://github.com/ISISComputingGroup/EPICS-CSS/blob/master/ArchiveEngine/start_block_archiver.bat) [3](https://github.com/ISISComputingGroup/EPICS-CSS/blob/master/ArchiveEngine/start_inst_archiver.bat) point at correct executables after running `create_icp_binaries.bat` and `setup_css.bat`. + +## ActiveMQ + +To update ActiveMQ in epics: + - Create a new branch on `...\EPICS\ISIS\ActiveMQ\master`. + - Put the latest ActiveMQ version (from https://activemq.apache.org/) on the vendor branch that already exists on the repo. + - Merge the vendor branch into your new branch + - Create a PR to merge this in to master (this PR will be merged as part of the update dependencies ticket). + +## IOCLogServer + +The IOC log server has a similar build process to the main GUI, and includes several `.jar` files in it's repository. These may have new versions and need to be updated. + +Update `EPICS\ISIS\IocLogServer\master\LogServer\pom.xml` with new dependency version numbers. + +## Cygwin Tools + +`procServ`, `conserver` and `console` are cygwin applications, compiled copies of which are kept centrally in `ICP_Binaries\EPICS_Tools` and then copied into subdirectories of `tools/master` of the EPICS distribution during a build. Their source code is located in the https://github.com/ISISComputingGroup/EPICS-tools repository and they are built by the EPICS_tools Jenkins job which places new versions in `kits$\EPICS_Tools`. The Jenkins job only runs when requested. + +If you update the source code of procServ/conserver the following applies too, but we should also update `cygwin` and rebuild the binaries periodically even if the source code is unchanged. To publish new binaries: +- determine the host running the Jenkins job, look at the LABEL parameter of the build (this is currently ndhspare53) +- log onto the machine and update cygwin - goto https://www.cygwin.com/ in a web browser from the machine and download and run `setup-x86_64.exe` +- let it update packages + +There are two cygwin distributions on the computer in `c:\cygwin64` and `c:\mini_cygwin64`, the one in `cygwin64` is used to build programs and the one in `mini_cygwin64` is a minimal distribution that is copied to the instrument along with the built procServ/conserver programs to provide a runtime environment. Both of these need to be updated, so you will **need to run the above setup twice** changing the installation directory between runs. when you update `c:\mini_cygwin64` there will likely be very few packages and nothing additional needs to be done other than update it. After you update the main `cygwin64` you will need to disable `ASLR` on cygwin1.dll - you can do this by running from a command window `disable_aslr.bat` (this is in the EPICS/too/master directory). If cygwin was installed from an admin account, then `disable_aslr` will need to be run from an admin terminal too or else you will get an access denied error when it tries to update the dll + +- now start the jenkins EPICS_Tools job, this will build and update `kits$\Binaries\EPICS_Tools` + +## NICOS +[Nicos Dependency update steps](/system_components/nicos/Developing-NICOS). + +**Note:** +We are using a fork of NICOS. If updating NICOS, ensure that the protocol versions defined in the client and the server match. The server version is specified in `C:\Instrument\Apps\EPICS\ISIS\ScriptServer\master\nicos\protocols\daemon\classic.py` and the client version in `/uk.ac.stfc.isis.ibex.nicos/src/uk/ac/stfc/isis/ibex/nicos/messages/ReceiveBannerMessage.java`. If these do not match, the script server connection will fail. + +## Updates to consider + +- PCRE (https://github.com/ISISComputingGroup/EPICS-pcre) +- procServ +- EPICS base +- Support modules e.g. StreamDevice, asyn, Lua etc. +- Google test for EPICS (`gtest`) (https://github.com/epics-modules/gtest) + diff --git a/doc/processes/dev_processes/External-Contributions.md b/doc/processes/dev_processes/External-Contributions.md new file mode 100644 index 000000000..777a6ee21 --- /dev/null +++ b/doc/processes/dev_processes/External-Contributions.md @@ -0,0 +1,11 @@ +# External Contributions + +The majority of IBEX code is open source, it uses code from the EPICS community and contributes code towards it, it is also used by a number of scientists that have good software development skills. Therefore it's expected that code will be contributed from outside the core ISIS developers and this should be encouraged as much as possible as it allows the project to move more quickly and react to people's requirements. To encourage this we should expedite any contributions with the following process: + +* When a contribution comes in a ticket should be created for the work, if it has not been already. The code should then be put into a PR linked to the ticket in the usual way. If possible encourage the contributor to do this themselves. +* This ticket is a support ticket as we're supporting someone in getting their code into IBEX +* The ticket goes straight into review and is reviewed by one of us in the usual process. Points are added to the ticket based on how much time it took to do this review. Remember during review that the contributor will not necessarily be aware of the wider ramifications to other IBEX users, be sure to be mindful of this! +* If we're happy then it's merged, if not we put it as rework, explaining our reasoning behind the rework. We then ask the contributor if they are happy to do this work themselves and explain that if they are not we will propose it as usual for one of us to look at in the next sprint, but cannot promise it will get done +* Propose it if the contributor is not happy with the rework or if they do not respond in a reasonable amount of time (~2 weeks?) + +This process worked well for https://github.com/ISISComputingGroup/IBEX/issues/5776 \ No newline at end of file diff --git a/doc/processes/dev_processes/Good-Ticket-Writing.md b/doc/processes/dev_processes/Good-Ticket-Writing.md new file mode 100644 index 000000000..7124cac3a --- /dev/null +++ b/doc/processes/dev_processes/Good-Ticket-Writing.md @@ -0,0 +1,61 @@ +# Good Ticket Writing + +## Purpose of this guide +This description of the process is aimed to help improve communication, rather than be followed with no consideration as to what is needed for the members of the group to flourish, please edit it as required. + +## Parts of a ticket + +### Title +This should be a succinct representation of the content of the ticket. +Where there is a logical grouping of themes, using a `Keyword: Title` format can make it easier to find related tickets without having to create umbrella tickets + +### User Story +This is *why* the work needs to be done, and for *whom*. +It will be centred around an actor who is part of the process, whether that is a user, developer, someone providing support, someone with a role will have a need and the story should highlight who has that need. +It should state what needs to happen, along with the justification of why it needs to work that way. + +As a role/method of interacting with the system, I want it to behave in this way, so that something does/doesn't happen. + +This should amount to a requirement, one or two lines at most. + +### Acceptance Criteria +This is how we define **done**, so this is *the outcome* or *what* we need to achieve. Reading this set of checkboxes it should be easy for a developer or a reviewer to be able to say "yes it does that". These need to be clear. There should also be as few of this as is practical to match the user story. +## Notes +This is a good place to provide extra context, who to talk to, what might impact this work and similar. This is extra information + +### Design/Details +This is an aspect that is usually not included, but may be of benefit to people with less experience and knowledge of the codebase and ways of working within a team. +It is described as not included as some developers do provide this level of detail at times. (e.g. https://github.com/ISISComputingGroup/IBEX/issues/1208 gave great detail of the changes that were needed for that work). As has happened in some places, comments can be added to detail more of the technical specification, including command sets, UI designs, and so on. +Because these can be more proscriptive than just an acceptance criteria (this would be *how* to do it, not just describing when it is done) care should be taken as to when this section is included so that it does not stifle creativity or limit the approach to be taken. However, it can make the difference between being able to look at something and undertake it with confidence, or not, as well. + +## Random Example + +### Title +Coffee Meeting: Take a cup of black instant coffee with me +### User Story +As someone attending a coffee meeting I want a cup of black instant coffee to take with me so that I have something to drink during the meeting. +### Acceptance Criteria +- [ ] There is a cup of black coffee available for me to take +### Notes +- The nicest kettle is in the kitchen by the meeting room +- The clean mugs are usually in the cupboard over the sink +- It may be worth double checking which brands are available so that something can be brought in if necessary +### Design/Details +1. Check to see if there is a suitable coffee brand in that kitchen the day before, if not bring some from home +2. 10 minutes before the meeting head off to the meeting room to have 5 minutes to make the coffee before the meeting once you get there, as the nicest kettle is in that kitchen +3. Fill the kettle to either the min level or to the number of people wanting to use it, this can't be predicted before arriving there +4. Whilst the kettle boils, get a mug from the cupboard, and put a teaspoon of coffee powder in it +5. Once the kettle has boiled, add water to the mug and stir +### How it maps to the purposes of each section +Title: If you know what you are doing it is enough to meet the acceptance criteria +User Story: This clarifies who, and why, it's someone attending the meeting, and they want to have something drink with them +Acceptance Criteria: This is clear, there is either a cup of coffee or there isn't, Schrödinger's Coffee is not an experiment worth conducting in this scenario. +Notes: Some extra context and information +Design/Detail: This is not at so low a level that there is no room for creativity, and it assumes some knowledge that should be straightforward enough for the majority of people. But, it does specify enough that someone who has never done this before has at least a good chance of succeeding when combined with the notes as that specified which cupboard. + +## When to add Design/Details +This will vary based on the task and scenario. +Sometimes that design/detail will be in other documents, and they could be referenced, especially for new items where interfaces have probably been discussed in greater detail. +If there are new team members this can be a benefit to them, as it breaks apart the system/architecture design side from the developer side. +If there are no other clear requirements anywhere, such as with some of the reverse engineering that has been undertaken in the past. That way the ticket has the history, but the wiki can stay rooted in the present. +It could be added before the ticket is considered at all by the team, but it might also be something to add after prioritisation in certain situations. diff --git a/doc/processes/dev_processes/Project-Documentation.md b/doc/processes/dev_processes/Project-Documentation.md new file mode 100644 index 000000000..52662394a --- /dev/null +++ b/doc/processes/dev_processes/Project-Documentation.md @@ -0,0 +1,20 @@ +# Project Documentation + +We have three wikis: + +1. [IBEX wiki](https://github.com/ISISComputingGroup/IBEX/wiki) +1. [IBEX user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki) + - [on shadow](http://shadow.nd.rl.ac.uk/ibex_user_manual/Home) +1. [IBEX developer wiki](/index) + +## IBEX wiki + +IBEX wiki this contains information at the project level for instrument scientists. Tells us about future plans, current activities, the current status of large items (e.g. version numbers) and information that instrument scientists need e.g. sprint review or instrument migrations. + +## IBEX user manual + +The IBEX user manual is for users of the system (including scientists and instrument scientists). This is details about how to use IBEX, there is some minimal information about setting up devices and there may be links to dev manual but we try not to add this. This includes a link to genie_python. + +## IBEX developer wiki + +The IBEX developer wiki, this is where all other information lives. It tells us how to develop and maintain IBEX and details of how we improve certain items. diff --git a/doc/processes/dev_processes/Things-to-know-as-a-developer.md b/doc/processes/dev_processes/Things-to-know-as-a-developer.md new file mode 100644 index 000000000..c113e6f53 --- /dev/null +++ b/doc/processes/dev_processes/Things-to-know-as-a-developer.md @@ -0,0 +1,38 @@ +# Things to know as a developer + +## Java licensing + +See [java licensing](/processes/dev_processes/Understanding-Java-Licensing) + +## Update my Instrument + +[Developer Updating](/iocs/compiling/Developer-Updating) + +## Make my Build Faster but less Safe + +See [Speed Up Building](/iocs/compiling/SpeedUpBuilding) + +## A couple of other building tricks + +### Run on one thread to clearly see the source of the error + +Navigate to the directory for the module that is erroring during a build and run: `make -j1` + +### Ignore any build errors + +Do this when you know that you can safely ignore them, for example, you need a build of a specific IOC and one you won't be using will be is stopping your build - use with extreme caution! Navigate to the top of the directories and run: `make -i` + +### No rule to make target + +This means a Makefile is trying to build something for which an external dependency has been listed and this dependency does not exist and it cannot find a way to generate it. This can be a typo/error in the Makefile itself, but if you have just done an update of many submodules it may mean that one of the auto-generated epics dependencies for the failing module refers to items in another module that now no longer exist. If for example the boost C++ library is updated, MySQL or ISISDAE may fail to build as they use headers from boost that may have been rearranged. If you "make clean uninstall" the failing module it should rebuild everything (including dependencies) and then work. You could also just remove the *.d files (which are auto-generated dependencies) and then make again. + +## Make Notepad++ Highlight DB Syntax + +In Notepad++ Menu -> Language -> Define Your Language +Then import the file `...EPICS\editors\Notepad++\userDefineLang.xml` + +Alternatively, download VSCode and install the epics extension. + +## Python conventions + +See [Python Conventions](/system_components/python/Python-conventions) for more information \ No newline at end of file diff --git a/doc/processes/dev_processes/Tickets-and-their-Workflow.md b/doc/processes/dev_processes/Tickets-and-their-Workflow.md new file mode 100644 index 000000000..6b61c3c91 --- /dev/null +++ b/doc/processes/dev_processes/Tickets-and-their-Workflow.md @@ -0,0 +1,82 @@ +# Ticket workflows + +## Ticket Types + +wf - means workflow and is how the IBEX Project Board decides on which column the ticket is in + +Type | Meaning +------ | ------- +**awaiting** | Tickets where it is hard to determine when they will be done because they are awaiting hardware, access etc. These tickets should be left on the backlog and pulled into the current sprint when ready. **Not the same as impeded**. +**bug** | A bug in the system reported by either a user or developer +**# (number)** | Estimate on how long in story points the ticket will take to develop not including review time +**rework** | Ticket has been through at least one review (with *ready* developer has changes to make, with *in progress* developer is working on it, with *review* developer has finished changes and they need re-reviewing +**under review** | Ticket is currently being reviewed +**urgent** | Ticket is urgent and should be rushed through the system +**support** | Ticket has been generated in response to a support issue for an instrument scientist or instrument problem. These are unscheduled and reactive tickets, but it is useful to keep track of time spent. +**wontfix** | Ticket will not be fixed, it is not needed or too complicated. +**duplicate** | Ticket is a duplicate of a different ticket and will not be fixed (usually the other ticket is referenced) +**for current release** | Ticket is needed for the current release and should be prioritised (allows us to keep track of whether a release can be made) +**proposal** | It is proposed that the ticket should be in the next sprint (removed each sprint) +**training** | Ticket is easy and not urgent, therefore suitable for new starters +**motion control** | Ticket may be of interest to the motion control hardware team. Tickets are relevant if they change how we interact with a motion control device or change the workflow for commissioning motion +**ready (wf)** | Ticket is in the current sprint and can be worked on +**in progress (wf)** | Ticket is currently in progress +**review (wf)** | Ticket is done and should be review by someone +**completed (wf)** | Ticket is complete +**impeded (wf)** | Ticket is in progress but cannot be completed because of something else. The reason for impediment should be added to the ticket. This should not be for long. +**fixed (wf)** | Ticket has been fixed (added at end of sprint only and by the person running the sprint) +**re-request** | Instrument scientist has requested a ticket and it has been requested by another instrument scientist in the past. A comment should record who asked with a +1. +**Needs pair review** | If a ticket is complicated or has many changes, the ticket should be reviewed by a pair of developers, not just one. Ideally one of the reviewers will already be familiar with the affected area of the code (to help with knowledge sharing). This label can be added at any time. It may be ignored for review of rework unless otherwise stated. | +**Code Review** | Ticket that could be reviewed by a wider group in a code review session. For particularly large or unique additions. +**Bucket** | Items that were proposed in the last proposal round, but were not high enough priority to make it into the sprint. If tickets in ready run out, these tickets should be picked up in preference to other tickets in the backlog.
If a "bucket" ticket is required in the next sprint, then add a `proposal` label before the bucket label is removed at the end of the current sprint. +**no_release_notes** | Items that do not need associated release notes, such as minor tasks that do not affect the code base. + +## Creation of Tickets + +Tickets should be created at need by developers as git issues using the IBEX Project Board. Initially, they should *not* have a milestone attached to them unless they are needed for a definite date and this is the LATEST that they could possibly be started. If you are doing this ensure that there is enough time for testing; bugs fixing etc. If the ticket is created for a scientist don't forget to note this in the ticket. + +When creating a ticked put on appropriate labels. When adding a 'Proposal' label, also add the ticket to the 'Planning' project in 'Proposals' column. + +## Movement of Tickets + +Developers should pick up a ticket as close to the top of the Ready column as they can (i.e. don't pick a ticket assigned to someone else). + +1. Assign the ticket to yourself and move it to in progress. +2. When the ticket is done. + - Create a pull request. If the pull request is in a public repository the PR size bot will automatically assign a label to it based on the number of lines changed. For details see https://github.com/noqcks/pull-request-size#sizing . Note that these labels are purely informational for other developers and do not form part of the sprint process (in particular, they are not related to ticket sizes in any way). + - If the config needs updating either: + - Add a change to the config updating script (usually if it affects multiple config/instruments) + - Add a step to the upgrade script (usually when it is a simple change which affects a single instrument) + - Go onto the instrument and add the change (be *very* sure it will not affect anything) + - Create a **pull request** modifying the release notes for the next release. + - Edit the file using instructions included on the page (Change types are explained near bottom of the page) + - Save your changes to a branch under same name as your ticket changes branch and add pull request + - Include link to your ticket in pull request description + - Link up your pull request in the ticket as a comment. It is helpful for the reviewer to have a comment clearly listing all of the relevant pull requests and documentation changes, including release notes, that they should review. + - Try to get an informal review to check for glaring problems. + - Either on the ticket under the Projects section use the drop-down menu to move the ticket to the review column or manually move the ticket to the bottom of the review column (unless it is of high priority, in that case, move it to the top). +3. Pick a ticket to review from the top of the review column. Review the ticket and if it is good close it (this will move it to the review complete) and add the complete label. If it needs work then add the rework label and move it back to the top of the ready column (you should inform the assignee that it is back in rework so they know how to prioritise their work). You must do a review when you move any ticket to the review column even if it is a rework. (A rework ticket review counts as a review). +4. Make sure to check the associated [upcoming release notes](https://github.com/ISISComputingGroup/IBEX/blob/master/release_notes/ReleaseNotes_Upcoming.md) PR makes sense and merge it. + +Sometimes you may want to split a larger ticket into smaller tickets to do this see [Umbrella Tickets](Umbrella-Tickets). + +## Flash reviews + +Some work is very small and doesn't warrant a full ticket and process e.g. fix a code comment, fix trivial Makefile error stopping build working. Such changes should have no potential side effects and take ~5 minutes to review. The procedure is: +* create a PR +* post PR into the flash-reviews Teams channel +* we are using emojis to indicate status and avoid creating a new line that a reply/comment would. We don't want tickets disappearing off the top of the screen. The emojis to use are: + * when somebody is looking at it, they add a "surprised" emoji + * if you are done and merged, add a "happy" + * if it is not a flash review and needs further changes, comment on github ticket and add a "sad" face + +## Adding tickets Mid Sprint + +If you are adding a ticket to the ready column mid-sprint. Please make sure it is added in priority order with estimates attached and unless there is a good reason clear it at standup. + +## Meeting where we manipulate tickets + +- [Program Increment Planning](../meetings/Program-Increment-Planning) +- [Backlog Preparation](../meetings/Backlog-Preparation) +- [Sprint Planning](../meetings/Sprint-Planning) +- [Sprint Review](../meetings/Sprint-Review-and-Retro) diff --git a/doc/processes/dev_processes/Umbrella-Tickets.md b/doc/processes/dev_processes/Umbrella-Tickets.md new file mode 100644 index 000000000..88763ce0e --- /dev/null +++ b/doc/processes/dev_processes/Umbrella-Tickets.md @@ -0,0 +1,25 @@ +# Umbrella tickets + +Umbrella tickets (or Epics in agile speak) are over arching tickets which contain multiple stages. Sometime you may feel it is beneficial for you or the reviewer to split a ticket which has been agreed in backlog prep into multiple smaller tickets; in this case you create an umbrella ticket. These can exist as actual tickets, but can also be raft tickets on the Program Increment (PI) board. + +## On the PI Board + +* These will have a classification of `Project/Feature`, and will have a whole number for the `PI Priority`. +* The smaller tickets will have a classification of `To do for a Project/Feature`, and will have a decimal value for the `PI Priority`. The number for the decimal point is the priority for the umbrella ticket, and the number after is the priority in relation to the other tickets underneath that umbrella. + +## Case 1 Before working on an original ticket it is clear it should be split + +* Create a smaller ticket it should be named `-1`: `` +* Estimate the ticket +* Assign the ticket to yourself +* Reference the ticket in the original ticket and reduce its estimate by the estimate of this ticket. If this makes the ticket negative by more than 1 then talk about it with a member of the team. +* Work on the ticket as if it were a normal ticket + +## Case 2 After working on a ticket for a while you want to spin out a new ticket + +* Create the umbrella ticket with the original estimate of the ticket minus the estimate of the present ticket. If this makes the ticket negative by more than 1 then talk about it with a member of the team. +* Name this ticket the same as the original +* Rename the original tickets as `-1`: `` +* Set the estimate on the original ticket +* Spin out any new tickets as in case 1 + diff --git a/doc/processes/dev_processes/Understanding-Java-Licensing.md b/doc/processes/dev_processes/Understanding-Java-Licensing.md new file mode 100644 index 000000000..f2980cd9b --- /dev/null +++ b/doc/processes/dev_processes/Understanding-Java-Licensing.md @@ -0,0 +1,90 @@ +# Understanding Java Licensing + +## OpenJDK + +The OpenJDK is what we use on the IBEX project. The OpenJDK is available under the GNU General Public License (GPL). The license is included in the JDK directory, for reference if required. + +## Oracle JDK + +```{danger} +We should **not** need an Oracle JDK for IBEX - we should be using OpenJDK instead. + +Please ensure you thoroughly understand Oracle's licensing model before installing or using an Oracle JDK (if you must use it). + +This information is left here for completeness only. +``` + +> In December 2016, Oracle announced that they would be taking steps to ensure customers using commercial features of Java paid the appropriate license fee (see [The Register](http://www.theregister.co.uk/2016/12/16/oracle_targets_java_users_non_compliance/) article). This announcement has caused some confusion. The purpose of this page is to clarify Java Licensing for IBEX developers. +> This page merely summarises the key points we, the IBEX development team, need to understand about Java licensing. For a complete understanding of Java Licensing terms & conditions consult Oracle's [Java SE and Java Embedded Products](http://www.oracle.com/technetwork/java/javase/terms/products/index.html) page. +> ## Java SE Product Editions +> Java SE comes in three different product editions. Each product edition provides a different set of functional capabilities, and is intended for different kinds of applications and development scenarios. The three product editions are: +> 1. Java SE +> 1. Oracle Java SE Advanced & Oracle Java SE Advanced Desktop +> 1. Oracle Java SE Suite +> All three product editions contain a mix of free-to-use & free-to-distribute features _**and**_ [Commercial Features](#java_licensing_features). If you only use free-to-use and free-to-distribute features, you do not incur a license fee. And, of course, if you use Commercial Features, you do incur a license fee. +> It is also important to understand that Oracle does _**not**_ provide installation programs that correspond directly to these 3 product editions. Depending on which of the products you have licensed, you will need to download one or more installer packages. Regardless of which product edition you intend to use, you must agree to the terms of the relevant product license. +> Taking these two points into consideration, understanding which features of the various Java SE product editions we use is essential to determining whether ISIS is liable to pay a license fee for its use of Java in IBEX. +> ### Java SE Product Edition +> The Java SE Product Edition includes the following components: +> * Java Runtime Environment (JRE), (Server and regular packages) +> * Java Development Kit (JDK), including the JavaFX Software Development Kit (SDK), +> * JavaFX Runtime, +> * JRockit JDK +> The JRE provides a Java Virtual Machine (JVM), the standard class libraries and other components to run applications written in the Java programming language. The JDK is a superset of the JRE, and contains everything in the JRE, plus tools such as compilers and debuggers for development. +> JavaFX is a software platform for creating and delivering desktop applications, as well as rich internet applications (RIAs) that can run across a wide variety of devices. JavaFX is intended to replace Swing as the standard GUI library for Java SE, but both are likely to be included for the foreseeable future. At the present time (March 2017), IBEX does not use JavaFX. +> JRockit is an alternative JVM for Oracle middleware applications. The JRockit JDK is a development toolkit for creating applications which use the JRockit JVM. At the present time (March 2017), IBEX does not use JRockit.
+> **Note:** JRockit JDK is free to use. However, applications developed using the JRockit JDK may attract a run-time license fee. Therefore, we should not use JRockit in IBEX. +> With the exception of those features designated as [Commercial Features](#java_licensing_features), Java SE can be used for free internally to run applications and may be redistributed in accordance with the [Oracle Binary Code License Agreement for the JAVA SE Platform Products](http://www.oracle.com/technetwork/java/javase/documentation/otn-bcl-02april2013-1966219.pdf) (the “Java BCLA”) - this version of the JAVA BCLA dates from 02-April-2013. +> ### Oracle Java SE Advanced & Oracle Java SE Advanced Desktop Product Edition +> Java SE Advanced and Java SE Advanced Desktop product editions are both supersets of Java SE. They include additional features for so-called "mission critical enterprise client and server deployments of Java", as well as Java-based applications and solutions from independent software vendors (ISVs). +> The Java SE Advanced and Java SE Advanced Desktop product editions both include Commercial Features. IBEX does not need to use any of the additional features (commercial or otherwise) provided by Java SE Advanced and Java SE Advanced Desktop. Therefore, we should not use them. +> ### Oracle Java SE Suite Product Edition +> Java SE Suite is a superset of Oracle Java SE Advanced. In addition to Java SE Advanced, it includes features for soft real-time applications, based on the JRockit Real Time JVM. +> The Java SE Suite product editions includes Commercial Features. IBEX does not need to use any of the additional features (commercial or otherwise) provided by Java SE Suite. Therefore, we should not use it. +> ### Installation Packages for Java SE Product Editions +> Oracle does not provide installation programs that correspond directly to various Java SE product editions described above. Depending on the products licensed, one or more of the following individual packages must be downloaded: +> * JRE (Server or regular packages) +> * JDK +> * Java Advanced Management Console +> * Microsoft Windows Installer (MSI) Enterprise JRE Installer +> * JavaFX Runtime +> * JRockit JDK +> * JRockit Mission Control +> For the purpose of developing IBEX, the only installer packages we need be concerned about are the first two, the JRE and the JDK. We do not use features provided by the other installer packages, so there is no need to download them. +> +> {#java_licensing_features} +> ## Commercial Features of Java SE Product Editions +> All of the Java SE product edition included one or more commercial features. The table below lists each of the commercial features and the product editions that contain those commercial features. +> Commercial Feature | Product Editions containing the Commercial Feature +> ------------------ | -------------------------------------------------- +> JRE Usage Tracking | JRE (version 7+, 8+) +> Java Flight Recorder | JDK (versions 7u40+, 8+) +> Java Mission Control | JDK (versions 7u40+, 8+) Java Mission Control +> Java Advanced Management Console | JDK (versions 8u20+), Java Advanced Management Console +> MSI Enterprise JRE Installer | JDK (versions 8u20+), MSI Enterprise JRE Installer +> JRockit Flight Recorder | JRockit JDK +> JRockit Mission Control Console observability | JRockit JDK, JRockit Real Time, JRockit Mission Control +> JRockit Mission Control Memory Leak Detector observability | JRockit JDK, JRockit Real Time, JRockit Mission Control +> JRockit Real Time, Deterministic GC | JRockit JDK, JRockit Real Time, JRockit Mission Control +> **Note:** Full details of [Java Commercial Features](http://www.oracle.com/technetwork/java/javase/terms/products/index.html) +> ### Using Commercial Features of Java SE +> According to the [Java command line options](http://docs.oracle.com/javase/7/docs/technotes/tools/windows/java.html) page:
+> ``-XX:+UnlockCommercialFeatures``: _Use this flag to actively unlock the use of commercial features. Commercial features are the products Oracle Java SE Advanced or Oracle Java SE Suite, as defined at the Java SE Products web page. If this flag is not specified, then the default is to run the Java Virtual Machine without the commercial features being available. After they are enabled, it is not possible to disable their use at runtime._ +> Oracle documentation makes it very clear that we cannot run commercial features of Java without specifying +> the ``-XX:+UnlockCommercialFeatures`` option on the command line. +> The key word in the above quote is "actively". We have to actively choose to use the commercial features of Java. Obviously, if we do choose to use commercial features of Java, then ISIS becomes liable to pay a license fee. Therefore, choosing to use commercial features of Java, implies that we seek prior approval to purchase the appropriate Java SE license. +> ### IBEX Policy Regarding Commercial Features of Java SE +> At the present time, IBEX does not need to use any of the commercial features of Java SE. Therefore, in developing IBEX, our policy is **TO NOT USE COMMERCIAL FEATURES** of Java SE. We should never use the ``-XX:+UnlockCommercialFeatures`` command line option. Nor should we use the MSI Enterprise JRE Installer. +> ### IBEX Build Scripts & Commercial Features of Java SE +> At the time of writing (2nd March 2017), the IBEX build scripts conform to the above policy. The scripts used to invoke Java in order to run the IBEX client are generated by the IBEX build server. The ``-XX:+UnlockCommercialFeatures`` command line option is **not** included in the configuration of the build script. See: +> 1. [build.bat](https://github.com/ISISComputingGroup/ibex_gui/blob/master/build/build.bat) and +> 1. [pom.xml](https://github.com/ISISComputingGroup/ibex_gui/blob/master/base/uk.ac.stfc.isis.ibex.client.tycho.parent/pom.xml) +> We also use ActiveMQ on the IBEX server. [ActiveMQ](http://activemq.apache.org/) is a Java application (from the [Apache Software Foundation](http://www.apache.org/)). ActiveMQ is invoked via one of two batch files: +> 1. [`activemq.bat`](https://github.com/ISISComputingGroup/EPICS-ActiveMQ/blob/master/bin/activemq.bat) +> 1. [`activemq-admin.bat`](https://github.com/ISISComputingGroup/EPICS-ActiveMQ/blob/master/bin/activemq-admin.bat) +> The ``-XX:+UnlockCommercialFeatures`` command line option is **not** a feature of either batch file. Given that the Apache SF is dedicated to the provision of open-source software, it seems highly unlikely that they will ever use +> the ``-XX:+UnlockCommercialFeatures`` command line option. +> ## Reference Material +> * [Java Licensing](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FJava%20Licensing&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}) folder on ICP Discussions site. +> * [Oracle Binary Code License Agreement for the JAVA SE Platform Products (02-April-2013)](http://www.oracle.com/technetwork/java/javase/documentation/otn-bcl-02april2013-1966219.pdf) +> * [Ticket: #1915](https://github.com/ISISComputingGroup/IBEX/issues/1915) \ No newline at end of file diff --git a/doc/processes/dev_processes/Unupdated-dependencies.md b/doc/processes/dev_processes/Unupdated-dependencies.md new file mode 100644 index 000000000..1c100c104 --- /dev/null +++ b/doc/processes/dev_processes/Unupdated-dependencies.md @@ -0,0 +1,24 @@ +# Unupdated dependencies + +This page lists some dependencies which we are not updating for various reasons: + +## Python + +| Dependency | Our version | Available version | Reason | +| --- | --- | --- | --- | +| `CaChannel` | 3.1.3 | 3.1.3 | Currently installed using `pip install github link`. `pip install CaChannel` copies dll onto itself. | +| Python3 | 3.11.6 | 3.12.0 | [`pathtools` does not build with Python 3.12](https://github.com/gorakhargosh/pathtools/issues/13) as of 11/23 check. This should be revisited in future updates. | +| Pylint | 2.13.9 | 3.0.3 | Pinned at version due to [Ticket #7034](https://github.com/ISISComputingGroup/IBEX/issues/7034), [investigation](https://github.com/ISISComputingGroup/IBEX/issues/8148) required. | + +## GUI + +| Dependency | Our version | Available version | Reason | +| --- | --- | --- | --- | +| Opal | 1.0.0 | ? | Project has been deprecated and moved into eclipse Nebula visualization widgets. Update process appears to be non-trivial. | +| `com.sun.istack.commons-runtime` | `3.x` | `4.x` | Causes `javax`/`jakarta` conflicts with `javax.activation`. | +| `activemq` | `5.x` | `6.x` | v6 uses `jakarta` packages, CSS needs `javax` packages. Need to update CSS to resolve. | + +## EPICS + +| Dependency | Our version | Available version | Reason | +| --- | --- | --- | --- | diff --git a/doc/processes/git_and_github/Adding-new-modules-via-Git.md b/doc/processes/git_and_github/Adding-new-modules-via-Git.md new file mode 100644 index 000000000..0a1202fb1 --- /dev/null +++ b/doc/processes/git_and_github/Adding-new-modules-via-Git.md @@ -0,0 +1,173 @@ +# Adding new modules + +There are two use cases for adding a submodule: + +1. When the code is new, e.g. for a support module for a new device (Marked with **New** below) +1. When importing code from a 3rd Party e.g. a fork of another github project (Marked with **3rd party** below) + > For most cases, this should be done by forking the repo, as it makes dependency updates *much* simpler to manage. However, there are instructions included on this page for how to do this without simply forking a repo, covered in 3a "3rd Party Initial Import". This should only be used in cases where forking is not possible for some extreme edge case. In most cases, forking a repo, creating an "IBEX" branch, and setting that branch as default in our ISISComputingGroup will suffice. + + +## 1 Create submodule in GitHub + +1. Create a new GitHub repository using the "new repository" button on https://github.com/ISISComputingGroup. Name it and fill in a description. The naming conventions are: + + * EPICS submodules should have an "EPICS-" prefix, e.g. for support modules. + +1. Decide on public or private repository, if you create it private one you can easily make it public later, but there is a limit on how many private repositories we can have at any one time. Repositories should be private if we do not wish to share the code. + +1. Start choice: + * **3rd Party** Create a blank repository i.e. without a README, licence or `.gitignore` file. + + * **New** Create a repository with a readme, but no `.gitignore` or licence. When it is created Use ISIS `.gitignore` files. + +1. Click `Create Repository` + +1. Add team access + + - Select Settings tab + - Manage Access (left) + - Click "Invite teams or people" and add both `ICP-Write` (give this `Write` access) and `ICP-WriteAndMerge` (give this `Maintain` access) to the repository + - If it is a private repository also add `ICP-Read` as `Read` access + +## 2 Get the repository + +1. Move to a new branch in the EPICS directory for your ticket. +1. Create a directory root for the submodule, e.g. for Danfysik in support +``` + cd EPICS/support + mkdir danfysikMps8000 +``` +1. Add a Makefile in the directory, copy it from e.g. ../calc/Makefile +1. Adjust the Makefile in the parent directory to include the new module, e.g. add to `SUPPDIRS` in `...EPICS\support\Makefile`. +1. Add all changes to git. + +## 3a *3rdParty* Initial import + +Software imported from outside should use a "vendor branch" so new versions are easy to merge in. + +1. Create new git repository called master: +``` + cd EPICS/support/danfysik8000 + mkdir master + cd master + git init +``` +1. Unpack the initial vendor code. Often unpacking the code on Linux is preferred as there are less line endings issues. +1. If it unpacks to something like 1-11/... then move everything up a level. You want the top level directory to contain the usual EPICS configure and *App directory layout. +1. Delete any files that are not source files or directories e.g. O.Common, O.linux-x86 , top level bin and lib, db that might have got left in. +1. Delete any .svn directories and files that end in a ~ (temporary files). Then add the files and push to GitHub + ``` + git add . + git commit -m "Imported danfysik 8000 version 1.11" + git remote add origin https://github.com/ISISComputingGroup/EPICS-danfysik8000.git + git push -u origin master +``` +1. Create the vendor branch +``` + git checkout -b vendor + git tag -a vendor_1_11 -m "Tag of danfysik 8000 import version 1.11" + git push origin vendor + git push --tags +``` +1. Switch to the branch for your ticket (When you create the pull request it can be created from this branch to master): +``` + git checkout TicketXXX_description +``` +1. Make local changes. + - Create a readme.md to say where we got the code originally from + - Add an initial `.gitattributes` and `.gitignore` (often using a copy from an older repo). + +1. Make sure it builds. You'll probably need to update `configure/RELEASE` to be like e.g. `../calc/master/configure/RELEASE`. Also check the Makefile for both *App and *app targets as well as iocBoot and iocboot - on windows this results in a double match due to case insensitivity, so remove the the *app and iocboot + +## 3b *New* Initial import + +1. Clone the repository in to the correct directory with the directory name master: +``` + cd EPICS/support/danfysik8000 + git clone https://github.com/ISISComputingGroup/.git master +``` +1. Switch to the branch for your ticket (When you create the pull request it can be created from this branch to master): +``` + git checkout TicketXXX_description +``` +1. Edit/Create a readme.md to say what the module does +1. Add an initial `.gitattributes` and `.gitignore` (often using a copy from an older repo). + +## 4 Adding module as submodule +This should be undertaken by the reviewer as part of the merge process to avoid issues with merge conflicts in epics top caused by IOC master. +1. Add the new repository as a submodule: + ``` + cd EPICS/support/danfysik8000 + git submodule add https://github.com/ISISComputingGroup/EPICS-danfysikMps8000.git master + ``` + note: if the default branch is not `master` e.g. `main` is now the default on github then you must specify this with `-b` e.g. + ``` + git submodule add -b main https://github.com/ISISComputingGroup/EPICS-danfysikMps8000.git master + ``` + The `master` at the end of the command is the local directory name, which we are keeping the same at the moment. + If you forget to do this, then you can edit `.gitmodules` at the top level directly and add a `branch = main` line to the submodule entry (see other entries in the file for example), then commit and push this change. + + +1. Add the files generated by adding the submodule + ```` + cd EPICS + git add .gitmodules + git commit -m "Add danfysikMps8000 submodule" + ```` +1. Next make sure that the whole thing builds from the support level (this will only work for a vendor branch). + ```` + cd ..\EPICS\support + make danfysikMps8000 + git status + ```` +1. Adjust `.gitignore` and `.gitattributes` to make sure they don't contain file that you have just built. Check `make clean uninstall` works. + +Once all the changes in the submodule are merged, create a commit in EPICS top using `git commit -m "add submodule "`, and then pushing `git push --recurse-submodule=check`. + +## Updating vendor branch + +First checkout the vendor branch and remove all local files. You need to remove all current files before you unpack the new files or else you will either not pick up files removed in the latest version, or a file that is renamed will not get tracked properly by git as the original will still exist in your local source. +``` + # If in git bash shell: + git checkout vendor + rm -fr * + ls -a + # now remove any file or directory starting with a `.` but _do not_ remove `.git` +``` +Then unpack the new code from the zip/tar into the directory in the same way as above. You'll have files added, removed and changed to handle. Type `git status` and remove unwanted added files like binaries and temporary files as described above. Then type: +``` + git add -u . +``` +This will add changed files. Again check with a `git status` that all is looking right before using: +``` + git add . +``` +which adds new and removes deleted files. As a final check run: +``` + git status --ignored +``` +This will show files in the directory currently ignored by git via a `.gitignore`. As you started from an empty directory, and unpacked a clean vendor release, consider carefully whether these files should actually be added. If they should, you will need to use `git add -f` to force an add. There are several reasons why files may get ignored when they should be added: +- Windows/Unix case sensitivity difference. A `.gitignore` may contain `db` or `db/` as a directory to ignore. The top level `db` directory is usually a top level install directory to be ignored, the real files are in the `*App/Db` subdirectory. However as the windows file system is case insensitive git treats `db` and `Db` as the same so just `db` in a `.gitignore` will incorrectly exclude `*App/Db` files and cause us to miss adding them. We usually change a `db` to something more selective in our local `.gitignore` version post merge e.g. `/db/` means only match `db` at top level and not in subdirectories. There may be other case sensitivity issues, but this is the most common one. +- a `.gitignore` pattern may be too selective for importing. For example it may exclude `*.local` to ignore files like `RELEASE.local` created later during development, but the distribution has an `EXAMPLE_RELEASE.local` that should be imported by us. + +Unless a new file is obviously included by error in the distribution, it is probably best to import everything in the vendor release distribution zip/tar file. + +Finally commit and tag the changes e.g. +``` + git commit -m "Imported danfysik 8000 version 1.12" + git tag -a vendor_1_12 -m "Tag of danfysik 8000 import version 1.12" + git push origin vendor + git push --tags +``` +Now you need to go back to your ticket branch and merge in new version of vendor code +``` + git checkout TicketXXX_add_danfysik_8000 + git pull + git merge vendor_1_12 +``` +And resolve conflicts before committing. + +### IMPORTANT: checking a vendor branch update +First, you need to be sure your tar/zip contained what you expected. If it came from an official release web page, probably fine. If it was just an auto-created github tagged version then it may be missing a lot of files. github will zip up a repository for you, if the repository contains submodules however these will not get included in the auto generated zip. In this cases if you just take the zip file from github you will end up remove lots of files you really wanted to update. If you do a `git status` and see loads of files being removed, this may have happened. The epics motor and areaDetector modules are cases where care need to be taken. + +You should confirm the number of changes in your new update branch when compared to the new vendor it is based on looks reasonable. For example if our current master when compared to its original old vendor branch had changed 5 files, then after updating vendor and merging you would expect to see something like the same number of file changes. You can do this via git branch compare - compare current master with old vendor tag and new update branch with new vendor branch tag. The number of changes will not always be identical in the new vendor branch as we do feed some of our changes back upstream if they are not locally specific hence those changes will not need re-applying and no longer show up as a changed file. If you see a large increase in files differences, this likely indicates a merge issue. diff --git a/doc/processes/git_and_github/Automation-Application.md b/doc/processes/git_and_github/Automation-Application.md new file mode 100644 index 000000000..a33e1bfac --- /dev/null +++ b/doc/processes/git_and_github/Automation-Application.md @@ -0,0 +1,87 @@ +# Automation Application + +With the move to using GitHub V2 projects a new method of automation was required. + +This is achieved using the application forked in https://github.com/ISISComputingGroup/board-automation-app +(Note this is a fork from an organisation related to one of the developers at the time, as organisation level access is required, and rather than develop and troubleshoot in this organisation, a simpler system was considered wise.) +As well as automating items in relation to project management, this also displays a burndown chart. + +## Automation +
+What this application does at a 'user' level +Note that this is based on our PI boards, which undertake some automation themselves. The term `status` below indicates things such as the column headings on the Current Sprint Board. + +If you add a `proposal` label to an issue in one of the included repos then it will be added to the PI and assigned to the next sprint with a `backlog` status +If you add an `added during sprint label` it will be added to the PI and assigned to the current sprint. No status is auto assigned here, so if the status labels are applied first then this might not appear on the board automatically. +If you add an `in progress`, `impeded`, or `review` label it will update the status to match. +If you add a `rework` label it will apply a `Backlog` status ready to be picked up again. +If you move things between columns on the board, it will update the labels applied, according to the following: + +- from backlog to in progress: add in progress label +- from backlog to impeded: add impeded label +- from backlog to review: add review label +- from backlog to done: do nothing +- from in progress to backlog: remove in progress label +- from in progress to impeded: remove in progress label, add impeded label +- from in progress to review: remove in progress label, add review label +- from in progress to done: remove in progress label +- from impeded to backlog: remove impeded label +- from impeded to in progress: remove impeded label, add in progress label +- from impeded to review: remove impeded label, add review label +- from impeded to done: remove impeded label +- from review to backlog: remove review and under review labels, add rework label +- from review to in progress: remove review and under review labels, add in progress and rework labels +- from review to impeded: remove review and under review labels, add impeded label +- from review to done: remove review and under review labels +
+
+Adding a new repository in the organisation to be watched by this application +To ensure all aspects of this application work on the repository being considered, ensure that the following labels exist in the repository: +* proposal +* added during sprint +* in progress +* impeded +* review +* under review + +Add the repository to the list in organisation settings > GitHub Apps > ibex-git-project-automation > configure +
+
+Installing the application on a host + +This will start with a pull of the application from GitHub. +The instruction for the configuration settings are held in the repo, and those instructions should be followed. +Starting the app is all that is needed after that by running `app.py` via python. + +Note that this has to use Python 3.10 or higher, and due to constraints on the system deployed to and installing on there, a specific python instance was created, which needs to be used for this. At time of writing, this is Python 3.13, with the following packages installed manually: +- `plotly` +- `numpy` +- `pandas` +- `tornado` +
+
+Updating the application +This should be as simple as synchronising the fork, and pulling to the server it is installed on. +
+
+Installing/Updating the application whilst JSON Bourne is in use +In order to simplify the addition of this application to the existing ecosystem, rather than run proxies it has been designed to run alongside an existing Tornado Application JSON Bourne. + +Basic installation and update is the same, but an additional step is needed to ensure both sets of code are run, the contents of the automation application, including the updated python, need to be copied into the JSON Bourne directory, and the contents of `webserver_forJSONBourne.py` should be replaced with the contents of `webserver_forJSONBourne_and_git_automation.py`. Alternatively `start_webserver.bat` could be updated to use `webserver_forJSONBourne_and_git_automation.py` instead of the existing `webserver.py`. +The python made available for the automation application will run JSON Bourne, and `start_webserver.bat` should be updated to use that specific python. + +
+
+Installing the application to the organisation +The instructions in the GitHub docs provides a good description of how to [register a GitHub Application](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/building-a-github-app-that-responds-to-webhook-events#register-a-github-app) +
+ +## Burndown Chart +
+What this shows +This shows a column for each day in a sprint with the number of tickets in each status/column on the current sprint board. +
+
+How it is updated +It is updated each day, the first time someone looks at the page. +
\ No newline at end of file diff --git a/doc/processes/git_and_github/Getting-started-with-Git-and-GitHub.md b/doc/processes/git_and_github/Getting-started-with-Git-and-GitHub.md new file mode 100644 index 000000000..fa8d689db --- /dev/null +++ b/doc/processes/git_and_github/Getting-started-with-Git-and-GitHub.md @@ -0,0 +1,41 @@ +# Getting started with git and github + +* Register with GitHub to create an account +* Download and install [git client for Windows](https://git-scm.com/download/win>) or [GitHub for Windows](https://windows.github.com/) or [Tortoise Git](https://tortoisegit.org/) + +(if asked, Choose `Checkout-as-is; commit Unix style line-endings`) + +During setup ensure you have selected "**Git from the command line and also 3rd-party software**" (do NOT add the optional Unix tools to your PATH). If `sh` is added to the PATH, Make will try to compile for Linux rather than Windows. This will cause syntax errors when reading the Makefiles (such as parse errors whenever a `(` is found in an echo message). + +* Set you username and email address via the command line: + +``` +git config --global user.name "YOUR NAME" +git config --global user.email "YOUR EMAIL ADDRESS" +``` + +(Or from TortoiseGit, select "Setting->Git" and select "global" and enter your details.) + +* Set a default commit editor (not needed for TortoiseGit): + +``` +git config --global core.editor "start notepad++" +``` + +* Set line-handling and passwords (Windows): +``` +git config --global core.autocrlf true +git config --global credential.helper wincred +``` + +* Set line-handling and passwords (Linux): +``` +git config --global core.autocrlf input +``` + +* Check submodules have been pushed before allowing an update of the master index +``` +git config --global push.recurseSubmodules check +``` + +* You may now need to consider ["First time installing and building"](/overview/First-Time-Build) diff --git a/doc/processes/git_and_github/Git-Cleanup.md b/doc/processes/git_and_github/Git-Cleanup.md new file mode 100644 index 000000000..d5742e505 --- /dev/null +++ b/doc/processes/git_and_github/Git-Cleanup.md @@ -0,0 +1,36 @@ +# Git cleanup + +If you wish to clean up your area of all built/generated (i.e. not under version control) files ready to do a complete rebuild, type: +``` +git clean -fdx +git submodule foreach "git clean -fdx" +``` + +The above is very similar to doing a "make clean uninstall" but much faster. **WARNING**: if you have created a new file and not yet added it to version control, it will get deleted too, so you may want to use "git status" first to check. + +If you also wish to discard all _staged_ uncommitted changes to any files in all directories, type: +``` +git reset --hard +git submodule foreach --recursive "git reset --hard" +``` + +If you want to discard all _unstaged_ changes, type: +``` +git checkout -- . +``` +to get rid of all unstaged changes, or: +``` +git checkout -- path/to/file +``` +to discard changes to a specific file. Note that this will have no effect on changes staged for commit. + +The above combination of clean and reset are equivalent to a new checkout, but much quicker. In fact this is what Jenkins does for clean builds. + +If appropriate, you can then do an update from the main git repository to get any recent changes: +``` +git pull +git submodule init +git submodule sync +git submodule update --merge +``` + diff --git a/doc/processes/git_and_github/Git-Commands.md b/doc/processes/git_and_github/Git-Commands.md new file mode 100644 index 000000000..70b02a76f --- /dev/null +++ b/doc/processes/git_and_github/Git-Commands.md @@ -0,0 +1,101 @@ +# Useful Git commands + +Git command | What it does +--------------------------------------------- | ---------------------------------------------------------- +`git checkout [branch-name]` | Checkout a branch that already exists (local or remote) +`git checkout -b [branch-name]` | Create and checkout a branch locally +`git checkout master` | Checkout the master branch +`git clone [url] [dir]` | Clone a repo from url to a directory +`git fetch origin` | Update and pull from origin +`git pull` | Get the latest copy of the repo +`git pull origin master` | Get the latest master from the origin +`git checkout -- [file-name]` | Revert file-name to the version in the repo +`git remote update` | Updates all of your local branches that are set to track remote ones, does not merge any commits. + \- | - +`git branch -a` | List all branches locally and origin +`git branch -d [branch]` | Delete a branch +`git remote prune origin` | Remove listings of deleted remote branches. Delete remote branches will still appear when you do git branch -a unless you use this command before. +`git remote prune origin --dry-run` | Shows what deleted remote branches would be removed by using git remote prune origin. Only shows what would be removed, does remove anything. + \- | - +`git add [file-name]` | Add a file to the staging area for committing +`git add \*.[type]` | Add all files of type to the staging area for committing +`git add -u` | Add all modified files to the staging area for committing + \- | - +`git mv [file-name] [folder]` | Move a file to a new folder + \- | - +`git commit -m "[descriptive-message]"` | Commit the staged area with a descriptive message +`git commit -m "Jenkins retest this please" --allow-empty` | Asks Jenkins to retest a pull request. `allow-empty` flag allows you to do it without any associated changes + \- | - +`git merge [branch-name]` | Merge the changes from [branch] to the one you have currently checked out, typically master to your dev branch +`git submodule update --merge` | Update all submodules and merge them +`git submodule update --remote` | Update the submodule references on the remote system to the current commits +`git submodule update` | Update your local submodules to those referenced on the remote system +`git rebase master` | USE ONLY ON LOCAL BRANCHES - reapply your changes to the most recently pulled master +`git submodule update --init` | 'clear out' any submodules with new commits +`git submodule status ` | status of the submodule(s) at this moment, e.g. what the current checkout commit of the submodule is +`git submodule status --cached ` | status of the submodule(s) as they appear in the index, e.g. the expected commit associated with EPICS working copy + \- | - +`git push -u origin [branch-name]` | Push a new branch to the origin +`git push origin [branch-name]` | Push a branch to the origin + \- | - +`git status` | Get the status information +`git status -s` | Get a summarised status, note that M means Modified, ?? Mean untracked + \- | - +`git clean -f` | Remove unstaged files +`git clean -f -d` | Remove unstaged files and directories + \- | - +`git ls-remote --get-url` | Get the remote URL of a repository + \- | - +`git stash` | Put all current modifications onto a stack +`git stash apply` | Apply the changes from the last stash +`git stash drop stash@{0}` | Drop the most recent stash from the stack +`git stash pop` | Apply and drop the most recent stash from the stack +`git stash branch [branch-name]` | Creates a new branch from the commit when the branch was stashed, applies the stash, and if successful drops the stash +`git stash clear` | Delete all stashes + \- | - +`git rm [file-name]` | Remove a file from the staging area + \- | - +`git log` | View the commit history +`git log -[n]` | Limit the number of entries shown by the log +`git log -p` | Show the diffs as well as the log +`git log --stat` | Abbreviated stats for each commit + \- | - +`git commit --amend` | Allows you to amend you last commit (missing files, or alter message) + \- | - +`git reset HEAD [file-name]` | Unstage a file +`git reset --hard [sha]` | Reset the branch to a specific checkout +`git reset -- soft HEAD ~` | Reverse the commit to the previous one +`git reset HEAD ~` | unstage all files + \- | - +`git log -1 HEAD` | Get the commit info for the most recent commit + \- | - +`git config -l --global | grep http` | Check for global proxies +`git config -l | grep http` | Check for local proxies +`git config --global --unset-all http.proxy` | Remove http proxies from global config +`git config --global --unset-all https.proxy` | Remove https proxies from global config +`git config --unset-all http.proxy` | Remove http proxies from local config +`git config --unset-all https.proxy` | Remove https proxies from local config + +## Remove/Move tags + +1. `git tag` list tags +1. `git tag -d ` remove the tag locally +1. `git push origin :refs/tags/` push the removed tag to remote +1. `git tag ` create the new tag pointing at the right place +1. `git push origin ` push the new tag to the repo + +## Set local branch to be the same as remote + +To set the local master branch to be the same as remote do: + +``` +git fetch origin +git reset --hard origin/master +``` + +This can be in response to the error: + +``` +remote: error: GH006: Protected branch update failed for refs/heads/master. +remote: error: At least 1 approving review is required by reviewers with write access. +``` \ No newline at end of file diff --git a/doc/processes/git_and_github/Git-workflow.md b/doc/processes/git_and_github/Git-workflow.md new file mode 100644 index 000000000..f744b5df3 --- /dev/null +++ b/doc/processes/git_and_github/Git-workflow.md @@ -0,0 +1,131 @@ +# Git workflow + +## Steps for Developing Code with existing Git Repos + +If you are not sure what a git command does, you can take a look at [git commands](Git-Commands) + +Development work +---------------------------------------------------------------------------------- +* Navigate to the appropriate directory in Git Bash +* Checkout the master branch (if not already in the master branch) `git checkout master` +* Make sure that the master branch is up to date (varies by repo) `git pull` +* Create a new branch: ``git checkout -b [branch-name]`` where `branch-name` is `TicketXXX_some_description` +* Do the development work in the most appropriate editor (e.g. Eclipse for the GUI, notepad++ for .db changes) +* Update copyright notices on edited files to the current year to read ``Copyright (C) - ...`` +* In Git Bash, add files to the branch (`git add `, or `git add -u` to add all modified files, or `git add -A` to add all changed files) and commit (`git commit -m "commit message"`) as appropriate +* Note: if master has updated, + * ensure that the master branch is up to date: ``git fetch`` + * Merge in the latest master: `git merge origin/master` + * Fix any merge errors (if required) +* Push changes using `git push` (if this is the first time the branch is being pushed, use `git push -u origin [branch-name]`) +* Go to GitHub and create the pull request, don't forget a test plan +* Bored of being asked for your username and password? Read [this](https://help.github.com/articles/caching-your-github-password-in-git/). +* If you are creating a new submodule for EPICS, find more info here: [Creating New Submodules in EPICS](/iocs/creation/Creating-New-Submodules-in-EPICS) + +## Reviewing work for the GUI + +* Navigate to the appropriate directory in Git Bash +* Update the repo: ``git remote update`` +* Checkout the branch with the changes to review: ``git checkout [branch-name]`` +* Open eclipse +* Import any new packages +* Refresh the packages +* Test the changes +* If the changes don't work, or you notice they break something else, talk to the developer and get it corrected +* Once the tests have been passed, go to GitHub and merge the pull request. There will need to be at least one "approving" review before the merge button is enabled - as the reviewer, you can add your review by clicking on "files changed" and then "review changes". +* Click on the delete branch button on the merged page + +## Reviewing work for the 'top' of EPICS (no other related changes) + +* Useful checklist for reviewing java code: https://dzone.com/articles/java-code-review-checklist +* Check the list of files to be merged on the GitHub PR page - if one of them is `.gitmodules` then the PR **should not be merged** and needs to be either amended or recreated. +* Navigate to the appropriate directory in Git Bash +* Update the repo: ``git remote update`` +* Checkout the branch with the changes to review: ``git checkout [branch-name]`` +* Test the changes, don't forget makes if required +* If the changes don't work, or you notice they break something else, talk to the developer and get it corrected +* Once the tests have been passed, go to GitHub and merge the pull request. On most repositories, there will need to be at least one "approving" review before the merge button is enabled - as the reviewer, you can add your review by clicking on "files changed" and then "review changes". +* Click on the delete branch button on the merged page + +{#git_workflow_reviewing_epics_submodules} +## Reviewing work for the submodules of EPICS + +* Navigate to the appropriate directory in Git Bash +* If the submodule is not initialised, run `git submodule update --init` in the directory above +* Update the repo: ``git remote update`` +* Checkout the branch with the changes to review: ``git checkout [branch-name]`` +* Build, make, update access to the code as necessary. If changing something in support, don't forget to make IOC entries as well +* Test the changes +* If the changes don't work, or you notice they break something else, talk to the developer and get it corrected +* Once the tests have been passed, go to GitHub and merge the pull request and delete the branch +* Get the merged pull request in repo: + * `git checkout master` + * `git pull` +* Navigate to the 'top' EPICS module in Git Bash, note that these changes are undertaken in the master thread +* Update master in 'top' + * `git checkout master` + * `git pull` +* Stage the tested submodules with: ``git add [tested merged submodules]`` (example: ``git add ioc/master/`` to add ``EPICS-ioc``) +* Commit the updated submodules, with the comment as "Update submodules ": ``git commit -m "Update submodules "`` +* Push the changed submodules back to GitHub: ``git push --recurse-submodule=check`` +NOTE: if you run `git config --global push.recurseSubmodules check` once, then the above becomes the default push action + +## Updating all Submodules + +If you want a clean start then you can use a [[Developer-Server-Build]] rather than an in place git update + +The following command used to be used in the project I leave it here for the minute. I believe that when run from the EPICS directory it updates all submodules by merging in the last commit from origin. This feels like you could have problems if your submodule is on a different branch + + git submodule update --remote --merge + +A different command, which will get the latest versions of all submodules from github (without merging in any local commits) is: + + git submodule update --init --recursive --remote + +Yet another command, which will leave some submodules like adsDriver and others which are forked and would cause a "modified content" warning if running `--remote`: + + git submodule update --recursive + +## Updating the GUI + +* Navigate to the appropriate directory in Git Bash +* Make sure you are in the master directory +* Get the latest version: ``git pull`` +* Open eclipse +* Import any new packages +* Refresh the packages + +## Updating EPICS + +* Navigate to the appropriate directory in Git Bash +* Make sure you are in the master directory +* Get the latest version: ``git pull`` +* Get the latest submodules to be used with the system: ``git submodule update --init --merge`` +* Rebuild as appropriate +* If you only want the head files for the submodules leave off the ``--merge`` from the submodule update + +## Cleaning up local installations (allowing yourself to see the wood, or branches, for the trees) + +* In Git Bash, browse to any repo that might need cleaning up (the GUI is a prime example for this) +* Update all local branches set to track remote branches (does not merge any changes) : ``git remote update`` +* Remove local listings of closed branches (can use --dry-run argument just to see what listings would be removed): ``git remote prune origin`` +* Check the list of branches: ``git branch -a`` +* Remove any out of date local branches: ``git branch -d [unused-local-branches]`` + +## Steps for Developing Code with new Git Repos + +See [Adding new modules](Adding-new-modules-via-Git). If reviewing this you will need to checkout the branch on main EPICS and do `git submodule update --init` to initialise the newly created submodule into the tree. Everything else is as normal. + +## Git Merge notes crib sheet + +The basic outline of the git merge entries are: + +``` + <<<< HEAD + The code in the local branch + ===== + The code in e.g. master + >>>> master +``` + +If there are any common ancestors, there may be extra blocks in the area to merge. diff --git a/doc/processes/git_and_github/Making-the-IBEX-workflow-work-with-SSH-authentication.md b/doc/processes/git_and_github/Making-the-IBEX-workflow-work-with-SSH-authentication.md new file mode 100644 index 000000000..451892775 --- /dev/null +++ b/doc/processes/git_and_github/Making-the-IBEX-workflow-work-with-SSH-authentication.md @@ -0,0 +1,39 @@ +# Git and SSH key authentication + +By default our workflows expect token based authentication and https connections to GitHub. If you are stuck in your ways and wish to use SSH authentication, the following will work. It assumes you have already [generated an SSH key, placed it in .ssh, and added it to your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh). + +## Git configuration +You could make your developer set up work by simply replacing all of the remotes for your git repositories with addresses in the ssh connection format. However, our processes often overwrite these and there are a large number of sub-modules which would also need changing. Luckily git has a configuration option that will globally rewrite these addresses for you. Simply add the following to your global git configuration: + +``` +[url "ssh://git@github.com/"] + insteadOf = https://github.com/ + pushInsteadOf = https://github.com/` +``` + +_(Remember if you use multiple instances of git for some reason, ensure that you add it to all the ones you wish to use for IBEX development!)_ + +If you grow tired of entering your passphrase every time you run a git command - follow the configuration instructions below. + +## Scripts that contain git commands +The above will work fine until you encounter a script which runs a git command (possibly only those using the python git library), this will not prompt you for your ssh key passphrase and so will fail with an authentication error. To overcome this, you need to already have an ssh agent running and have added your key. To do this you can do the following + +### Initial configuration +You need to configure your environment to launch an ssh agent and add your key. It will do this every time you open a new window - prompting you to enter your passphrase. To do this add the following to ~/.bash_profile (or if you don't have one, create one with this in): + +``` +eval `ssh-agent` +ssh-add +``` + +### Every time you wish to run a script + +#### To run a script in Git Bash +Having followed the configuration steps above, simply open git bash (enter your passphrase) and run your script + +#### To run a script in Windows Command Prompt +Having followed the configuration steps above, open git bash (enter your passphrase) and run `start cmd`. Run your script in the window that opens, it will use the ssh agent that started when you started Git Bash. + +#### To run a script in an EPICS terminal +Having followed the configuration steps above, open git bash (enter your passphrase) and run `start /c/Instrument/Apps/EPICS/config_env +`. Run your script in the window that opens, it will use the ssh agent that started when you started Git Bash. \ No newline at end of file diff --git a/doc/processes/git_and_github/New-Local-Git-Repository.md b/doc/processes/git_and_github/New-Local-Git-Repository.md new file mode 100644 index 000000000..258d8586c --- /dev/null +++ b/doc/processes/git_and_github/New-Local-Git-Repository.md @@ -0,0 +1,16 @@ +# New local git repository + +These are created in /isis/git on control-svcs.isis.cclrc.ac.uk see [here](https://control-svcs.isis.cclrc.ac.uk/git/) + +When creating a new repository make sure you add + + [http] + receivepack = true + +to the config file of the repository or pushes will not work + +Cloning is via something like + + git clone http://control-svcs.isis.cclrc.ac.uk/gitroot/instconfigs/common.git + + diff --git a/doc/processes/git_and_github/git-hub-repo-tools.md b/doc/processes/git_and_github/git-hub-repo-tools.md new file mode 100644 index 000000000..2f7808c17 --- /dev/null +++ b/doc/processes/git_and_github/git-hub-repo-tools.md @@ -0,0 +1,13 @@ +# Github Repository Tools + +These are stored in [ibex_utils](https://github.com/ISISComputingGroup/ibex_utils) and there are install instructions and help. This is a list of common tasks run under epics terminal: + +* List repositories belonging to ISISComputingGroup + + `python Scripts\change_repos.py -u John-Holt-Tessella -o ISISComputingGroup` + +* For the standard repositories close all old milestones (with no issues), create new milestone, ensure labels exist (adding --dry-run to this, or any command, will show you what it will do without altering the repository) + + `python Scripts\change_repos.py -u John-Holt-Tessella -o ISISComputingGroup --repo-file standard_repos.txt --label-file standard_labels.txt --ms-from 2016-05-19 --ms-to 2016-06-15 --ms-close` + + diff --git a/doc/processes/instrument_details/ALF-Instrument-Details.md b/doc/processes/instrument_details/ALF-Instrument-Details.md new file mode 100644 index 000000000..a327bea2e --- /dev/null +++ b/doc/processes/instrument_details/ALF-Instrument-Details.md @@ -0,0 +1,43 @@ +# ALF + +This page collects information that will be useful for the implementation of the new control system on ALF. + +### Background & Timeline +The ​ALF web page describes the background to the instrument. + +### GUI +ALF will use the IBEX control system. + +### ALF Equipment +The equipment listed below is used on ALF. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | [#218](https://github.com/ISISComputingGroup/IBEX/issues/218) |[see DAE note](#alf_noteDAE) +[Galil](http://www.galilmc.com/) | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | EPICS |[see Galil note](#alf_noteGalil) + | | JAWS | | | [#179](https://github.com/ISISComputingGroup/IBEX/issues/179), [#180](https://github.com/ISISComputingGroup/IBEX/issues/180) |[see Jaws note](#alf_noteJaws) +unknown | 6-axis | Goniometer | | |[see Goniometer note](#alf_noteGoniometer) + +{#alf_noteDAE} +##### Note: DAE ##### +Location: Screened Room
+DAE-2 + +{#alf_noteGalil} +##### Note: Galil ##### +Model: [DMC2280](http://www.galilmc.com/products/dmc-22x0.php).
+Photograph of [ALF Galil Crate](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ALF/ALF_Galil_Crate.jpg).
+Note that GALIL_02 had trouble communicating asynchronously so Freddie switched it to synchronous (done by making the third argument of `GalilCreateController("Galil", "$(GALILADDR02)", 20)` negative in `galil2.cmd`). + +{#alf_noteJaws} +##### Note: Jaws ##### +Location: various
+Driven by [Galils](#alf_noteGalil).
+Photograph of [ALF Slits/Jaws](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ALF/ALF_Slits.jpg). + +{#alf_noteGoniometer} +##### Note: Goniometer ##### +Model: 6-axis Goniometer, unknown model & manufacturer.
+Driven by [Galils](#alf_noteGalil).
+See [ALF Goniometer Axes](/specific_iocs/motor_extensions/ALF-Goniometer-Axes) for details of how the Goniometer is set up and configured.
+Photographs of [ALF Goniometer (1)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ALF/ALF_Goniometer_1.jpg) and [ALF Goniometer (2)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ALF/ALF_Goniometer_2.jpg). diff --git a/doc/processes/instrument_details/CHIPIR-Instrument-Details.md b/doc/processes/instrument_details/CHIPIR-Instrument-Details.md new file mode 100644 index 000000000..c4bc62759 --- /dev/null +++ b/doc/processes/instrument_details/CHIPIR-Instrument-Details.md @@ -0,0 +1,107 @@ +# CHIPIR + +This page collects information that will be useful for the implementation of the new control system on CHIPR. +## Background & Timeline ## +CHIPIR is a neutron instrument dedicated to the irradiation of microelectronics with atmospheric-like neutrons, on TS2. The [CHIPIR](https://www.isis.stfc.ac.uk/Pages/Chipir.aspx) web page describes the background to the instrument. + +It was originally intended that IBEX should be piloted on CHIPIR, but commissioning of the instrument has taken longer than expected and, therefore, this idea has been shelved. Nevertheless, there is no reason why IBEX cannot be installed on CHIPIR at some future date. + +## Control System ## +CHIPIR currently uses the SECI control system. It will migrate to the IBEX control system at a convenient date (yet to be decided). + +## CHIPIR Equipment ## +The equipment listed below is used on CHIPIR. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#chipir_note-galil) +[HAMEG](https://www.rohde-schwarz.com/general_information/hameg/rohde-schwarz-company_230166.html) | HM8123 | Gated Counter | | #109, #110, #117 | [see HAMEG note](#chipir_note-hameg) +[AGILENT](http://www.home.agilent.com/agilent/home.jspx?cc=GB&lc=eng) | E3613A | PSU | | | [see Agilent note](#chipir_note-agilent) +[AGILENT](http://www.home.agilent.com/agilent/home.jspx?cc=GB&lc=eng) | 33220A | Function Generator | | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Agilent), #102, #111, #118 | [see Agilent note](#chipir_note-agilent) +[AGILENT](http://www.home.agilent.com/agilent/home.jspx?cc=GB&lc=eng) | 53220A | Counter | | #103, #112, #119 | [see Agilent note](#chipir_note-agilent) +[TEKTRONIX](http://www.tek.com/digital-multimeter) | DMM4040 | Multimeter | | #120 | [see Tektronix note](#chipir_note-tektronix) +[TEKTRONIX](http://www.tek.com/digital-multimeter) | DMM4050 | Multimeter | | #121 | [see Tektronix note](#chipir_note-tektronix) +[Stanford RS](http://www.thinksrs.com/) | PS350 | PSU | | #122 | [see Stanford RS note](#chipir_note-stanford-rs) +[Stanford RS](http://www.thinksrs.com/) | SR400 | Photon Counter | | #123 | [see Stanford RS note](#chipir_note-stanford-rs) +ISIS | | Filter Set | [see Galil note](#chipir_note-galil) | | [see Filter Set note](#chipir_note-filter-set) +ISIS | | Collimator | [see Galil note](#chipir_note-galil) | | [see Collimator note](#chipir_note-collimator) +ISIS | | XYZ Table | [see Galil note](#chipir_note-galil) | | [see XYZ Table note](#chipir_note-xyz-table) +ISIS | | Pre-sample Table | [see Galil note](#chipir_note-galil) | | [see Pre-sample Table note](#chipir_note-pre-sample-table) +[ORTEC](http://www.ortec-online.com) | [Easy-NIM 928](http://www.ortec-online.com/Solutions/928-EASY-NIM.aspx) | MCA | | | [see ORTEC note](#chipir_note-ortec) +[SuperLogics](https://www.superlogics.com) | 8017 series | Data Acquisition | | | [see SuperLogics note](#chipir_note-super-logics) +TBD | | Camera | | | [see Camera note](#chipir_note-camera) +Beckhoff | | | | | [see Beckhoff note](#chipir_note-beckhoff) + +{#chipir_note-galil} +##### Note: Galil ##### +Model: [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) + +Other than the collimator, the only other quirk to CHIPIR's galil 02 is that the jaws in/out axis uses a non-standard homing routine - [`galil_Home_JogForwLimit.dmc`](https://github.com/ISISComputingGroup/EPICS-galil/pull/79/files#diff-64af6fffc6cf1ad849a8685d6848ae66746109ac8457de498d3fa758491681e1) + +{#chipir_note-filter-set} +##### Note: Filter Set ##### +Driven by Beckhoff controlled motor.
+See [see Galil note](#chipir_note-galil) + +These are driven through `TC` using some variables for in/out readback and control. There is also some web HMI screen which IDD motion wrote which may now be defunct now that IBEX works with the filter set. + +{#chipir_note-collimator} +##### Note: Collimator ##### +Driven by Galil controlled motor.
+ +This runs some [custom galil code](https://github.com/ISISComputingGroup/EPICS-galil/blob/master/GalilSup/Db/galil_CHIPIR_Collimator.dmc) but essentially does not accept moves like a normal motor. it has an encoder as an auxiliary feedback only ie. to see if it's moving, but uses physical switches fed into the Galil's digital ins for positioning. It uses a "read" thread that runs on thread 7 to monitor these during a move. + +{#chipir_note-xyz-table} +##### Note: XYZ Table ##### +Driven by Galil controlled motor.
+See [see Galil note](#chipir_note-galil) + +{#chipir_note-pre-sample-table} +##### Note: Pre-sample Table ##### +Driven by Galil controlled motor.
+See [see Galil note](#chipir_note-galil) + +{#chipir_note-hameg} +##### Note: HAMEG ##### +[HAMEG](https://www.rohde-schwarz.com/general_information/hameg/rohde-schwarz-company_230166.html) has now been acquired by [Rohde & Schwarz](https://www.rohde-schwarz.com).
+[HM8123 Universal Counter](https://www.rohde-schwarz.com/product/hm8123-productstartpage_63493-44102.html) + +{#chipir_note-agilent} +##### Note: Agilent ##### +Agilent has spun-off its electronic measurement business, which now trades as [Keysight Technologies](http://www.keysight.com/main/home.jspx?cc=GB&lc=eng).
+ +1. [E3613A power supply](http://www.keysight.com/en/pd-838240-pn-E3610A/30w-power-supply-8v-3a-or-15v-2a?cc=GB&lc=eng&lsrch=true&searchT=E3613A) is now discontinued. +1. [33220A function generator](http://www.keysight.com/en/pd-127539-pn-33220A/function-arbitrary-waveform-generator-20-mhz?cc=GB&lc=eng&lsrch=true&searchT=33220A) will be discontinued from 01/12/2016. +1. [53220A counter](http://www.keysight.com/en/pd-1893411-pn-53220A/350-mhz-universal-frequency-counter-timer-12-digits-s-100-ps?nid=-33609.959903.00&cc=GB&lc=eng). + +{#chipir_note-tektronix} +##### Note: Tektronix ##### +[DMM4050 and DMM4040 digital multimeters](https://uk.tek.com/datasheet/dmm4050-4040-digital-multimeter) + +{#chipir_note-stanford-rs} +##### Note: Stanford RS ##### + +1. [PS300 series high-voltage power supplies](http://www.thinksrs.com/products/PS300.htm) +2. [SR400 series photon counters](http://www.thinksrs.com/products/SR400.htm) + +{#chipir_note-ortec} +##### Note: ORTEC Easy-NIM 928 ##### +1. [High Performance, Multi-Function Nuclear MCA/Counter/Timer/Rate Meter](https://www.ortec-online.com/products/electronics/multichannel-analyzers-mca/basic-analog/928). + +{#chipir_note-super-logics} +##### Note: SuperLogics ##### +1. [8017/18/19 series - 8 Channel Analog Input Module, 16-Bit A/D Converter](https://www.superlogics.com/data-acquisition-99/data-acq-analog-input/analog-input-rs485/8017.html).
+Copy of manual for [8017/18/19 series](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Chipir/Manuals%20for%20Equipment/Superlogics_8019R.pdf). + +{#chipir_note-camera} +##### Note: Camera ##### +Not required for day 1. Open to suggestions. + +{#chipir_note-beckhoff} +##### Note: Beckhoff #### +1. Needed prior to migration, should be revisited during migration to move to using more standard motion control under IBEX +1. There are three items, a secondary shutter and 2 filters being controlled by a Beckhoff +1. Initially these will be controlled via SECI, with the Beckhoff managing the reporting and moving of the items in/out of the beam, these will be reported back to IBEX using the standard Beckhoff tagging method +1. A VI will be needed which interacts with the PVs generated by TCIOC + + diff --git a/doc/processes/instrument_details/CHRONUS-Instrument-Details.md b/doc/processes/instrument_details/CHRONUS-Instrument-Details.md new file mode 100644 index 000000000..13f742c9b --- /dev/null +++ b/doc/processes/instrument_details/CHRONUS-Instrument-Details.md @@ -0,0 +1,45 @@ +# CHRONUS + +This page collects information that will be useful for the implementation of the IBEX control system on CHRONUS. + +## Background & Timeline + +CHRONUS is a muon spectroscopy instrument at ISIS, on TS1, as part of the RIKEN instruments. The [CHRONUS](https://www.isis.stfc.ac.uk/Pages/CHRONUS.aspx) web page describes the background to the instrument. + +## Control Systems + +CHRONUS is in the process of migrating to the IBEX control system alongside Argus as of May 2024, due to both be completed by the end of the year. + +## CHRONUS Equipment + +The Equipment listed below is used on CHRONUS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes +------------ | ------------ | ------------ | ------------ | ------------ | ------------------------------------------ +ISIS | DAE 2 | Detector Electronics | Ethernet | | | +CAEN | V895 | Discriminator | ??? | support: CAENVME ioc: CAENV895| [See CAEN note](#chronus_caen-v895-note)| +CAEN | SY4527 | HV power supply | Ethernet | HVCAENx527 | | +Kepco | BOP | Bi-Polar Power Supply | RS-232 | KEPCO | [See Kepco note](#chronus_kepco-note)| +National Instruments | DAQmx | Magnetometer | Ethernet | ZFMAGFLD_01 | [See magnetometer note](#chronus_magnetometer-note) + +## Notes + +{#chronus_caen-v895-note} +### CAEN V895 Note + +The CHRONUS CAEN v895 IOC is connected to 3 crates. Two of the crates have 15 cards, numbered 0-14, plugged in, and the third has 8, numbered 0-7. There are therefore a total of 38 cards, and 608 channels as there are 16 channels per card. + +A script exists in the muon instrument scripts repo to set all channels to a user's choice of a defined value, or an array of values specified in a file. + +{#chronus_kepco-note} +### Kepco Note + +There are 3 Kepcos being used on CHRONUS, all of which for the zero-field system. The axis are aligned as follows: + - X: Left - Right + - Y: Up - Down + - Z: Forward - Back + +{#chronus_magnetometer-note} +### Magnetometer Note + +There are two magnetometers used on Chronus, one fixed and used in the zero field controller, the other a removable probe. \ No newline at end of file diff --git a/doc/processes/instrument_details/CRISP-Instrument-Details.md b/doc/processes/instrument_details/CRISP-Instrument-Details.md new file mode 100644 index 000000000..383d5a54b --- /dev/null +++ b/doc/processes/instrument_details/CRISP-Instrument-Details.md @@ -0,0 +1,157 @@ +# CRISP + +This page collects information that will be useful for the implementation of the IBEX control system on CRISP. +## Background & Timeline ## +CRISP is a reflectometer instrument at ISIS, on TS1. The [CRISP](https://www.isis.stfc.ac.uk/Pages/CRISP.aspx) web page describes the background to the instrument. + +## Control System ## +CRISP will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## CRISP Equipment ## +The equipment listed below is used on CRISP. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#crisp_noteDAE) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#crisp_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +??? | | 4-blade jaws | | | [see Jaws note](#crisp_noteJaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#crisp_noteVacuum) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#crisp_noteVacuum) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#crisp_noteEurotherm) +Oxford Instruments | Orange Cryostat| Cryogenic System | | | [see Oxford Instruments note](#crisp_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | | | [see Oxford Instruments note](#crisp_noteOxfordInstruments) + | | Closed Cycle Refrigerator | | | | [see CCR note](#crisp_noteCCR) +Julabo | FP-50 | Water Bath | RS-232 | | [see Water Bath note](#crisp_noteWaterBath) +Julabo | FP-52 | Water Bath | RS-232 | | [see Water Bath note](#crisp_noteWaterBath) +~Grant Instruments~ | ??? | ~Water Bath~ | RS-232 | | [see Water Bath note](#crisp_noteWaterBath) +~Haake~ | ~N6~ | ~Water Bath~ | RS-232 | | [see Water Bath note](#crisp_noteWaterBath) +Keithley | 2000 | Multimeter | RS-232 | | [see Keithley note](#crisp_noteKeithley) +Keithley | 2602 | Source Meter | RS-232 | | [see Keithley note](#crisp_noteKeithley) +Hirst | GM05 | Gaussmeter | ??? | | [see Hirst note](#crisp_noteHirst) +Knauer | K-6 | Electric Valve Drive | ??? | | [see Knauer note](#crisp_noteKnauer) +Knauer | 1050 | HPLC pump | ??? | | [see Knauer HPLC note](#crisp_noteKnauerHPLC) +Hitachi | L-7100 | HPLC pump| ??? | | [see Hitachi note](#crisp_noteHitachi) +Nima Trough | ??? | Trough | ??? | | [see Nima Trough note](#crisp_noteNimaTrough) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#crisp_noteThurlby) +Watson Marlow | 323 | Syringe Pump | ??? | | [see Syringe Pumps note](#crisp_noteSyringePumps) +WPI | Aladdin-1000 | Syringe Pump | ??? | | [see Syringe Pumps note](#crisp_noteSyringePumps) +WPI | SP2xx | Syringe Pump | ??? | | [see Syringe Pumps note](#crisp_noteSyringePumps) +??? | ??? | He3 Monitor | ??? | | [see He3 Monitor note](#crisp_noteHe3Monitor) + +{#crisp_noteDAE} +##### Note: DAE ##### +See multi-detector and single-detector below. + +{#crisp_noteChopper} +##### Note: Choppers ##### +CRISP has a Mk3 chopper.
+ +{#crisp_noteJaws} +##### Note: Jaws ##### +Provide information about CRISP jaws. + +{#crisp_noteVacuum} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#crisp_noteEurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/CRISP/CRISP_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#crisp_noteOxfordInstruments} +##### Note: Oxford Instruments ##### +1. Orange Cryostat: [models](https://www.isis.stfc.ac.uk/Pages/Low-Temperature-Cryostats.aspx) to be determined. Orange cryostats are controlled via a temperature controller (e.g. Eurotherm), so nothing on the cryostat itself for IBEX to control. +1. Blue Cryostat: [models](https://www.isis.stfc.ac.uk/Pages/Low-Temperature-Cryostats.aspx) to be determined. + +{#crisp_noteCCR} +##### Note: Closed Cycle Refrigerators ##### + 1. [CCR](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/CRISP/CRISP_CCR.jpg) is the most frequently used low-temperature device used in on CRISP. +[CCRs](https://www.isis.stfc.ac.uk/Pages/Closed-Cycle-refrigerators,-CCRs.aspx). Not directly computer-controlled - controlled via Eurotherm. + +{#crisp_noteWaterBath} +##### Note: Water Baths ##### +CRISP uses two types of water bath +1. Julabo (which model(s)?) +1. Grant Water Bath + * Update (25-06-2019): Grant water baths are no longer used. No longer any need to support them. See [#4457](https://github.com/ISISComputingGroup/IBEX/issues/4457) +1. Haake N6 Water Bath. Haake has now been taken over by [ThermoFisher](https://www.thermofisher.com/uk/en/home/life-science/lab-equipment/water-baths-circulators-chillers.html). N6 model may be obsolete. Check existing SECI VI for logic and manual. + * Update (25-06-2019): Haake water baths are no longer used. No longer any need to support them. See [#4456](https://github.com/ISISComputingGroup/IBEX/issues/4456) + +{#crisp_noteKeithley} +##### Note: Keithley ##### +1. [Keithley 2000 Series Multi-Meter](https://uk.tek.com/tektronix-and-keithley-digital-multimeter/keithley-2000-series-6%C2%BD-digit-multimeter-scanning). See also [#2460](https://github.com/ISISComputingGroup/IBEX/issues/2460). +1. [Keithley 2600 Series Source Meter](https://uk.tek.com/keithley-source-measure-units/smu-2600b-series-sourcemeter).
+ +{#crisp_noteHirst} +##### Note: Hirst ##### +[Hirst GM05 gaussmeter](http://www.hirst-magnetics.com/instruments/gm05_p1.shtml) + +{#crisp_noteHitachi} +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump. Can't find L-7100 on Hitachi web-site (may be obsolete). Check existing SECI VI for logic and manual. + +{#crisp_noteKnauer} +##### Note: Knauer ##### +Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. + +{#crisp_noteKnauerHPLC} +##### Note: Knauer HPLC ##### +Knauer HPLC 1050 [Knauer HPLC 1050 is discontinued.](https://www.knauer.net/en/discontinued-smartline-pump-1050-successor-azura-p-61l/p14161). Check existing SECI VI for logic and manual. + +{#crisp_noteNimaTrough} +##### Note: Nima Trough ##### +Nima Trough: SECI used a manufacturer supplied VI. We may need to do the same in IBEX (via lvDCOM).
+ * **Note:** NIMA Technologies Ltd now seems to be part of [Biolin Scientific](https://www.biolinscientific.com/ksvnima).
+ * The NIMA trough is used regularly on SURF. The manufacturer supplied VI is used to view graphs showing information about thin films. + * The manufacturer has made additional software available for download. A copy of this software is located in `\\isis\shares\ISIS_Experiment_Controls\NIMA Trough\Nima_TR8.1.zip`. + +{#crisp_noteThurlby} +##### Note: Thurlby ##### +Thurlby EX355P PSU - see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). + +[Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + +{#crisp_noteSyringePumps} +##### Note: Syringe Pumps ##### +1. [Watson Marlow 323 Peristaltic Pump](http://www.watson-marlow.com/gb-en/range/watson-marlow/300-tube-pumps/323d/) +1. [WPI Aladdin-1000 Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps/al1000-220.aspx) +1. [WPI SP2xx Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps.aspx) - check specific model. + +{#crisp_noteHe3Monitor} +##### Note: He3 Monitor ##### +He3 Monitor is an obsolete piece of equipment. It is no longer used. No need to support it. + +## CRISP SECI Configs ## +Document information about CRISP SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +CRISP_place_holder1.conf | - | dd/mm/yyyy | - | +CRISP_place_holder2.conf | - | dd/mm/yyyy | - | + +## CRISP Genie Scripts ## +Similarly, Document information about CRISP SECI Genie scripts here. + +## CRISP Notes ## +CRISP has the following specialist panels: +1. CRISP setup dialogue +1. CRISP User Front Panel +1. CRISP analyser stage +1. INTER High Voltage +1. SURF Galil analogue output +1. SURF Galil DIO + +CRISP has the following devices under motion control: +1. goniometer +1. jaws & height +1. jaws +1. multi-detector +1. single-detector +1. supermirror +1. transmission monitor +1. XYZ stage + +CRISP has the following item: +1. LSS Scan calculator (is this a script?) \ No newline at end of file diff --git a/doc/processes/instrument_details/DETMON-Instrument-Details.md b/doc/processes/instrument_details/DETMON-Instrument-Details.md new file mode 100644 index 000000000..85a84092d --- /dev/null +++ b/doc/processes/instrument_details/DETMON-Instrument-Details.md @@ -0,0 +1,96 @@ +# DETMON + +## Background & Timeline ## +DETMON is a support system for use by the detector teams to monitor and view the status of the HV (High Voltage) crates that are in use across ISIS. +A pilot system should be created before Cycle 2019-01, the final system does not at present have a dead line. + +## Equipment ## +The only equipment to consider for this system are CAEN crates (types of HV crate), however, there are about 30+ of them to consider. + +## System Requirements ## +### Monitoring ### +The following items need to be monitored for each part of the system: + +#### Crates #### + +Parameter | Variable | What to record | Type +--- | --- | --- | --- +ON/OFF available in network | `ConnStatus` | Any variation | String +Crate fan status | `HVFanStat` | Any variation | String +Crate fan speed | `HVFanSpeed` | Any variation >+-5% | String +Which slots are used in the crate | | Any variation | +Model Name | `ModelName` | Any variation | String +Software Release | `SwRelease` | Any variation | String +Power Supply Status | `HvPwSM` | Any Variation | String +IP address | `IPAddr` | Any Variation | String +Boards slots | `Slots` | Any Variation | Integer +CPU Load | `CPULoad` | Any Variation>=+-??? | Integer +Commands in the queue | `CmdQueueStatus` | Any Variation>=+-??? | Integer +Memory Status | `MemoryStatus` | Any Variation >=+-??? | Integer + +#### Boards #### + +Parameter | Variable | What to record | Type +--- | --- | --- | --- +Model | Model | Any Variation | String +Firmware release | `Fmw Release` | Any Variation | String +Serial Number | `SerNum` | Any Variation | Integer +Number of Channels | `NrOfCh` | Any Variation | Integer +Temperature | `Temp` | Any variation >= +-1 deg C | Float +Max Voltage | `HVMax` | Any Variation | Float +Board Status | `BdStatus` | Any Variation | Integer + +#### Channels #### + +Parameter | Variable | What to record | Type +--- | --- | --- | --- +Channel Name | Name | Any Variation | String +Voltage Setting | `V0Set` | Any Variation | Float +Current Setting | `I0Set` | Any Variation | Float +Voltage1 Setting | `V1Set` | Any Variation | Float +Current1 Setting | `I1Set` | Any Variation | Float +Rump Up speed | `RUp` | Any Variation | Float +Rump Down speed | `RDWn` | Any Variation | Float +Trip time | Trip | Any Variation | Float +Max Voltage Software limit | `SVMax` | Any Variation | Float +Monitor Voltage | `VMon` | Any Variation >= +-1 V | Float +Monitor Current | `IMon` | Any Variation >= +-2 uA | Float +Status | Status | Any Variation | Integer +Power Down Mode | `PDwn` | Any Variation | Integer + +### Notes for the solution ### +* This monitoring would be for every crate in use at ISIS, and for any offline/test crates as well. Note that a crate has many boards, and a board has many channels – in the test system available at time of writing there is 1 crate, 8 cards, and 23 channels per card +* The IOC already deals with the channel requirements, although we may need to verify the trip +* The boards and crates may need some work to add them +* This monitoring also has to be independent of the instrument status or behaviour +* The rate of capture for this information can be relatively slow (1 second per item) +* It is quite possible to link multiple IOCs to the same crate +* Recording data to nexus files is acceptable – as such by creating a large number of blocks we should be able to run a script that is based on time and uses dummy neutron data – the nexus files can be stored, and the data regarding behaviour used to provide long term information. The log plotter will provide short term information for the numeric values. A python script could start a run and end and restart at predetermined times/after certain durations +* Whilst IBEX and NeXus files provide a solution, the data is being logged into a database, and longer term, either logging to a different database or forwarding the data that way may be more suitable – this should be explored mid-long term, but IBEX at least provided short term +* The number of blocks would need to be created by scripting, and the majority should likely be hidden from use on IBEX. +* The sheer number of blocks did raise a concern as to whether or not IBEX could deal with the numbers, as such some testing and proving will be needed + * to get an idea of the potential numbers of blocks, assume every instrument has one fully populated CAEN crate + * if every channel is defined as a block (so that it can be recorded in a Nexus file), that's + * `35 x 8 x 23` = `6440` blocks + * clearly `6440` blocks is an over-estimate, but it illustrates the scale of the problem +* Currently to tick over log files at midnight a cron job is used to end and begin a run + +## Configuration set up + +DETMON has a lot of blocks to set up and the CAEN crates create have a dynamic amount of boards and channels set up. This set up is scripted with scripts from the NDADETMON config area (in `Python/inst`). There are two scripts: + +1. The script `record_channels.py` searches for the connected boards and channels for the crates (see the script help option for arguments to pass). The output is a JSON file containing information on the crate name, boards and channels. +1. Using this JSON file you can then create the relevant `blocks.xml`, `groups.xml`, `block_config.xml` and `gwblock.pvlist` files using the `create_config_files_from_recorded_channels.py`. Once produced put them in the config area alongside the meta.xml and other configuration-specific files. In the meta.xml ensure that the `configuresBlockGWAndArchiver` tag is set to true and the blockserver will handle the use of the files you have created. + +This two step process lets you edit the intermediate JSON to include channels and crates which are not currently on the network. + +## Notes + +- DETMON runs the block gateway and block archiver with customised `gwblock.pvlist` and `block_config.xml` files to tackle handling the high number of blocks that we require on IBEX. See ticket https://github.com/ISISComputingGroup/IBEX/issues/5069. + + +## Known issues + +- The number of blocks is so high that they often timeout when getting them from the blockserver. We do not at the moment care too much as long as they are written correctly to the nexus file and Davide seems quite happy with that. + - Tackled in https://github.com/ISISComputingGroup/IBEX/issues/5069 by using customised `gwblock.pvlist` and `block_config.xml` files + - Possible future performance upgrade in https://github.com/ISISComputingGroup/IBEX/issues/5917 \ No newline at end of file diff --git a/doc/processes/instrument_details/EMMA-Instrument-Details.md b/doc/processes/instrument_details/EMMA-Instrument-Details.md new file mode 100644 index 000000000..b06b63696 --- /dev/null +++ b/doc/processes/instrument_details/EMMA-Instrument-Details.md @@ -0,0 +1,130 @@ +# EMMA + +This page collects information that will be useful for the implementation of the IBEX control system on EMMA. +## Background & Timeline ## +EMMA is an instrument at ISIS, on TS1 and used primarily for test purposes. The primary users of EMMA are the sample environment team and the detector team. There appears to be no web pages (or other material) describing the background to the instrument. + +Instrument scientist contact is **Jeffrey Sykora**. + +## Control System ## +EMMA will migrate from the SECI control system to the IBEX control system in summer 2017. + +## EMMA Equipment ## +The equipment listed below is used on EMMA. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | Use in ISIS Cycle | +------------ | ------------- | ------------- | ------------- | ------------- | -------------------------------------------|-------------- +ISIS | DAE 2 & 3 | Detector Electronics | Ethernet | | [see DAE note](#emma_noteDAE)|17_1 +ISIS | | Monitors | | | [see DAE note](#emma_noteDAE)|17_1 +ISIS | Mk2 Chopper | Chopper | RS-232 | See [#2130](https://github.com/ISISComputingGroup/IBEX/issues/2130) | [see Chopper note](#emma_noteChopper)|?? +SKF | | Fermi Chopper | RS-232 | | [see FermiChopper note](#emma_noteFermiChopper)|?? +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | |17_1 +ADC | | 1 x 4-blade jaws | | | Blades driven by Galils. [see Jaws note](#emma_noteJaws) +ISIS | TPG300 | ISIS Vacuum System | | |[see Vacuum System note](#emma_noteVacuum) +ISIS | | YZ Table | [see Galil note](#emma_noteYZTable) | | [see YZ Table note](#emma_noteYZTable) +ISIS | | Rotation Stage | [see Galil note](#emma_noterotationstage) | | [see RotationStage note](#emma_noterotationstage) +EURO-THERM | | Temperature Controller | +[CAEN](http://www.caen.it/csite/HomePage.jsp) | HT | Detector HV Control | Ethernet | [#212](https://github.com/ISISComputingGroup/IBEX/issues/212) |[see CAEN HT note](#emma_noteCAENHT) + +The following items are on EMMA's wish-list. They are items for the future (i.e. support is not required until such times as the individual items can be procured). + +Manufacturer | Model | Type | Connection | Driver | Notes | Use in ISIS cycle +------------ | ------------- | ------------- | ------------- | ------------- | -------------------------------------------|---------- +[CAEN](http://www.caen.it/csite/HomePage.jsp) | | Digitiser | Ethernet | |[see CAEN Digitiser note](#emma_noteCAENDigitiser) +Acqiris | | Digitiser | RS-232 | | [see Acqiris Digitiser note](#emma_noteAcqirisDigitiser) +NI | | Digitiser | RS-232 | | [see NI Digitiser note](#emma_noteNIDigitiser) +[Tektronix](http://www.tek.com/oscilloscope#all) | | Oscilloscope | Ethernet | |[see Tektronix note](#emma_noteTektronix) + +{#emma_noteDAE} +##### Note: DAE ##### +Main Detector banks + fixed monitors. +EMMA has 4 monitors: + +1. Monitor 1: stationary; located after the T0 chopper +1. Monitor 2: stationary; located after the Fermi chopper +1. Monitor 3: stationary; located between the jaws and the Y-Z table +1. Monitor 4: currently stationary, but would like it to be moveable in future; located after the rotation stage + +{#emma_noteChopper} +##### Note: Choppers ##### +One T0 chopper.
+Chopper is ISIS Mk2 chopper. Mk2 choppers have a serial interface (not Ethernet like Mk3). +See [#2130](https://github.com/ISISComputingGroup/IBEX/issues/2130) for IOC & OPI for Mk2 chopper.
+The front panel of the Mk2 chopper control crate (located in the EMMA rack) looks like this: +* [wide angle view](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_T0_Chopper_Front_Panel_(wide)_2017_06_05.jpg) +* [zoomed-in view](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_T0_Chopper_Front_Panel_(zoom)_2017_06_05.jpg) + +{#emma_noteFermiChopper} +##### Note: Fermi Chopper ##### +EMMA has a single Fermi chopper (manufactured by SKF).
+Link to [Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/Mirrortron-SKF%20892-0053%20MB350PC_R%20Rev%20C.pdf)
+The Fermi chopper control crates (located in the EMMA rack) look like this: +* [Front View (upper & lower crates)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_SKF_FermiChopper_Controller_Front_Upper_Lower_2017_06_05.jpg) +* [Rear View (upper crate)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_SKF_FermiChopper_Controller_Rear_Upper_2017_06_05.jpg) +* [Rear View (lower crate)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_SKF_FermiChopper_Controller_Rear_Lower_2017_06_05.jpg) + +{#emma_noteFermiChopperLift} +##### Note: Fermi Chopper Lift ##### +The Fermi chopper is lifted into position by a Galil controlled motor.
+The Fermi chopper must not be operated when it is not in the beam (i.e. in the parked/upper position) +There is a hardware interlock to prevent the Fermi chopper from being operated in the parked position. This interlock is implemented in the Galil controller, via a Galil program (described in the [EMMA chopper lifter - Handover](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA%20chopper%20lifter%20-%20Handover.docx) document). In SECI a simple VI is used to provide a GUI, which passes parameters to the Galil program, so that the user can control the Fermi Chopper lift. IBEX should provide a similar capability via an IOC and OPI. + +{#emma_noteJaws} +##### Note: Jaws ##### +EMMA has one four-blade set of jaws. All blades driven by motors connected to Galil controllers. +Jaw set is manufactured by Advanced Design Consultancy. + +{#emma_noteVacuum} +##### Note: Vacuum System ##### +The vacuum system ([TPG300](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_TPG300_2017_06_05.jpg)) on EMMA is currently (06/03/2017) not operational. Outside possibility that it may be operational by cycle 2017/02. + +{#emma_noteYZTable} +##### Note: Y-Z Table ##### +Driven by Galil controlled motors. +In SECI, Y-Z Table has its own GUI. No particular reason for this; it could be merged with the Rotation Stage UI.
+[Photograph of Y-Z table](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_Y_Z_Table_2017_06_05.jpg). + +{#emma_noteRotationStage} +##### Note: Rotation Stage ##### +Driven by Galil controlled motors. +In SECI, Rotation Stage has its own GUI. No particular reason for this; it could be merged with the Y-Z Table UI.
+[Photograph of Rotation Stage](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_Rotation_Stage_2017_06_05.jpg). It sits on top of the Y-Z table when in use. + +{#emma_noteCAENHT} +##### Note: CAEN HT ##### +CAEN high-voltage system is a [SYS2527](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMMA/EMMA_CAEN_HV_PSU_2017_06_05.jpg) (see tickets: #212, #241, #320, #419, #424) + +{#emma_noteCAENDigitiser} +##### Note: CAEN Digitiser ##### +For the future - it's on the wish list (model not currently known). + +{#emma_noteAcqirisDigitiser} +##### Note: Acqiris Digitiser ##### +For the future - it's on the wish list. +Desired model is Acqiris digitiser - PXI8570 controller chassis with DC 440 (400MS/s 100 MHz) digitiser card.
+Note that Acqiris is now part of Keysight (see [Keysight digitizers](http://www.keysight.com/en/pc-1128783/High-Speed-Digitizers-and-Multichannel-Data-Acquisition-Solution?cc=GB&lc=eng)). + +{#emma_noteNIDigitiser} +##### Note: NI Digitiser ##### +For the future - it's on the wish list. +Desired model is National Instruments digitiser - PXIe-1082 chassis, PXIe-8135 controller and a PXIe-5162 digitizer (5GS/s and 1.5GHz). +See [NI Digitizers/Oscilloscopes](http://sine.ni.com/np/app/main/p/bot/no/ap/mi/lang/en/pg/1/sn/n17:mi,n21:40,n24:PXI-FSLASH-CompactPCI/). + +{#emma_noteTektronix} +##### Note: Tektronix ##### +For the future - it's on the wish list. Desired models are: +1. [Tektronix Oscilloscope DPO7000](http://www.tek.com/oscilloscope/dpo7000-digital-phosphor-oscilloscope) +1. [Tektronix Oscilloscope MSO4054](http://www.tek.com/oscilloscope/mdo4000c-mixed-domain-oscilloscope) + +## EMMA SECI Configs ## +Document information about EMMA SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +----------------------------------------------------------|----------------------------------------------------------------|---------------|----------| + +## EMMA Genie Scripts ## +Similarly, Document information about EMMA SECI Genie scripts here. + +## EMMA Notes ## +Add any notes about special items of equipment, setup or conditions on EMMA that might impact the deployment and configuration of IBEX. +1. Screenshots of [EMMA VIs](https://github.com/ISISComputingGroup/ControlsWork/issues/239) diff --git a/doc/processes/instrument_details/EMU-Instrument-Details.md b/doc/processes/instrument_details/EMU-Instrument-Details.md new file mode 100644 index 000000000..9318740bd --- /dev/null +++ b/doc/processes/instrument_details/EMU-Instrument-Details.md @@ -0,0 +1,160 @@ +# EMU + +This page collects information that will be useful for the implementation of the IBEX control system on EMU. +## Background & Timeline ## +EMU is a new 96-detector muon spin rotation spectrometer which is optimised for zero field and longitudinal field measurements at ISIS, on TS1. The [EMU](https://www.isis.stfc.ac.uk/Pages/EMU.aspx) web page describes the background to the instrument. + +## Control System ## +EMU will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## EMU Equipment ## +The equipment listed below is used on EMU. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Motion note](#note-motion) +??? | | 4-blade jaws | | | [see Jaws note](#note-jaws) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +Julabo | ??? | Water Bath | RS-232 | | [see Water Bath note](#note-water-baths) +ICE | | Oxford ICE Cube | ??? | | [see Oxford ICE Cube note](#note-oxford-ice-cube) +[CAEN](http://www.caen.it/csite/HomePage.jsp) | V895 | Discriminator | USB VXI via NI DLL | |[see CAEN note](#note-caen) +Danfysik | 8000 | PSU | RS232 | DFKPS | [see Danfysik note](#note-danfysik) +TDK | Lambda Genesys | PSU | RS232 | TDK_ LAMBDA_ GENESYS | [see Genesys note](#note-genesys) | +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 100-10MG | Bi-Polar Power Supply | RS232 | #187 |[see Kepco note](#note-kepco) +Kepco | BIT 4886 | I/F card | GPIB, RS-232 | | [see Kepco note](#note-kepco) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Mercury | Temperature Controller | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | ITC503 | Temperature Controller | | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Mercury | Pressure Controller | | | [see Oxford Instruments note](#note-oxford-instruments) +Lakeshore | 372| Temperature Controller | | | [see Lakeshore note](#note-lakeshore ) +Chell | CCD100 | Pressure Transducer | RS-232 | | [see Chell note](#note-chell) +[Stanford RS](http://www.thinksrs.com/) | DG645 | Delay Generator | | | [see Stanford RS note](#note-stanford-rs) +Aeroflex/IFR | 2030 | Signal Generator | RS-232 | | [see Aeroflex note](#note-aeroflex) + +##### Note: DAE ##### +See multi-detector and single-detector below. + +##### Note: Motion ##### +There is a project relating to the motion on the South Side Muons in progress, this information may be out of date. + +##### Note: Choppers ##### +Muon instruments do not have choppers. + +##### Note: Jaws ##### +Provide information about EMU jaws. + +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/EMU/EMU_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +##### Note: Water Baths ##### +EMU uses Julabo water baths +##### Note: Water Baths ##### +1. Julabo Water Bath + * no information on specific Julabo Water Baths - maybe EMU uses water baths from the pool? + * consult the [Julabo commands](/specific_iocs/temp_controllers/julabo_commands.xlsx) spreadsheet for specific details of which set of commands are used individual water baths. + * EMU may use oil or glycol as the coolant/heating medium (Julabo/Presto A40 device). +1. See also [Water Baths](https://www.isis.stfc.ac.uk/Pages/Water-Baths.aspx) list + +##### Note: Neocera ##### +Low temperature experiments?[Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +##### Note: Oxford ICE Cube ##### +1. The ICE Fridge is a dilution fridge. + 1. **N.B.** It is **not** an Oxford Instruments device. It is from a different manufacturer. + 1. It cannot be compared to other models of dilution fridge. +1. The ICE control PC is referred to as an ICECube. + 1. This PC runs Windows XP. It should not be connected to the ISIS network. + 1. Communication with the equipment is via serial and a MOXA unit. +1. The dilution fridge temperature at low temperatures is controlled via a Lakeshore 370. +1. There is only one ICE dilution fridge which is used on MUSR mainly. It is available for EMU. +1. See ticket [#3876](https://github.com/ISISComputingGroup/IBEX/issues/3876) + +Documentation can be found at : `C:\LabVIEW Modules\Drivers\ICEOxford\ICECube\Documentation` +LabVIEW client can be found at : `C:\LabVIEW Modules\Drivers\ICEOxford\ICECube\ICECube - Client.llb\ICECube - Client.vi` + +##### Note: CAEN ##### +1. The [CAEN V895](http://www.caen.it/csite/CaenProd.jsp?parent=11&idmod=49) is a 16-channel Leading Edge Discriminator. +1. Documentation (\\isis\shares\ISIS_Experiment_Controls\Manuals\CAEN\V895 16-Channel Discriminator + +##### Note: Danfysik ##### +Need to check which model of Danfysik is used on EMU. +1. [Danfysik Power Supplies](http://www.danfysik.com/en/products/power-supplies/): model 8000 appears to have been superseded by model 8500. +1. [User and Software Manuals](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FPower%20Supplies&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence) for System 8500. +1. See [#1208](https://github.com/ISISComputingGroup/IBEX/issues/1208) for comms settings. + +##### Note: Genesys ##### +1. [TDK Lambda Genesys Power Supplies](https://uk.tdk-lambda.com/products/programmable-power-supplies/). +1. [Safety, User and Programming Manuals](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FPower%20Supplies&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence) for System 8500. +1. See also [#983](https://github.com/ISISComputingGroup/IBEX/issues/983), [#2276](https://github.com/ISISComputingGroup/IBEX/issues/2276), [#2458](https://github.com/ISISComputingGroup/IBEX/issues/2458) + +##### Note: Kepco ##### +1. [Kepco BIT 4886](http://www.kepcopower.com/bit.htm) is an interface card for Kepco bi-polar ([BOP](http://www.kepcopower.com/bop.htm)) power supplies. See also `C:\LabVIEW Modules\Drivers\Kepco 4886 Serial` +1. [Kepco BOP 100-10MG PSU](http://www.kepcopower.com/bophimod.htm).
See also [#3005](https://github.com/ISISComputingGroup/IBEX/issues/3005) + +##### Note: Oxford Instruments ##### +1. Dilution fridge: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx) to be determined. + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) +1. Support for OI ITC503 created as part of [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) +1. Intelligent Pressure Controller - what do we know about this? + +##### Note: LakeShore ##### +1. On muon instruments, the Triton fridges have an additional Lakeshore controller (the 372) to allow control and continuous readout of the sample temperature (the OI software doesn't allow this using the in-built Lakeshore, despite the unit having a dedicated channel for the sample thermometry). +1. [Model 372](https://www.lakeshore.com/products/categories/overview/temperature-products/ac-resistance-bridges/model-372-ac-resistance-bridge-temperature-controller) temperature controller. +1. There are [EPICS drivers](https://epics-controls.org/resources-and-support/modules/hardware-support/) for a variety of Lakeshore temperature controllers. + +##### Note: Chell ##### +1. Experiments requiring pressure transducer. [Chell CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100) +1. Documentation (\\\isis\shares\ISIS_Experiment_Controls\CCD100) + +##### Note: Stanford RS ##### +1. [Stanford DG645 Digital Delay Generator](http://www.thinksrs.com/products/dg645.html) +1. See also `C:\LabVIEW Modules\Drivers\Stanford DG645` +1. Documentation (\\\isis\shares\ISIS_Experiment_Controls\Stanford_DG645_Digital_Delay_Generator) + +##### Note: Aeroflex ##### +Aeroflex has been through a complicated series of sales in recent years. The Aeroflex manuals (\\\isis\shares\ISIS_Experiment_Controls\AeroflexIFR 2023A) are probably a good place to start, along with any existing VIs. See also [IFR 2023A](https://www.atecorp.com/products/ifr/2023a). + +There appears to be no driver for the Aeroflex 2023A in `C:\LabVIEW Modules\`. However, there is a driver for an Aeroflex 2030 in `C:\LabVIEW Modules\Drivers\Aeroflex 2030`. + +## EMU SECI Configs ## +Document information about EMU SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +-----------------------------------|----------------------------------------------------|---------------|----------| +EMU_place_holder1.conf | - | dd/mm/yyyy | - | +EMU_place_holder2.conf | - | dd/mm/yyyy | - | + +## EMU Genie Scripts ## +Similarly, Document information about EMU SECI Genie scripts here. +### Booster heater issues + +Scientists initially raised concerns about intermittent invalid alarms and it appears this is due to communication failures to a mercury (see [#6286](https://github.com/ISISComputingGroup/IBEX/issues/6286)). + +These intermittent caused a failure to switch on the booster heater. The reason for this was that the booster heater script refers to the `Temp_Sample` block for determining whether it should switch the heater on, and if the target temperature is higher than `Temp_Sample`, it will start using the booster heater by running `cset` on the same block. If the block is temporarily in ‘invalid’, then it certainly fails to read and set the temperature (and just carries on). [#6286](https://github.com/ISISComputingGroup/IBEX/issues/6286) was then closed with a temporary increased stream device lock timeout - from 5 to 30 seconds. + +The problems from [#6286](https://github.com/ISISComputingGroup/IBEX/issues/6286) then reoccurred on the 22nd and 23rd of May 2021, which spawned tickets [#6270](https://github.com/ISISComputingGroup/IBEX/issues/6720) (issues concerning communicating with the mercury) and [#6271](https://github.com/ISISComputingGroup/IBEX/issues/6721) (issues concerning the booster heater). In [#6270](https://github.com/ISISComputingGroup/IBEX/issues/6720), there was some debate over whether there were differences in the mercury's setup on the day (some addresses of the daughter boards were changed) which could have caused the recurrence of the issue. + +Investigation showed the device failed to reply as fast in busy periods and didn't reply as quickly as usual, and that a reply timeout trips up the IOC until it gets time to send the commands again and get a correct response. It was suggested that increasing the reply timeout slightly would fix this, but also that it was pertinent to investigate why on the 23rd and 22nd of May this was more prominent. Adding `@replyTimeout` handlers didn't seem to help very much either. + +Sep 2021: Added a [retry mechanism](https://github.com/ISISComputingGroup/EPICS-StreamDevice/compare/master...Ticket6720_add_max_retries_for_mercury_issues) for protocols. +Lacking documentation about whether this deployment to EMU was successful, and how above changes have affected the status of [#6271](https://github.com/ISISComputingGroup/IBEX/issues/6721). + +Marking [#6271](https://github.com/ISISComputingGroup/IBEX/issues/6721) as fixed and will be closed, please open a new issue if this reoccurs. + +## EMU Notes ## +EMU has the following specialist panels: +1. MUSR RF Kit +1. EMU Traverse field supply +1. EMU Steering Magnet +1. HIFI Metrolab PT2025 (NMR Readout) +1. [MUON Zero Field Controller](/specific_iocs/magnets/Zero-field-controller) +1. MUON Magnets Danfysik +1. MUON FrontEnd Control +1. MUON FrontEnd Magnet Monitoring +1. MUON Jaws + +EMU also has the following system: +1. MUON Script (generator) diff --git a/doc/processes/instrument_details/ENGIN-X-Instrument-Details.md b/doc/processes/instrument_details/ENGIN-X-Instrument-Details.md new file mode 100644 index 000000000..a50b9159f --- /dev/null +++ b/doc/processes/instrument_details/ENGIN-X-Instrument-Details.md @@ -0,0 +1,173 @@ +# ENGIN-X + +This page collects information that will be useful for the implementation of the IBEX control system on ENGIN-X. +## Background & Timeline ## +ENGIN-X is a long established instrument at ISIS, on TS1. The [ENGIN-X](http://www.isis.stfc.ac.uk/instruments/engin-x/engin-x2900.html) web page describes the background to the instrument. ENGIN-X is dedicated to engineering science experiments. It is optimized for the measurement of strain, and thus stress, deep within crystalline materials. + +## Control System ## +ENGIN-X will migrate from the SECI control system to the IBEX control system. + +## ENGIN-X Equipment ## +The equipment listed below is used on ENGIN-X. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | N/A | Shutter | N/A| | [see Shutter note](#enginx_noteShutter) +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#enginx_noteDAE) +ISIS | Monitors | | | [#265](https://github.com/ISISComputingGroup/IBEX/issues/265) | [see Monitors note](#enginx_noteMonitors) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | [#169](https://github.com/ISISComputingGroup/IBEX/issues/169) | [see Mk3 Chopper note](#enginx_noteMk3Chopper) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#enginx_noteGalil) | +[LINMOT](http://www.linmot.com/) | [P0x-23](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) | Linear Motors and Motion Controller | RS-232 | [#2098](https://github.com/ISISComputingGroup/IBEX/issues/2098) | [see LinMot note](#enginx_noteLinMot) | +??? | ??? | 3 x 4-blade jaws | | | [see Jaws note](#enginx_noteJaws) +??? | ??? | 2 x collimators | | | [see Collimators note](#enginx_noteCollimators) +xxxx | 5-axis Sample Positioning System | | | [#435](https://github.com/ISISComputingGroup/IBEX/issues/435) | [see Sample Positioning System note](#enginx_noteSampleStack) +[Cybaman](http://www.cybamantech.co.uk/) | 3-axis Sample Positioning System | | | | [see Cybaman Positioning System note](#enginx_noteCybaman) +xxxx | xxxx | ISIS Vacuum System | N/A | | Analog only. No computer control. +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#enginx_noteEurotherm) +[Instron](http://www.instron.co.uk/) | Stress Rig | xxxx | GPIB | [#2109](https://github.com/ISISComputingGroup/IBEX/issues/2109) - [#2116](https://github.com/ISISComputingGroup/IBEX/issues/2116) | [see Stress Rig note](#enginx_noteStressRig) +SuperLogics | POE-8019(?) | Thermocouple Box | Ethernet | [#2378](https://github.com/ISISComputingGroup/IBEX/issues/2378) | [see Thermocouple Box note](#enginx_noteThermocoupleBox) +NI | GPIB-ENET/100| Data Acquisition Box | Ethernet | TBD | [see NI High-Speed DAQ Box note](#enginx_noteNI_DQA_Box) + +Computer control of the equipment listed below is is considered a "nice-to-have" for ENGIN-X. + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +Leica | TM5100A | Theodolite | TBD | TBD | [see Theodolite note](#enginx_noteTheodolite) +xxxx | xxxx | Function Generator | TBD | TBD | +xxxx | xxxx | Oscilloscope | TBD | TBD | +[TDK Lambda](https://www.emea.lambda.tdk.com/uk/products/genesys-1u) | Genesys | Power Supply | TBD | [#983](https://github.com/ISISComputingGroup/IBEX/issues/983)| [see TDK Lambda Genesys note](#enginx_noteTDKGenesys) +Berkeley | MCP Detector | Neutron Camera | TBD | TBD | [see Berkeley MCP Detector note](#enginx_noteBerkeleyMCP) +Basler | Pilot | Camera | GigE | TBD | [see Basler Pilot Camera note](#enginx_noteBaslerCamera) +[Huber](https://www.xhuber.com/en/products/1-components/12-rotation/eulerian-cradles/)| [511](https://www.xhuber.com/en/products/1-components/12-rotation/eulerian-cradles/511/) | Eulerian Cradle| TBD| TBD | [see Huber note](#enginx_noteHuber) + +{#enginx_noteShutter} +##### Note: Shutter ##### +ENGIN-X and HRPD share the same primary shutter (which is directly equivalent to the single shutter used on other instruments). ENGIN-X and HRPD also have their own secondary shutters (so that each instrument can operate independently of the other when the primary shutter is open. +1. When the primary shutter is closed, neither instrument receives neutrons. The status of the primary shutter is available in the same way as it is for any other instrument (i.e. via the appropriate PV). +1. When the primary shutter is open, ENGIN-X receives neutrons only if the secondary shutter is also open. The status of the secondary shutter is not available to IBEX. + +The IBEX dashboard should continue to report the status of the primary shutter.
+**Note:** IBEX only ever reports the status of the shutter. IBEX _**never**_ controls the shutter (control of the shutter is part of the instrument safety system and is strictly outside the scope of IBEX). + +{#enginx_noteDAE} +##### Note: DAE ##### +Main Detector banks (transmission bank and diffraction banks) + 2 monitors. + +{#enginx_noteMonitors} +##### Note: Monitors ##### +2 fixed monitors. Monitors operate in histogram mode. + +{#enginx_noteMk3Chopper} +##### Note: ISIS Mk3 Choppers ##### +ENGIN-X has two ISIS Mk3 double-disk choppers. + +{#enginx_noteLinMot} +##### Note: LinMot ##### +HRPD uses LinMot P0x-23 motors, controlled by LinMot drives.
+[LinMot User Manual](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) + +{#enginx_noteGalil} +##### Note: Galil ##### +As of 04/08/2017, ENGIN-X has a single Galil (support for up to 8 axes). 5 of these axes are used to control the Sample Positioning System. 2 more are used to control the collimators.
+When the collimators are further upgraded to 4 motors, an additional Galil rack will be required. All 4 collimator motors should be moved to the new rack (so that all 4 can be exclusively controlled using EPICS). + +{#enginx_noteJaws} +##### Note: Jaws ##### +As of April 2017, ENGIN-X has three sets of 4-blade jaws. All three sets of jaws are driven by LinMot P0x-23 motors. The first two sets of jaws (jaws-1 and jaws-2, counting downstream from the shutter) are buried under shielding. The third set of jaws (jaws-3) is located near the beam snout (i.e. close to the sample, not under shielding). + +jaws-1 and jaws-2 are "traditional" four-blade jaws (i.e. each blade can be operated independently - 4 degrees of freedom). + +jaws-3 also has 4 blades, but they operate as 2 linked pairs. In jaws-3: +1. the north-south pair operate symmetrically and simultaneously. +1. the east-west pair operate symmetrically and simultaneously. + +In effect, jaws-3 has only 2 degrees of freedom (instead of 4, if the blades could operate independently). In practice, you can only control the horizontal & vertical gaps; you cannot control the position of the centre of the blades. + +The initial jaws settings are documented here: [ENGIN-X Jaws Settings](/specific_iocs/motor_extensions/jaws/EnginX-Jaws) + +There is a plan to upgrade the third set of jaws (jaws-3) to a 4-independent-blade set, which will be driven by Galil controllers. The timing of this upgrade is yet to be determined. + +{#enginx_noteCollimators} +##### Note: Collimators ##### +Two collimator units (to be installed in summer 2017). Each collimator is driven by a single Galil controlled axis. + +{#enginx_noteSampleStack} +##### Note: Sample Positioning System ##### +Sample Positioning System on ENGIN-X has 5 degrees of freedom. All axis motors are controlled by Galil. +1. X - translation parallel to the beam (in the horizontal plane) +1. Y - translation perpendicular to the beam (in the horizontal plane) +1. Z - translation perpendicular to the beam (in the vertical plane) +1. W (omega) - rotation about the vertical axis +1. S (slit rails) - translation of slits (i.e. jaws-3) parallel to the beam (in the horizontal plane) + +ENGIN-X uses a jog box for positioning samples. IBEX will need to work with the existing jog-box interlock mechanism (for safety reasons). There is an existing Galil program (i.e. a program that gets loaded into the Galil firmware) for communicating with the jog box. IBEX needs to be set up to load this program alongside the Galil IOC (speak to Gareth to discover how to do this). + +**Note:** ENGIN-X would like an IMAT-style tomography stack. However, at the current time (April 2017), there are insufficient Galil axes on ENGIN-X to permit such a device to be used. Need to wait until additional Galil axes can be made available. + +{#enginx_noteCybaman} +##### Note: Cybaman Positioning System ##### +3 axis sample positioning system (manufactured by [Cybaman](http://www.cybamantech.co.uk/)) - 3 degrees of freedom. Currently has its own, independent control system (which runs on a separate PC).
+There is also a simple LabVIEW VI which can be used to drive the Cybaman.
+To drive the Cybaman directly from EPICS, we'd probably need to create our own driver - this might be a lot of work for very little gain. + +{#enginx_noteEurotherm} +##### Note: Eurotherm ##### +There are 3 Eurotherm devices on ENGIN-X, arranged in a single crate. Typically used to control temperature of CryoBox and Furnace devices. + +{#enginx_noteStressRig} +##### Note: Stress Rig ##### +Used to put stress on samples under test on ENGIN-X. This device is shared between IMAT and ENGIN-X. +The ICP discussion site contains [Stress Rig documentation](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FInstron%20Stress%20Rig&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}).
+**Note:** Stress Rig uses GPIB communications protocol.
+The Stress Rig is driven from its own PC (fitted with a GPIB card). IBEX communicates with the Stress Rig PC via the GPIB card. See [#2338](https://github.com/ISISComputingGroup/IBEX/issues/2338) for ideas on how to test the Stress Rig IOC. + +{#enginx_noteHuber} +##### Note: Huber ##### +2-degree of freedom Eulerian cradle. The Huber can be driven from a Galil. Gareth has created a VI to control this via SECI. + +{#enginx_noteTheodolite} +##### Note: Theodolite ##### +ENGIN-X has a Leica Theodolite TM5100A, which is used to survey the instrument. There is currently a VI to control the Theodolite from LabVIEW, but none of the current ENGIN-X team has ever needed to use the Theodolite.
+The [Leica web-site](https://leica-geosystems.com/en-gb) no longer lists a TM5100A model of theodolite. It does, however, list a [TM6100A](https://www.hexagonmi.com/en-GB/products/industrial-theodolites-and-laser-stations/leica-tm6100a-industrial-theodolite) model. + +{#enginx_noteThermocoupleBox} +##### Note: Thermocouple Box ##### +ENGIN-X has a Thermocouple Box, which allows for the connection of up to 8 thermocouples. This device is often used with the furnace. Hardware is from [SuperLogics](http://www.superlogics.com). The device sounds like it is one of these: [POE-8019](https://web.archive.org/web/20171015083151/http://www.superlogics.com/data-acquisition/ethernet/POE-8019/86-5660.htm#). Need to consult with Matt North about continued support for this device.
+The command set for the Thermocouple Box is, apparently, quite simple. It should not be too difficult to reproduce the capabilities of the existing LabVIEW VI in EPICS. + +{#enginx_noteTDKGenesys} +##### Note: TDK Lambda Genesys Power Supply ##### +ENGIN-X has previously used a TDK Lambda Genesys power supply, although it was not connected to SECI. Ask ENGIN-X team if they will need IBEX to communicate with a TDK Lambda Genesys power supply.
+**Note:** a generic power supply IOC & OPI would be a useful thing. + +{#enginx_noteNI_DQA_Box} +##### Note: NI High-Speed DAQ Box ##### +ENGIN-X has a National Instruments (NI) [GPIB-ENET/100](http://sine.ni.com/nips/cds/view/p/lang/en/nid/10622) Ethernet-to-GPIB controller for high-speed data acquisition. It has been used to make independent voltage measurements from various points on the stress rig. There are two options for collecting these voltage measurements in future: +1. continue to use the GPIB-ENET/100 box (integration with IBEX would need prior investigation) +2. use a "data-into-DAE" card to acquire the voltage data and embed it in the NeXus data file (will need prior consultation with the detector electronics team). + +**Note:** according to the NI web-site, the GPIB-ENET/100 box is now considered obsolete. + +Need to discover how data acquired by the NI High-Speed DAQ Box gets written to file. We could use a signal generator to drive the DAQ box for testing purposes. + +{#enginx_noteBerkeleyMCP} +##### Note: Berkeley MCP Detector ##### +ENGIN-X would like to use the Berkeley MCP detector (neutron camera) used on IMAT. This device is controlled by its own PC, but it uses EPICS to communicate with IBEX. The ENGIN-X team would like to set up the Berkeley MCP detector in the same way as IMAT.
+**Note:** the ENGIN-X team would also like to use a hardware box to delay the signal between the Berkeley MCP detector and its control PC. This hardware box uses a USB interface. + +{#enginx_noteBaslerCamera} +##### Note: Basler Pilot Camera ##### +ENGIN-X are consider whether it would be useful to use a high-performance GigE camera to acquire images. A [Basler Pilot](https://www.baslerweb.com/en/products/cameras/area-scan-cameras/pilot/) area scan camera is being considered.
+**Note:** GigE area scan cameras can generate a lot of data (hence the need for a gigabit Ethernet interface). If there is a decision to use a GigE area scan camera, we will need to consider acquiring additional Ethernet cards (to avoid swamping the current network) or a separate PC to capture images. + +## ENGIN-X Notes ## +* For each run, ENGIN-X generates .log and .txt files from the NeXus data file. The .txt files are "3-column" files (as opposed to the "2-column" data files used on other instruments. We need to determine if these "3-column" files are still required in the future and, if so, document how these "3-column" files are generated. +* OpenGENIE is currently used to analyse ENGIN-X data. In due course, the ENGIN-X team should consider migrating to Mantid. + +## ENGIN-X SECI Configs ## +Document information about ENGIN-X SECI configs here. + +## ENGIN-X Genie Scripts ## +Similarly, Document information about ENGIN-X SECI Genie scripts here. +* ENGIN-X currently has one large master OpenGENIE script, which is used in (almost?) all runs. This script has to be migrated to genie_python (or to a library of genie_python methods). +* ENGIN-X also uses a position compensation script (to compensate for the movement of the target position as the sample is stretched on the stress rig). diff --git a/doc/processes/instrument_details/GEM-Instrument-Details.md b/doc/processes/instrument_details/GEM-Instrument-Details.md new file mode 100644 index 000000000..0913bab3a --- /dev/null +++ b/doc/processes/instrument_details/GEM-Instrument-Details.md @@ -0,0 +1,144 @@ +# GEM + +This page collects information that will be useful for the implementation of the IBEX control system on GEM. +## Background & Timeline ## +GEM is a diffractometer instrument at ISIS, on TS1. The [GEM](http://www.isis.stfc.ac.uk/instruments/gem/gem2467.html) web page describes the background to the instrument. + +## Control System ## +GEM will migrate from the SECI control system to the IBEX control system in summer 2017. During the summer 2017 shutdown, new jaws will be installed on GEM. + +## GEM Equipment ## +The equipment listed below is used on GEM. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#gem_noteDAE) +ISIS | Mk2 Chopper | Chopper | RS-232 | see [#2130](https://github.com/ISISComputingGroup/IBEX/issues/2130) | [see Chopper note](#gem_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +??? | | 5 x 4-blade jaws | | | [see Jaws note](#gem_noteJaws) +??? | | 1 x 4-blade jaws | | | "Beamscraper" Jaws [see Jaws note](#gem_noteJaws) +ISIS | Collimator | Oscillating Radial | | | [see Collimator note](#gem_noteCollimator) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#gem_noteVacuum) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#gem_noteEurotherm) +Oxford Instruments | Orange Cryostat| Cryogenic System | | | [see Oxford Instruments note](#gem_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | | | [see Oxford Instruments note](#gem_noteOxfordInstruments) +Oxford Instruments | Dilution Fridge (Kelvinox) | Cryogenic System | | | [see Oxford Instruments note](#gem_noteOxfordInstruments) + | | Closed Cycle Refrigerator | | | | [see CCR note](#gem_noteCCR) +ISIS | GEM-Furnace | Furnace | | | [see ISIS Furnaces note](#gem_noteISISFurnaces) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#gem_noteMcLennan) +ISIS | GEM Sample Changer | Sample Changer | RS-232 | | Same as POLARIS sample changer. See [#2173](https://github.com/ISISComputingGroup/IBEX/issues/2173) +MKS | PDR2000 | Pressure Transducer | RS-232 | | see [#1406](https://github.com/ISISComputingGroup/IBEX/issues/1406) and [#1620](https://github.com/ISISComputingGroup/IBEX/issues/1620) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | see [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + + +From time-to-time, GEM has also used magnets, gas handling and high-pressure systems, but not frequently. + +{#gem_noteDAE} +##### Note: DAE ##### +Main Detector banks + fixed monitors. + +{#gem_noteChopper} +##### Note: Choppers ##### +3 choppers: 2 double-disk and one T0 chopper.
+Choppers are ISIS Mk2 choppers. Mk2 choppers have a serial interface (not Ethernet like Mk3). + +{#gem_noteCollimator} +##### Note: Oscillating Radial Collimator ##### +The oscillating radial collimator is currently driven by a McLennan system, controlled via a custom controller at present. At some point this controller is going to be replaced by a Galil system, but this will not be done by September 2017. + +There have been instances where the collimator stops responding (see: https://github.com/ISISComputingGroup/IBEX/issues/3167). These cases present as the device responding to status requests, but not acting on requests to reinitialise or start the oscillator. In this case, the device has to be power-cycled. To do this, it must first be switched off via the front panel, then the power cable must be removed. Leave the device for several seconds before reconnecting and switching back on. If successful, the number of cycles should reset to zero. + +{#gem_noteJaws} +##### Note: Jaws ##### +GEM has 5 four-blade jaws and one bespoke set of jaws (called the "beam-scraper").
+For the "beam-scraper" jaws - see #[2138](https://github.com/ISISComputingGroup/IBEX/issues/2138)
+The 5 four-blade jaws were due to be replaced with Galil driven units during summer 2017. However, as of May 2017, the upgrade has been postponed:
+` +From Ivan da Silva: +Finally, due to other instrument works’ priorities, the GEM works for replacing the jaws will not be carried out during the next summer shutdown, but during winter 2017/2018. +Then, for 2017/02 cycle we will be still using the same jaws sets as for now. In principle, we will start using the new jaws on cycle 2017/04 (Feb-Apr 2018). +`
+Therefore, all jaws on GEM will continue to be controlled by LinMot controllers until, at least, cycle 2017/04. +Confirmed by Nick Webb (04-07-2017): new GEM jaws, controlled by Galil, will be installed in November 2017. + +{#gem_noteVacuum} +##### Note: Vacuum System ##### +GEM does not need to control the vacuum system, but it does need to be able to view and log vacuum gauge readings. + +{#gem_noteEurotherm} +##### Note: Eurotherm ##### +Used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#gem_noteOxfordInstruments} +##### Note: Oxford Instruments ##### + +1. Orange Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. Orange cryostats are controlled via a temperature controller (e.g. Eurotherm), so nothing on the cryostat itself for IBEX to control. +1. Blue Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. + 1. Are these the Heliox and ITC devices? + 1. The Heliox device has a sorption insert (is that relevant to the control system?) +1. Dilution fridge: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/dilution-refrigerators8825.html) to be determined. + 1. [Kelvinox dilution fridge](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/kelvinox-dilution-fridge/kelvinox-dilution-fridge13981.html) + +{#gem_noteCCR} +##### Note: Closed Cycle Refrigerators ##### +[CCRs](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/ccrs/closed-cycle-refrigerators-ccrs8446.html). Not directly computer-controlled - controlled via Eurotherm. + +{#gem_noteISISFurnaces} +##### Note: ISIS Furnaces ##### +More information on [IRIS Furnaces](http://www.isis.stfc.ac.uk/sample-environment/high-temperature/standard-furnaces/standard-furnaces13745.html). GEM uses: + +1. GEM Furnace +1. RAL 4 Furnace +1. Low temperature furnace + +Furnaces are controlled via a temperature controller (e.g. Eurotherm), so nothing on the furnace itself for IBEX to control. + +{#gem_noteMcLennan} +##### Note: McLennan ##### +[see Collimator note](#gem_noteCollimator) + +## GEM SECI Configs ## +Document information about GEM SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +----------------------------------------------------------|----------------------------------------------------------------|---------------|----------| +`gem_heliox.conf` | - | 14/11/2016 | - +`gem_sample_changer_matt.conf` | Choppers | 20/05/2016 | - +`gem_ccr_matt.conf` | Choppers | 28/04/2016 | - +`gem_sample_changer_Alex.conf` | Choppers | 24/02/2016 | - +`gem_lowT_furnace_matt.conf` | Choppers | 04/12/2015 | - +`gem_kelvinox_matt.conf` | - | 01/10/2015 | - +`GEM_CRYOSTAT.conf` | Choppers | 24/07/2015 | - +`gem_furnace_matt.conf` | Choppers, Furnace Pressure | 15/06/2015 | - +`gem_polaris_sample_changer_matt.conf` | Choppers | 26/04/2015 | - +`gem_furnace_ramp.conf` | Furnace Pressure | 03/04/2015 | - +`gem_furnace.conf` | Furnace Pressure | 02/04/2015 | - +`gem_3eurotherms.conf` | - | 20/03/2015 | - +`gem_sample_changer.conf` | Choppers | 18/03/2015 | - +`GEM_CRYOSTAT_matt.conf` | - | 02/06/2014 | - +`gem_ccr.conf` | Choppers | 02/04/2014 | - +`gem_8t cryo.conf` | GEM_Base, Choppers | 18/12/2013 | - +`5 Axis.conf` | - | 20/07/2013 | - +`gem_sample_changer_+_3_Eurotherms.conf` | 3_Eurotherms | 18/03/2013 | - +`gem_tizrcell_matt.conf` | - | 16/10/2012 | - +`gem_3eurotherms_pressure.conf` | - | 12/10/2012 | - +`gem_ambient.conf` | - | 04/10/2011 | - +`gem_3eurotherms_baritron.conf` | - | 16/04/2011 | - +`gem_express.conf` | - | 23/02/2011 | - +`gem_gazcell.conf` | - | 16/02/2011 | - +`gem_cryo + eurotherm.conf` | - | 15/02/2011 | - +`gem_7t cryomagnet.conf` | - | 15/02/2011 | - +`Dave.conf` | - | 15/02/2011 | - +`gem_kelvinox.conf` | - | 27/01/2011 | - +`gem_cryo + ilm.conf` | - | 27/01/2011 | - +`gem_cryo + ilm + SC.conf` | - | 27/01/2011 | - +`gem_cryo + heliox.conf` | - | 27/01/2011 | - +`gem_8t cryo + rotation.conf` | - | 27/01/2011 | - +`gem_8t cryo + kelvinox.conf` | - | 27/01/2011 | - +`gem_10t cryomagnet.conf` | - | 27/01/2011 | - + +## GEM Genie Scripts ## +Similarly, Document information about GEM SECI Genie scripts here. + +## GEM Notes ## +Add any notes about special items of equipment, setup or conditions on GEM that might impact the deployment and configuration of IBEX. diff --git a/doc/processes/instrument_details/HIFI_CRYOMAG-Instrument-Details.md b/doc/processes/instrument_details/HIFI_CRYOMAG-Instrument-Details.md new file mode 100644 index 000000000..84daea5ef --- /dev/null +++ b/doc/processes/instrument_details/HIFI_CRYOMAG-Instrument-Details.md @@ -0,0 +1,20 @@ +# HIFI-CRYOMAG + +This page contains information pertinent to the conversion of the HIFI_CRYOMAG control system from LabVIEW to EPICS. + +## Background & Timeline ## +A control system was supplied on a separate PC for use with the HIFI cryomagnet. In order to utilise this system more completely and to update the code it should be migrated to use EPICS. At present the interaction between HIFI and the cryomagnet is undertaken via Channel Access, the LabVIEW VI which provides this access can be slowly replaced using IOCs rather than the current drivers. + +## Control System ## +The control will be undertaken via EPICS and the IOCs. Some GUI interaction is likely to be required using IBEX for local control of the system, and via LabVIEW for the control from the instrument, + +## Equipment ## +Note that the order listed below is the most logical order for replacing the code within the system. Some items are required for multiple devices, e.g. the simple PSU controller is needed for X, Y and Z magnets. + +| Manufacturer | Model | Type | Connection | Driver | Notes | +| --- | --- | --- | --- | --- | --- | +| Lakeshore | LS460 | Gaussmeter | | | | +| Keithley | 2700 | Temperature Scanner | | | | +| [Cryomech](http://www.cryomech.com/) | | Compressor | | | This uses a specific protocol, so will not be an asyn driver | +| [Cryogenic](http://www.cryogenic.co.uk/) | SMS | PSU | | | There is need for a driver which includes the control of required cryogenic components | +| [Cryogenic](http://www.cryogenic.co.uk/) | SMS | PSU | | | There is need for a driver without the cryogenic components | \ No newline at end of file diff --git a/doc/processes/instrument_details/HRPD-Instrument-Details.md b/doc/processes/instrument_details/HRPD-Instrument-Details.md new file mode 100644 index 000000000..745608b34 --- /dev/null +++ b/doc/processes/instrument_details/HRPD-Instrument-Details.md @@ -0,0 +1,130 @@ +# HRPD + +This page collects information that will be useful for the implementation of the IBEX control system on HRPD. +## Background & Timeline ## +HRPD, the High Resolution Powder Diffractometer is an instrument on TS1 at ISIS. It is the highest resolution neutron powder diffractometer of its type in the world. The [HRPD](http://www.isis.stfc.ac.uk/instruments/hrpd/hrpd.html) web page describes the background to the instrument. + +There is a proposal to re-build HRPD at some point in the next 3-5 years. + +## Control System ## +HRPD will migrate from the SECI control system to the IBEX control system. + +## HRPD Equipment ## +The equipment listed below is used on HRPD. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | N/A | Shutter | N/A| | [see Shutter note](#hrpd_noteShutter) +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#hrpd_noteDAE) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | #169 | [see Mk3 Chopper note](#hrpd_noteMk3Chopper) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#hrpd_noteGalil) | +[LINMOT](http://www.linmot.com/) | [P0x-23](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) | Linear Motors and Motion Controller | RS-232 | [#2098](https://github.com/ISISComputingGroup/IBEX/issues/2098) | [see LinMot note](#hrpd_noteLinMot) | +??? | ??? | 1 x 4-blade jaws | | | [see Jaws note](#hrpd_noteJaws) +??? | ??? | Intermediate Shutter | | | [see Intermediate Shutter note](#hrpd_noteIMShutter) | +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | #216 |[see Pfeiffer note](#hrpd_notePfeiffer) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG26x | Gas Handling System | RS232 | #1411 |[see Pfeiffer note](#hrpd_notePfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#hrpd_noteEurotherm) +Chell | [CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100) | Pressure Transducer | RS-232 | #1827 | [see Chell note](#hrpd_noteChell) +Oxford Instruments | Dilution Fridge (Kelvinox) | Cryogenic System | | | [see Oxford Instruments note](#hrpd_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | | | [see Oxford Instruments note](#hrpd_noteOxfordInstruments) +??? | ??? | Pressure Transducer | | | [see Paris-Edinburgh Press note](#hrpd_notePEPress) +??? | ??? | Helium Level Meter | | | [see He Level Meter note](#hrpd_noteHeLM) +??? | ??? | Magnet| | | [see Magnets note](#hrpd_noteMagnets) +B&WTek | i-Raman Plus | Raman Spectrometer| | | [see mini-Raman Spectrometer note](#hrpd_noteRamanSpect) + +{#hrpd_noteShutter} +##### Note: Shutter ##### +HRPD and ENGIN-X share the same primary shutter (which is directly equivalent to the single shutter used on other instruments). HRPD and ENGIN-X also have their own secondary shutters (so that each instrument can operate independently of the other when the primary shutter is open. +1. When the primary shutter is closed, neither instrument receives neutrons. The status of the primary shutter is available in the same way as it is for any other instrument (i.e. via the appropriate PV). +1. When the primary shutter is open, HRPD receives neutrons only if the secondary shutter is also open. The status of the secondary shutter is not available to IBEX. + +The IBEX dashboard should continue to report the status of the primary shutter.
+**Note:** IBEX only ever reports the status of the shutter. IBEX _**never**_ controls the shutter (control of the shutter is part of the instrument safety system and is strictly outside the scope of IBEX). + +{#hrpd_noteDAE} +##### Note: DAE ##### +Main Detector banks + one working fixed monitor upstream of sample position. + +{#hrpd_noteGalil} +##### Note: Galil ##### +HRPD does not use any Galils at the present time. + +{#hrpd_noteLinMot} +##### Note: LinMot ##### +HRPD uses LinMot P0x-23 motors, controlled by LinMot drives.
+[LinMot User Manual](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) + +{#hrpd_noteJaws} +##### Note: Jaws ##### +Single 4-blade jaws. Mounted between guide exit and incident collimation slug. Jaws are driven by LinMot P0x-23 motors. + +{#hrpd_noteIMShutter} +##### Note: Intermediate Shutter ##### +HRPD features an "intermediate shutter". This is hardware controlled and does **NOT** fall within the scope of IBEX (i.e. IBEX will provide no facility to control the intermediate shutter). + +{#hrpd_noteMk3Chopper} +##### Note: ISIS Mk3 Choppers ##### +HRPD has two ISIS Mk3 choppers. + +{#hrpd_notePfeiffer} +##### Note: Pfeiffer ##### + +1. [Model TPG 26x] (https://www.pfeiffer-vacuum.com/en/products/measurement/activeline/controllers/?detailPdoId=5724), used on the furnace vacuum system +2. [Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used on the tank/guide vacuum system + +{#hrpd_noteChell} +##### Note: Chell ##### +[Chell CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100): +For future experiments requiring pressure transducer (pressure/gas flow data from gas panels). May need to consult with Chris Goodway - new transducer readouts are in progress. CCD100 is used in the gas panel. + +{#hrpd_noteEurotherm} +##### Note: Eurotherm ##### +Three Eurotherms in use on HRPD. Used to control temperature of all top-loading CCR/ He-cryostats and furnaces. + +{#hrpd_noteOxfordInstruments} +##### Note: Oxford Instruments ##### +Making use of these devices will be a priority for HRPD over the next 12 months (from May 2017 onwards). + +1. Dilution fridge: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/dilution-refrigerators8825.html) to be determined. + 1. [Kelvinox dilution fridge](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/kelvinox-dilution-fridge/kelvinox-dilution-fridge13981.html) +2. Blue Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. + +{#hrpd_notePEPress} +##### Note: Paris-Edinburgh Press ##### +Pressure transducers for supporting use of [Paris-Edinburgh press](http://www.isis.stfc.ac.uk/sample-environment/high-pressure/clamped-cells/paris-edinburgh-cells/paris-edinburgh-cells14179.html) will be required. HRPD may adopt same or similar configurations to PEARL. + +{#hrpd_noteHeLM} +##### Note: He Level Meter ##### +May be required in the future. No further information at present. + +{#hrpd_noteMagnets} +##### Note: Magnets ##### +May be required in the future. No further information at present. +See [magnets at ISIS](http://www.isis.stfc.ac.uk/sample-environment/high-magnetic-field/high-magnetic-field-8812.html) + +{#hrpd_noteRamanSpect} +##### Note: mini-Raman Spectrometer ##### +1. [B&W Tek i-Raman Plus](http://bwtek.com/products/i-raman-plus/) A Raman Spectrometer is something that HRPD would like to use at a future date (TBD). See also SXD. + +## HRPD SECI Configs ## +HRPD has numerous heritage SECI configurations only two of which are now used routinely. + +1. One is used for day-to-day operation with samples at RT, in cryostats or furnace (`HRPD_Eurotherms_2.conf`). +1. The second is used with the sample changer (`HRPD_Sample_Changer_new_oct16.conf`). + +All located in `C:\\Program Files (x86)\STFC ISIS Facility\SECI\Configurations` + +The HRPD team would probably wish to have separate configurations for experiments involving: + +1. High-pressure devices (incl. various pressure transducers) +1. Gas handling +1. Magnets +1. In-situ light scattering (laser control, spectrometer data acquisition) + +## HRPD Genie Scripts ## +The critical OpenGenie scripts, for initialisation and focussing, are in `C:\\OG` (duplicated, more or less, in `C:\\scripts\OG`) + +HRPD has a large collection of Genie scripts accumulated over the years, each of results from a specific user running a specific operation. A number of editable seed scripts are kept in (`C:\\scripts\keep`). These might appropriately be converted to genie_python. + +## HRPD Notes ## +Add any notes about special items of equipment, setup or conditions on HRPD that might impact the deployment and configuration of IBEX. diff --git a/doc/processes/instrument_details/IMAT-Instrument-Details.md b/doc/processes/instrument_details/IMAT-Instrument-Details.md new file mode 100644 index 000000000..8ee53aeaa --- /dev/null +++ b/doc/processes/instrument_details/IMAT-Instrument-Details.md @@ -0,0 +1,118 @@ +# IMAT + +This page collects information that will be useful for the implementation of the new control system on IMAT. +## Background & Timeline ## +The [IMAT](http://www.isis.stfc.ac.uk/instruments/imat/imat8259.html) web page describes the background to the instrument. Additional material is also available on the [IMAT sharepoint](http://www.facilities.rl.ac.uk/isis/projects/ts2/phase2instruments/IMAT/Forms/AllItems.aspx). + +Slides sets from IMAT operational review of 21st May 2014 are available: [Intro](http://www.facilities.rl.ac.uk/isis/projects/ts2/phase2instruments/IMAT/Reviews/IMAT%20Operational%20Review%20intro.pdf), [Science](http://www.facilities.rl.ac.uk/isis/projects/ts2/phase2instruments/IMAT/Reviews/IMAT%20Operational%20Review%20-Science.pdf), [Engineering](http://www.facilities.rl.ac.uk/isis/projects/ts2/phase2instruments/IMAT/Reviews/IMAT%20Operational%20Review%20-%20Main.pdf). + +The presentation [IMAT Overview](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_Overview.pptx) provides an overview of the IMAT system from a computing/control perspective. + +## Control System ## +IMAT will use the IBEX control system. + +## Data Collection ## +Discussions regarding IMAT data collection requirements are documented in [Minutes (15/10/2014)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_DataCollection_Storage_Minutes_15Oct2014.pdf) and [Minutes (28/04/2015)](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_DataCollection_CCDcamera_28April2015_Notes.docx). The [proposed file formats](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_DataCollection_Format_proposal_v2.docx) are also documented. + +## IMAT Equipment ## +The equipment listed below is used on IMAT. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +[SKF](http://www.skf.com/group/products/magnetic-systems/magnetic-systems-applications/neutron-chopper-instrumentation/index.html) | Double Disc Chopper | CHOPPER | Ethernet/Modbus | #617, #622 | [see SKF Choppers note](#imat_noteSKFChoppers) +[SKF](http://www.skf.com/group/products/magnetic-systems/magnetic-systems-applications/neutron-chopper-instrumentation/index.html) | T0 Chopper | CHOPPER | Ethernet/Modbus | #617, #622 | [see SKF Choppers note](#imat_noteSKFChoppers) +[SKF](http://www.skf.com/group/products/magnetic-systems/magnetic-systems-applications/neutron-chopper-instrumentation/index.html) | Double Disc Chopper | CHOPPER | Ethernet/Modbus | #617, #622 | [see SKF Choppers note](#imat_noteSKFChoppers) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG 300 | ISIS Vacuum System | RS232 | #216 | +OMRON PLC | | | TCP/FINS | #215 | [see OMRON PLC note](#imat_noteOMRONplc) +ISIS | Sample Attenuator | Pneumatic Actuator | | | [see Beam Attenuator note](#imat_noteBeamAttenuator) +ISIS | Pin Hole Selector | | | #265 | [see Pinhole Selector note](#imat_notePinhole) +ISIS | JAWS | GALIL-based | Ethernet | #178, #179 | [see Jaws note](#imat_noteJaws) +ISIS | Incident Slits | GALIL-based | Ethernet | #178, #179 | [see Incident Slits note](#imat_noteSlits) +ISIS | Monitors | | | #265 | [see Incident Slits note](#imat_noteMonitors) + | 7-axis Sample Positioning System | | | #435 | [see Sample Positioning System note](#imat_noteSampleStack) +ISIS | | Sample Environment | Various | | [see Sample Environment note](#imat_noteSampleEnvironment) +TBD | TBD | Camera Positioning Robot | TBD | | [see Camera Positioning Robot note](#imat_noteCameraRobot) +UC(Berkeley) | Camera1 | Med. Res. CCD Imaging Camera | Specific to camera | | [see Berkeley Camera note](#imat_noteBerkeleyCamera) +CNR | Camera2 | | | | [see Messina Camera note](#imat_noteMessinaCamera) +CNR | Camera Focussing | | | | [see Messina Camera note](#imat_noteMessinaCamera) +ISIS | DAE 2 or 3? | Detector Electronics | Ethernet | | [see DAE note](#imat_noteDAE) +Mantracourt | ADW15 | Load cell meter | RS232 or RS485 | | [see Mantracourt note](#imat_noteMantracourt) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | EPICS | | +Newport | [M-ILS50PP](https://web.archive.org/web/20130828020800/http://search.newport.com:80/?q=*&x2=sku&q2=ILS50PP) | Linear Stage, Stepper Motor | Ethernet | EPICS | [see Newport Motor note](#imat_noteNewportMotor) + +{#imat_noteSKFChoppers} +##### Note: SKF Choppers ##### +IMAT will have three sets of choppers: a double-disk chopper, a T0 chopper and a second double-disk chopper. All three SKF choppers use the same control system: see [SKF Modbus/TCP interface](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/892-0117%20Rev%20B%20modbus%20(Updated%202014-10-17).pdf) for details. Instrument scientists would like to control choppers as 5 independent disks. There is also a need to "park" the choppers in the open position (to allow white beam imaging). + +{#imat_noteBeamAttenuator} +##### Note: Beam Attenuator ##### +The purpose of beam attenuator is to protect the sample & camera while either is being positioned. The time taken to open/close the beam attenuator is about 1s. Driven by pneumatic actuator between limits (2 limit switches). Position is In/Out. Must communicate with sample positioner - needs both automatic operation and manual control. Also communicate with data acquisition (c.f. Pause/resume on SANS2D) and camera system. See [IMAT Fast Shutter](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_FastShutter.pptx) for a diagram. **Note:** the beam attenuator was originally known as the "fast shutter", but this name is now deprecated. See also: [IMAT Sample Attenuator Minutes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_SampleAttenuator_M5_computing_minutes_29Oct2014.docx). + +{#imat_noteOMRONplc} +##### Note: OMRON PLC ##### +The OMRON PLC will be used to control gate valves and instrument vacuum. It will also be used for control of the pneumatic actuator for the Beam Attenuator. + +{#imat_notePinhole} +##### Note: Pinhole Selector ##### +The Pinhole Selector is a wheel containing mounts for 6 pinhole apertures - set positions for each. Single axis motion control (stepper motor). Controlled by Galil. See [IMAT Pin Hole Selector](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_PinHoleSelector.pptx) for a diagram. See also: [IMAT Pin Hole Selector Details](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/PinholeSelector_2014.pdf) and [IMAT Pin Hole Selector Minutes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_PinholeSelector_Computing_minutes_28Nov2014.docx). + +{#imat_noteJaws} +##### Note: Jaws ##### +5 x beam-collimating jaw-sets, controlled by Galil.
See [IMAT Jaws & Collimating Slits](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_Jaws_Collimating_Slits.pptx) for a diagram. See also: [IMAT Jaws/Slits Minutes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Jaws-Slits_computing_minutes_17Nov2014.docx) + +{#imat_noteSlits} +##### Note: Slits ##### +X-Y slits (2 linear translations), controlled by Galil.
See [IMAT Jaws & Collimating Slits](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_Jaws_Collimating_Slits.pptx) for a diagram. + +{#imat_noteMonitors} +##### Note: Monitors ##### +6 monitors, controlled by Galil. In/Out set positions. + +1. M1 is positioned immediately upstream of the first double-disk chopper. +2. M2 is positioned immediately upstream of the second double-disk chopper. +3. M3 is positioned immediately downstream of the second double-disk chopper. +4. M4 is positioned immediately downstream of the pin-hole selector. +5. M5 is positioned immediately downstream of the beam attenuator. See [IMAT M5 Monitor](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_M5Monitor.pptx) for a diagram of the M5 monitor. +6. M6 position - to be communicated. + +{#imat_noteSampleStack} +##### Note: Sample Positioning System ##### +Comprises upper and lower stacks.
+Lower stack has X, Y & Z translational degrees of freedom, plus rotational degree of freedom about the Z-axis. Z range = 0-1000mm, X & Y ranges = +/-500mm. Rotation is 0-370 degrees.
+Upper stack will be an "off-the-shelf" tomography stack (3 rotational degrees of freedom). Most axes motors will be controlled by Galil (using Galil 4400 firmware).
+Some axes motors will be controlled by a Beckhoff system (further details required).
+Sample stack needs to communicate with fast shutter.
+See [IMAT Sample Positioning System](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_SamplePositioning.pptx) for a diagram. + +{#imat_noteSampleEnvironment} +##### Note: Sample Environment ##### +For information about the IMAT sample environment see: [​IMAT Sample Environment](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_SampleEnvironment_Dec2014.pdf) and [IMAT Sample Environment Minutes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_SampleEnvironment_Minutes_17Dec2014.docx). + +{#imat_noteCameraRobot} +##### Note: Camera Positioning Robot ##### +Details of robot positioning system are yet to be decided. In practice, control of the camera positioning robot may be handled separately from the instrument control system. See [IMAT Camera](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_Computing_Camera.pptx) for a illustration of the concept. + +{#imat_noteBerkeleyCamera} +##### Note: Berkeley Camera ##### +Medium resolution CCD Imaging Camera. The camera is being manufactured by UC(Berkeley). Assumption is that this camera will have its own control PC. + +{#imat_noteMessinaCamera} +##### Note: Messina Camera ##### +High resolution CCD Imaging Camera. The camera is being manufactured by CNR (University of Messina). Camera is being manufactured by CNR. Control of Andor i334T is via USB 2.0 connector. Camera will have its own control PC. See [IMAT Camera Autofocussing System​](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/RevSciInstrum_84_093701_IMAT%20Camera.pdf) for more details.
+Focussing of camera is performed using a 1 x linear motor, controlled by Newport. See [IMAT Camera Autofocussing System​](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/RevSciInstrum_84_093701_IMAT%20Camera.pdf)​ for more details. 2 x motors to control mirror tilt. Further details in the ​[camera handbook](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_CCD_camera_handbook.pdf) and [​camera electronics](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_CCD_camera_electronics_handbook.pdf) handbook. + +This camera has its own control PC that runs the IOCs. The hostname can be found in the `globals.txt`. + +{#imat_noteDAE} +##### Note: DAE ##### +12+ racks for 20K detector elements. + +{#imat_noteMantracourt} +##### Note: Mantracourt ##### +Used to measure load on IMAT sample stack . + +{#imat_noteNewportMotor} +##### Note: Newport Motor ##### +Newport Linear Stepper Motor. 50mm linear travel. Used to focus CNR camera. +Does this motor have its own controller h/w, or does it need a separate controller? +Further details in the [camera handbook](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_CCD_camera_handbook.pdf) and [​camera electronics](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IMAT/IMAT_CCD_camera_electronics_handbook.pdf) handbook. diff --git a/doc/processes/instrument_details/INES-Instrument-Details.md b/doc/processes/instrument_details/INES-Instrument-Details.md new file mode 100644 index 000000000..7b12013b0 --- /dev/null +++ b/doc/processes/instrument_details/INES-Instrument-Details.md @@ -0,0 +1,107 @@ +# INES + +This page collects information that will be useful for the implementation of the IBEX control system on INES. +## Background & Timeline ## +INES is a powder diffractometer instrument at ISIS, on TS1. The [INES](https://www.isis.stfc.ac.uk/Pages/ines.aspx) web page describes the background to the instrument. + +## Control System ## +INES will migrate from the SECI control system to the IBEX control system in late October 2018 (prior to Cycle 2018/03). + +## INES Equipment ## +The equipment listed below is used on INES. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | N/A | Shutter | | | [see Shutter note](#ines_noteShutter) +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#ines_noteDAE) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#ines_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#ines_noteGalil) | +??? | | 4-blade jaws | | | [see Jaws note](#ines_noteJaws) +ISIS | N/A | ISIS Vacuum System | | |[see Vacuum System note](#ines_noteVacuum) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#ines_noteVacuum) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#ines_noteEurotherm) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#ines_noteKeithley) + +{#ines_noteDAE} +##### Note: DAE ##### +INES uses DAE-2. INES has 2 fixed monitors. + +{#ines_noteShutter} +##### Note: Shutter ##### +INES shares a shutter with TOSCA. INES needs to know the status of the TOSCA shutter.
+INES also has a secondary shutter. The status of the secondary shutter cannot currently (September 2018) be read (it requires the installation of an appropriate electronic device to provide a signal). At the present time, INES has no need to monitor the secondary shutter via IBEX (for the same reasons, the secondary shutter cannot be monitored with SECI). + +{#ines_noteChopper} +##### Note: Choppers ##### +INES has a Mk3 chopper, which is shared with TOSCA. The chopper is usually set up to suit both instruments.
+TOSCA is first in the beamline, but INES needs to know the chopper settings. User should not be able to change the chopper settings (i.e. protect these with Manager Mode). + +{#ines_noteGalil} +##### Note: Galil ##### +INES has two Galil controllers. Used to control jaws, XY table, goniometers, transmission monitors. + +{#ines_noteJaws} +##### Note: Jaws ##### +INES has a 4-blade jaws. The jaws were manufactured in Italy, but are driven by Galil. + +{#ines_noteVacuum} +##### Note: Vacuum System ##### +INES occasionally uses a vacuum tank, which is mounted on the sample table (the XY table gets removed first!). +INES would like to be able to monitor the vacuum pressure in IBEX.
+The vacuum is monitored using a TPG300 pressure gauge. TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#ines_noteEurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/INES/INES_Eurotherms.jpg) are used to control temperature in the vacuum tank. + +{#ines_noteKeithley} +##### Note: Keithley ##### +1. [Keithley 2400 Series Source Meter](https://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter).
+See also tickets [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#2695](https://github.com/ISISComputingGroup/IBEX/issues/2695), [#2801](https://github.com/ISISComputingGroup/IBEX/issues/2801) and [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176). +1. Keithley 2410 is used only rarely (for neutron instrumentation projects) + +## INES SECI Configs ## +Document information about INES SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +5_Axis | - | 14/02/2011 | - | +AncientCharm | - | 14/02/2011 | - | +Blank | - | 18/02/2011 | - | +Eurotherms & Keithley | Keithley | 06/06/2014 | Yes | +Eurotherms | - | 19/05/2014 | Yes | +Goniometer | - | 01/07/2011 | Yes | +INES Base | - | 28/04/2014 | Yes | +practice | - | 14/02/2011 | - | +Sample_Changer | - | 01/07/2011 | Yes | +TEST | - | 19/08/2010 | - | +Tomography | - | 24/05/2011 | - | + +## INES Genie Scripts ## +Similarly, Document information about INES SECI Genie scripts here. + +## INES Notes ## +INES has the following specialist panels: +1. INES IV + 1. This is a SECI VI, customised for INES. We should check what information it displays that is not already included in the standard IBEX dashboard. A new OPI may be required. +1. GEM Jaws + 1. **Note:** GEM has been upgraded and now has new jaws. "GEM Jaws" may refer to the old GEM jaws VI. + 1. Check if the behaviour of the standard jaws OPI meets the needs of INES. +1. Walter SE `Rotacq2` + 1. This refers to an old rotation stage, which is no longer used. + +INES has the following devices under motion control: +1. Goniometer + 1. INES has 2 goniometers: one large, one small. + 1. Both goniometers are controlled by Galil controllers. +1. Jaws + 1. The jaws are controlled by Galil controllers. +1. Sample Changer + 1. INES has a 4 position sample changer, controlled by a Galil controller. +1. Transmission Monitor + 1. Transmission monitors on INES are fixed. +1. Neutron Camera + 1. Used to take images of the samples. Camera is mounted on a single vertical axis. Users should have the ability to move the camera in and out of the beam only. The instrument scientist would like the ability to control precise position of the camera (using manager mode). +1. XY Table + 1. INES XY table is used to move the sample parallel (y-axis) and perpendicular (x-axis) to the beam. Controlled by a Galil controller. + 1. The XY table will be upgraded (probably sometime before April 2019) with the addition of a rotational stage. The rotation axis will be the vertical axis (z-axis) and will also be controlled by a Galil. diff --git a/doc/processes/instrument_details/INTER-Instrument-Details.md b/doc/processes/instrument_details/INTER-Instrument-Details.md new file mode 100644 index 000000000..88d22133e --- /dev/null +++ b/doc/processes/instrument_details/INTER-Instrument-Details.md @@ -0,0 +1,212 @@ +# INTER + +This page collects information that will be useful for the implementation of the IBEX control system on INTER. +## Background & Timeline ## +INTER is a high-intensity chemical interfaces reflectometer instrument at ISIS, on TS2. The [INTER](https://www.isis.stfc.ac.uk/Pages/inter.aspx) web page describes the background to the instrument. + +## Control System ## + +As of 23/06/2021 INTER is running IBEX. + +## INTER Equipment ## +The equipment listed below is used on INTER. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#inter_noteDAE) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#inter_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +??? | | 4-blade jaws | | | [see Jaws note](#inter_note-jaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#inter_note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#inter_note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#inter_note-eurotherm) +Julabo | FP-50 | Water Bath | RS-232 | | [see Water Bath note](#inter_note-water-baths) +Julabo | FP-52 | Water Bath | RS-232 | | [see Water Bath note](#inter_note-water-baths) +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 100-10MG | Bi-Polar Power Supply | RS232 | #187 |[see Kepco note](#inter_note-kepco) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#inter_note-keithley) +[Linkam](http://www.linkam.co.uk/) | T95 | | RS232 | |[see Linkam note](#inter_note-linkam) +[Knauer](http://www.knauer.net/) | K6 | Electric valve drive | RS232 | |[see Knauer note](#inter_note-knauer) +[Knauer](http://www.knauer.net/) | 1050 | HPLC pump | ??? | | [see Knauer HPLC note](#inter_note-knauer-HPLC) +~Hitachi~ | ~L-7100~ | ~HPLC pump~ | ~???~ | | [see Hitachi note](#inter_noteHitachi) +[Jasco](https://jascoinc.com/products/chromatography/)| PU-4180 | HPLC pump| ??? | | [see JASCO HPLC note](#inter_note-jasco-hplc-pump) +Keyence | LK-G | Positioning Sensor | RS-232 | | [see Keyence note](#inter_note-keyence) +Nima Trough | ??? | Trough | ??? | | [see Nima Trough note](#inter_note-nima-trough) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#inter_note-thurlby) +Omega | iBTHX | Transmitter | Ethernet | | [see Omega note](#inter_note-omega) +Sensirion | SHT75 | Humidity Sensor| ??? | | [see Sensirion note](#inter_note-sensirion) +Watson Marlow | 323 | Peristaltic Pump | ??? | | [see Peristaltic Pumps note](#inter_note-peristaltic-pumps) +WPI | Aladdin-1000 | Syringe Pump | ??? | | [see Syringe Pumps note](#inter_note-syringe-pumps) +WPI | SP2xx | Syringe Pump | ??? | | [see Syringe Pumps note](#inter_note-syringe-pumps) +Biologic| SP200 | Potentiostat | ??? | | [see Biologic note](#inter_note-biologic) +Weeder | WTDAC-M | Analog Output Card | ??? | | [see Weeder note](#inter_note-weeder) +ISIS | | Environment Monitor | ??? | | [see ISIS Environment Monitor note](#inter_noteISISEnvironmentMonitor) + +{#inter_noteDAE} +##### Note: DAE ##### +See multi-detector and single-detector below. + +{#inter_noteChopper} +##### Note: Choppers ##### +INTER has a Mk3 chopper.
+ +##### Note: Motion safety ##### +INTER has a safety system which includes light curtains and some other bits, but these are controlled via a safety-rated PLC within a Beckhoff. It forwards a signal when it is "about to trip" motion, which we then use to send an `AB 1` command (abort all motion) to all of the galils. this is done to avoid positions being lost on open-loop axes. The actual forwarding code for this lives in `custom_records.db` in the settings area, and is loaded in by the `INSTETC` IOC. + +{#inter_note-jaws} +##### Note: Jaws ##### +Provide information about INTER jaws. (After reflectometry meeting these seem to be micro jaws which they borrow from another instrument this is not a galil controlled but have not yet got any extra information) + +{#inter_note-vacuum-system} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#inter_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/INTER/INTER_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#inter_note-water-baths} +##### Note: Water Baths ##### +INTER uses Julabo water baths +1. Julabo (which model(s)?) + +{#inter_note-kepco} +##### Note: KEPCO ##### +1. [Kepco BOP 100-10MG PSU](http://www.kepcopower.com/bophimod.htm).
See also [#3005](https://github.com/ISISComputingGroup/IBEX/issues/3005) + +{#inter_note-keithley} +##### Note: Keithley ##### +1. [Keithley 2400 Series Source Meter](https://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter).
+See also tickets [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#2695](https://github.com/ISISComputingGroup/IBEX/issues/2695), [#2801](https://github.com/ISISComputingGroup/IBEX/issues/2801) and [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176). + +{#inter_noteHitachi} +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs (may be obsolete). Check existing SECI VI for logic and manual.~ + +{#inter_note-linkam} +##### Note: Linkam ##### +There are VIs for this.
+See also [Linkam T95 controller](http://www.linkam.co.uk/t95-system-controllers/) and tickets [#1106](https://github.com/ISISComputingGroup/IBEX/issues/1106), [#1496](https://github.com/ISISComputingGroup/IBEX/issues/1496), [#1509](https://github.com/ISISComputingGroup/IBEX/issues/1509). + +{#inter_note-knauer} +##### Note: Knauer ##### +Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. + 1. Support now implemented (28-03-2019). See [#3781](https://github.com/ISISComputingGroup/IBEX/issues/3781). + +{#inter_note-knauer-HPLC} +##### Note: Knauer HPLC ##### +Knauer HPLC 1050 [Knauer HPLC 1050 is discontinued.](https://www.knauer.net/en/discontinued-smartline-pump-1050-successor-azura-p-61l/p14161). Check existing SECI VI for logic and manual. + 1. Support now implemented (04-01-2019). See [#3262](https://github.com/ISISComputingGroup/IBEX/issues/3262). + +{#inter_note-jasco-hplc-pump} +##### Note: JASCO HPLC Pump ##### +[JASCO HPLC pump](https://jascoinc.com/products/chromatography/hplc/modules/hplc-pumps/) is a new (for ISIS) model of HPLC pump. + * See `C:\LabVIEW Modules\Drivers\Jasco PU-4180 HPLC Pump\Documentation` for documentation. + * See also [#3743](https://github.com/ISISComputingGroup/IBEX/issues/3743) & [#3923](https://github.com/ISISComputingGroup/IBEX/issues/3923) + +{#inter_note-keyence} +##### Note: Keyence ##### +1. [Keyence Web-site](https://www.keyence.co.uk/index.jsp) +1. The Keyence LK-G is a laser positioning sensor (possibly superseded by a newer model). It is used when adjusting the height of the sample stage. Check the existing VI. + 1. See `C:\LabVIEW Modules\Instruments\INTER\Keyence LK-G` for existing VI. + 1. Current models of [Keyence LK-G laser sensors](https://www.keyence.co.uk/products/measure/laser-1d/index.jsp) + 1. The Keyence LK-G is always in position on the instrument, although not always used. + 1. Now implemented (24-07-2019). See [#3745](https://github.com/ISISComputingGroup/IBEX/issues/3745) + +{#inter_note-nima-trough} +##### Note: Nima Trough ##### +Nima Trough: SECI uses a manufacturer supplied VI (see `C:\LabVIEW Modules\Drivers\Nima Trough`). We may need to do the same in IBEX (via lvDCOM). + * **Note:** NIMA Technologies Ltd now seems to be part of [Biolin Scientific](https://www.biolinscientific.com/ksvnima).
+ * The NIMA trough is used regularly on SURF. The manufacturer supplied VI is used to view graphs showing information about thin films. + * The manufacturer has made additional software available for download. A copy of this software is located in `\\isis\shares\ISIS_Experiment_Controls\NIMA Trough\Nima_TR8.1.zip`. + * Support now implemented (25-05-2019). See [#3783](https://github.com/ISISComputingGroup/IBEX/issues/3783) + +{#inter_note-thurlby} +##### Note: Thurlby ##### +Thurlby EX355P PSU - see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). + * [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + * IOC & OPI updated (18-07-2019). See [#3784](https://github.com/ISISComputingGroup/IBEX/issues/3784) + +{#inter_note-omega} +##### Note: Omega ##### +[OMEGA™ iBTHX transmitter](https://www.omega.co.uk/pptst/IBTX_IBTHX.html) is a device to monitor and record barometric pressure, temperature, relative humidity, and dew point over an Ethernet network or the Internet. + * OMEGA devices have proved unreliable on other instruments (e.g. ZOOM, LOQ). Consider other types of device. + +{#inter_note-sensirion} +##### Note: Sensirion ##### +[Sensirion SHT75](https://www.sensirion.com/en/environmental-sensors/humidity-sensors/pintype-digital-humidity-sensors/) is a device to monitor and record humidity, temperature,… etc. + * According to the manufacturer's web-site the SHT7x series of devices is "end-of-life" as of 31/12/2018. + +{#inter_note-biologic} +##### Note: Biologic ##### +[Biologic SP200](https://www.bio-logic.net/products/multichannel-conductivity/sp-200-potentiostat-galvanostat/) is a potentiostat. + * See also [#618](https://github.com/ISISComputingGroup/IBEX/issues/618) + +{#inter_note-peristaltic-pumps} +##### Note: Peristaltic Pumps ##### +1. [Watson Marlow 323 Peristaltic Pump](http://www.watson-marlow.com/gb-en/range/watson-marlow/300-tube-pumps/323d/) + * Support now implemented (14-08-2019). See [#3786](https://github.com/ISISComputingGroup/IBEX/issues/3786) + +{#inter_note-syringe-pumps} +##### Note: Syringe Pumps ##### +1. [WPI Aladdin-1000 Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps/al1000-220.aspx) + * Support now implemented (20-06-2019). See [#3787](https://github.com/ISISComputingGroup/IBEX/issues/3787) +1. [WPI SP2xx Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps.aspx) - check specific model. + * Support now implemented (05-07-2018). See [#3261](https://github.com/ISISComputingGroup/IBEX/issues/3261) + +{#inter_note-weeder} +##### Note: Weeder ##### +1. [Weeder WTDAC-M](https://weedtech.com/wtdac-m.html) analog output card. + +{#inter_noteISISEnvironmentMonitor} +##### Note: ISIS Environment Monitor ##### +1. The ISIS Environment Monitor is a device to monitor various environmental properties (temperature, pressure, humidity, etc). There is an existing list of commands and a VI to work from. + * Suspect this is the Sensirion SHT75 device. + +##### Note: Vacuum Chamber ##### +1. Vacuum chamber – specialist setup using HV and vacuum with some safety trips integrated (instrument scientists says this is be a high priority to enable in IBEX). + * Check SECI VI. + * Is the vacuum chamber related to the INTER High Voltage VI (listed below)? + +## INTER Mirror Guides ## +INTER is due to get new mirror guides (prior to Cycle 2019/01). These will be operational (under SECI) before INTER is migrated to IBEX. There is a [sketch showing the three mirror guide sections](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/INTER/Defined%20Axis%20and%20Rotations.pdf) (MG1 - MG3). MG1 is fixed; it requires no control. MG2 and MG3 are moveable and, therefore, do require control. + +There will be, essentially, 3 modes of operation: + 1. **no bounce:** all three sections centred in height around the incident beam height to allow straight through beam on sample + 1. **one bounce:** this could be either from the middle section (MG2) or the focusing section (MG3), the latter being more likely + 1. **two bounces:** MG2 and MG3 both intercept the (reflected beam) with their inner bottom surface and deflect the beam up + +In modes 2 and 3 several beamline components need to track: intermediate slit (S1b), slit 2, sample monitor, sample height, slit 3 and detector. + +In May 2019, SECI was updated to handle the new mirror guides. IBEX will need to replicate the new functionality in SECI. The is a document [describing the SECI changes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/INTER/INTER%20SECI%20changes.docx). + +## INTER SECI Configs ## +Document information about INTER SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +-------------------------------------|----------------------------------------------------|---------------|----------| +INTER_place_holder1.conf | - | dd/mm/yyyy | - | +INTER_place_holder2.conf | - | dd/mm/yyyy | - | + +## INTER Genie Scripts ## +Similarly, Document information about INTER SECI Genie scripts here. + +## INTER Notes ## +INTER has the following specialist panels: +1. INTER Front Panel +1. INTER KEYENCE LK-G +1. INTER Galil DIO +1. INTER High Voltage +1. OFFSPEC Piezo Stage +1. SURF Galil DIO + +INTER has the following devices under motion control: +1. FOM +1. Jaws & Height +1. Jaws +1. Laser Gimbal +1. Reflectometer Sample Stack +1. Transmission Monitor +1. INTER Multi Detector +1. INTER Point Detectors +1. INTER Supermirror diff --git a/doc/processes/instrument_details/IRIS-Instrument-Details.md b/doc/processes/instrument_details/IRIS-Instrument-Details.md new file mode 100644 index 000000000..e00b0f70b --- /dev/null +++ b/doc/processes/instrument_details/IRIS-Instrument-Details.md @@ -0,0 +1,152 @@ +# IRIS + +This page collects information that will be useful for the implementation of the new control system on IRIS. + +## Online info + +* [Instrument Scientists](https://www.isis.stfc.ac.uk/Pages/Instrument-Scientists.aspx) IRIS row +* https://www.isis.stfc.ac.uk/Pages/iris.aspx +* [Beam Line Manual](https://www.isis.stfc.ac.uk/Pages/irisman.pdf) + +## Background & Timeline ## +IRIS is a long established instrument at ISIS, on TS1. The [IRIS](http://www.isis.stfc.ac.uk/instruments/iris/iris4691.html) web page describes the background to the instrument. A schematic layout of IRIS is shown in ​[this diagram](http://www.isis.stfc.ac.uk/images/instruments/iris-/iris-schematic4923.jpg).
+IRIS shares a port with [OSIRIS](http://www.isis.stfc.ac.uk/instruments/osiris/osiris4667.html). We may need to have a way of sharing information between the IRIS and OSIRIS control systems. It might also be worth migrating IRIS and ​OSIRIS to IBEX at the same time. + +## Control System ## +IRIS will migrate from the SECI control system to the IBEX control system. + +## IRIS Equipment ## +The equipment listed below is used on IRIS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#iris_noteDAE) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +[Pfeiffer](https://www.pfeiffer-vacuum.com/en/products/) | TPG 26x | ISIS Vacuum System | RS232 | | [see Pfeiffer note](#iris_notePfeiffer) +[Pfeiffer](https://www.pfeiffer-vacuum.com/en/products/) | TPG 300 | ISIS Vacuum System | RS232 | #216 | [see Pfeiffer note](#iris_notePfeiffer) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | #169 | +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#iris_noteMcLennan) +ISIS| | Sample Changer | via McLennan | | [see Sample Changer note](#iris_noteSampleChanger) +Leybold | 1040 100mm TL GM | Top-Loading Closed Cycle Refrigerator | | | [see Leybold note](#iris_noteLeybold) +Sumitomo | 4K 100mm TL GM | Top-Loading Closed Cycle Refrigerator | | | [see Sumitomo note](#iris_noteSumitomo) +Oxford Instruments | Dilution Fridge | Cryogenic System | RS-232 | | [see Oxford Instruments note](#iris_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | RS-232 | | [see Oxford Instruments note](#iris_noteOxfordInstruments) +ILL?| Orange Cryostat | Cryogenic System | RS-232 | | [see Orange Cryostat note](#iris_noteOrangeCryostat) +ISIS | Furnace | High temperature furnace | RS-232 | | [see Furnace note](#iris_noteFurnace) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#iris_noteEurotherm) +LakeShore | 218 | | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore)| [see LakeShore note](#iris_noteLakeshore ) +LakeShore | 336| | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#iris_noteLakeshore ) +LakeShore | 340 | | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#iris_noteLakeshore ) +MKS | PDR2000 | Pressure Transducer | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS) | [see MKS note](#iris_noteMKS ) +ISIS | Helium 3 Controller | All models at ISIS | RS-232 | | [see He3 Controller note](#iris_noteHe3Controller) +ISIS | Helium Level Gauge | All models at ISIS | RS-232 | | [see He Level Gauge note](#iris_noteHeLevelGauge) +ISIS | Exchange Gas Controller | All models at ISIS | RS-232 | | [see Exchange Gas Controller note](#iris_noteExchangeGas) +ISIS | CryoValve Controller | All models at ISIS | RS-232 | | [see CryoValve Controller note](#iris_noteCryovalve) + +{#iris_noteDAE} +##### Note: DAE ##### +Main Detector banks + 2-3 monitors. + +{#iris_notePfeiffer} +##### Note: Pfeiffer ##### + +1. [Model TPG 26x](https://www.pfeiffer-vacuum.com/en/products/measurement/activeline/controllers/?detailPdoId=5724) +2. [Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407) + +{#iris_noteMcLennan} +##### Note: McLennan ##### +1. One McLennan will eventually be used to control the rotating centre-stick, McLennan-Newport Rotation Stage +2. One McLennan is used to control the Sample Changer, currently set on port 1 +3. One McLennan has been used to control a stretching rig, currently set on port 2 + +The homing behaviour for the McLennans is currently under review. The McLennan IOC startup scripts have been partially customised for Iris based on observed behaviour for each device. This behaviour is subject to review by the development team. + +{#iris_noteSampleChanger} +##### Note: Sample Changer ##### +The Sample Changer is driven by the McLennan motor (i.e. it raises/lowers sample cans attached to the sample changer). There is a photograph of the [Sample Changer](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IRIS/IRIS_Sample_Changer.jpg), which shows the sample cans attached to the bottom of the sample changer. There is also a [diagram](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IRIS/IRIS_Sample_Changer_Diagram(log%20book).JPG) showing the dimensions of the Sample Changer (the diagram is taken from the instrument scientist's notebook - it is a sketch, not an engineering drawing (just to set your expectations :-) ). + +{#iris_noteLeybold} +##### Note: Leybold ##### +[CCR-11](https://www.isis.stfc.ac.uk/Pages/Top-Loading-CCRs.aspx). Not directly computer-controlled - controlled via Eurotherm. + +{#iris_noteSumitomo} +##### Note: Sumitomo ##### +[CCR-64](https://www.isis.stfc.ac.uk/Pages/Top-Loading-CCRs.aspx). Not directly computer-controlled - controlled via Eurotherm. + +{#iris_noteOxfordInstruments} +##### Note: Oxford Instruments ##### + +1. Dilution fridge: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/dilution-refrigerators8825.html) to be determined. +2. Blue Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. + +{#iris_noteOrangeCryostat} +##### Note: Orange Cryostat ##### +Orange Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. Eurotherm controlled. + +{#iris_noteEurotherm} +##### Note: Eurotherm ##### +Used to control temperature of top-loading CCRs, Be Filter, Furnace, Orange Cryostat devices. +There are at least 6 Eurotherm devices on IRIS, arranged in two crates of 3, as illustrated in the photograph of the [Eurotherm Crate](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IRIS/IRIS_Eurotherm_Triple_Crate.jpg). + +{#iris_noteLakeshore} +##### Note: LakeShore ##### + +1. [Model 218](http://www.lakeshore.com/products/Cryogenic-Temperature-Monitors/Model-218/Pages/Overview.aspx): Monitors temperature of Analyser Banks +2. [Model 336](http://www.lakeshore.com/products/cryogenic-temperature-controllers/model-336/Pages/Overview.aspx): +3. [Model 340](http://www.lakeshore.com/products/Cryogenic-Temperature-Controllers/Model-340/Pages/Overview.aspx): this model is now obsolete, having been replaced by the 336 and 350 models. + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) for a variety of Lakeshore temperature controllers. + +The Lakeshore 336 on IRIS is shown in the linked [photograph](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/IRIS/IRIS_Lakeshore_336.jpg). + +Documentation on the Lakeshore 336 IOC setup can be found [here](/specific_iocs/temp_controllers/Lakeshore336). + +{#iris_noteFurnace} +##### Note: ISIS High temperature furnace ##### +Controlled via Eurotherm. + +{#iris_noteMKS} +##### Note: MKS PDR2000 Pressure Transducer ##### +The [MKS PDR2000A](http://www.mksinst.com/product/Product.aspx?ProductID=175) provides power and readout up to two (2) Baratron® pressure transducers. + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS%20Instruments) for a variety of MKS devices (although there is a driver for a PDR4000 pressure transducer, there does not appear to be one for a PDR2000). + +{#iris_noteHe3Controller} +##### Note: He3 Controller ##### +Consists of ACM1000 Gauge controller, ACT200H Pump Controller, Neocera LTC-21 Temperature Controller + +{#iris_noteHeLevelGauge} +##### Note: He Level Gauge ##### +_no comment (as yet)_ + +{#iris_noteExchangeGas} +##### Note: Exchange Gas Controller ##### +_no comment (as yet)_ + +{#iris_noteCryovalve} +##### Note: CryoValve Controller ##### +Due to be superseded by the Exchange Gas Controller. + +## IRIS SECI Configs ## +Over the years, IRIS has built up a significant collection of SECI configs. Not all of them are current (some may no longer be used, or used only infrequently). We will need to investigate how to convert them to IBEX configs. The current list of IRIS configs is listed on [trac](https://trac.isis.rl.ac.uk/ICP/wiki/IRIS). +The config files are located in the folder `SECI\Configurations\` on the control server and have the extension `.conf`. Sub-configuration files have the extension `.comp` (component). The files with numbers as extensions (`.1`, `.2`, `.3`,etc.) are backups of the configuration and sub-configuration files. + +## IRIS Genie Scripts ## +Similarly, IRIS has built up a significant collection of genie scripts over the years. Again, many scripts are old and may no longer be used regularly. Most runs on IRIS are controlled via scripts. There will be a need to convert some of these scripts to genie-python, but probably not all. + +The current list of IRIS genie scripts is listed on [trac](https://trac.isis.rl.ac.uk/ICP/wiki/IRIS). + +## Tested IOCs ## + +| What | When | Notes | +| ---- | ---- | ----- | +| Eurotherm 1 | 15/09/2016 | Temps 1-3 com 6 | +| Eurotherm 2 | 15/09/2016 | Temps 4-6 com 14 | +| Eurotherm 3 | 15/09/2016 | single reading com 8 | +| Lakeshore 218 | 15/09/2016 | com 20 | +| Lakeshore 336 | 15/09/2016 | ip ls336-1 | +| MK3 Choppers | 15/09/2016 | | +| CryoValve | 15/09/2016 | com 13 | +| TPG300 | 15/09/2016 | | +| TPG268 | 15/09/2016 | com11 | + diff --git a/doc/processes/instrument_details/LARMOR-Instrument-Details.md b/doc/processes/instrument_details/LARMOR-Instrument-Details.md new file mode 100644 index 000000000..29665af7d --- /dev/null +++ b/doc/processes/instrument_details/LARMOR-Instrument-Details.md @@ -0,0 +1,184 @@ +# LARMOR + +This page collects information that will be useful for the implementation of the new control system on LARMOR. + +## Background & Timeline ## +The [Larmor](http://www.isis.stfc.ac.uk/instruments/larmor/larmor8239.html) web page describes the background to the instrument. The layout of the LARMOR beamline is shown in [this diagram](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR_Beamline_Diagram.pdf). +The proposed architecture for the LARMOR control system is described [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/Larmor%20Architecture.ppt). We also have a list of LARMOR's [motion control components](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR%20Beam%20Line%20-%20Motion%20Control%20Components.docx). + +## Control System ## +Larmor is currently using the IBEX control system. + +## LARMOR Equipment ## +The equipment listed below is to be used on Larmor. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS |Mk3 Chopper | CHOPPER | Ethernet/.NET | #170 |[see ISIS Mk3 Chopper note](#larmor_noteMk3Chopper) + | | Spin Flipper | | |[see SpinFlipper note](#larmor_noteSpinFlipper) +Delft | Spin Echo| Spin Echo control system| TBD | TBD |[see Spin Echo System note](#larmor_noteSpinEcho) +[JJ XRAY](http://www.jjxray.dk/) | [7-axis Sample Stack](http://www.jjxray.dk/products/positioning/multi-axis-stages/sample-stacks) [Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR%20JJXRAY%20SAMPLE%20STACK.pdf) | Sample Stack | | #304 |[see JJ X-Ray note](#larmor_noteJJXray) +ISIS | [Two Tier](https://github.com/ISISComputingGroup/IBEX/wiki/TwoTierSampleChanger) | Sample Changer || |[see SampleChanger note](#larmor_noteSampleChanger) + | | In/Out Monitor | | #267 |[see Monitor note](#larmor_noteMonitor) +ISIS | DAE 3 | Detector Electronics | Ethernet | #219 |[see DAE note](#larmor_noteDAE) +[CAEN](http://www.caen.it/csite/HomePage.jsp) | HT | Detector HV Control | Ethernet | #214 |[see CAEN HT note](#larmor_noteCAEN) +[Galil](http://www.galilmc.com/) | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | EPICS |[see Galil note](#larmor_noteGalil) +OMRON | | PLC | TCP/FINS | #217 |[see OMRON PLC note](#larmor_noteOMRON) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | #216 |[see Pfeiffer note](#larmor_notePfeiffer) +[Julabo](http://www.julabo.com/us?p=1) | [FL300](http://www.julabo.de/en/products/recirculating-coolers/fl300-recirculating-cooler) | Water Bath | RS232 | #187 |[see Julabo note](#larmor_noteJulabo) +[Julabo](http://www.julabo.com/us?p=1) | [FP50](http://www.julabo.de/en/products/refrigerated-circulators/refrigerated-heating-circulators) | Water Bath | RS232 | #188 |[see Julabo note](#larmor_noteJulabo) +[Eurotherm](http://www.eurotherm.co.uk/) | All models at ISIS | Temp Controller | RS232 | [ EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) |[see Eurotherm note](#larmor_noteEurotherm) +[Oxford Instruments](http://www.oxford-instruments.com/products) | ITC503 | Temp Controller | RS232 | |[see Oxford Instruments Temperature Controller note](#larmor_noteOI_ITC503) +[Oxford Instruments](http://www.oxford-instruments.com/products) | ILM200 series | Helium Level Meter | RS232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Oxford%20Instruments) |[see Oxford Instruments Helium Level Meter note](#larmor_noteOI_ILM200) +[Thurlby](http://www.tti-test.com/) | [EX355P](http://www.tti-test.com/products-tti/text-pages/psu-ex355p.htm) | DC Power Supply | RS232 | #156 |[see Thurlby note](#larmor_noteThurlby) +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 100-10MG | Bi-Polar Power Supply | RS232 | #187 |[see Kepco note](#larmor_noteKepco) +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 50M | Bi-Polar Power Supply | RS232 | #188 |[see Kepco note](#larmor_noteKepco) +[Tektronix](http://www.tek.com/oscilloscope#all) | [MSO 4104B](http://www.tek.com/oscilloscope/mso4000-dpo4000) | Oscilloscope | Ethernet | #234 |[see Tektronix note](#larmor_noteTektronix) +[Tektronix](http://www.tek.com/function-generator) | [AFG 3022B](http://www.tek.com/datasheet/signal-generator/afg3000-function-generator-arbitrary-function-generators) | Function Generator | Ethernet | #237 |[see Tektronix note](#larmor_noteTektronix) +[PI](http://www.physikinstrumente.com/en/index.php) | Rotation Stage | Nano-positioning Rotation Stage | RS232 | |[see Physik Instrument note](#larmor_notePI_Nano) +[Bio-Logic](http://www.bio-logic.info/) | | Stopped Flow Cell | RS232 | |[see BioLogic note](#larmor_noteBioLogic) + | 17T Cryomag | Super conducting magnet | RS232 | | +[Scientific Magnetics](http://www.scientificmagnetics.co.uk/) | | 3 Axis Super conducting magnet & VTI | RS232 | |[see Scientific Magnetics note](#larmor_noteSciMag) +[Lakeshore](http://www.lakeshore.com/Pages/Home.aspx) | All models at ISIS | | TBC | |[see Lakeshore note](#larmor_noteLakeshore) + | Syringe Pumps| | RS232 | |[see Syringe Pumps note](#larmor_noteSyringePumps) +~[Hitachi](http://www.hitachi-hightech.com/global/about/corporate/group/hhs/)~ | ~L-7100~ | ~HPLC Pump~ | ~RS232~ | |[see Hitachi note](#larmor_noteHitachi) +[Knauer](http://www.knauer.net/) | K6 | Electric valve drive | RS232 | |[see Knauer note](#larmor_noteKnauer) +[Linkam](http://www.linkam.co.uk/) | | | RS232 | |[see Linkam note](#larmor_noteLinkam) +[Keithley](http://www.keithley.co.uk/) | All models in use at ISIS | | RS232, USB | |[see Keithley note](#larmor_noteKeithley) + +{#larmor_noteMk3Chopper} +##### Note: ISIS Mk3 Chopper ##### +Location: Front End
+ +{#larmor_noteJaws} +##### Note: Jaws ##### +Location: various
+Driven by [Galils](#larmor_noteGalil) + +{#larmor_noteSpinFlipper} +##### Note: SpinFlipper ##### + +{#larmor_noteSpinEcho} +##### Note: Spin Echo System ##### +[Delft RF Flipper Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/Delft_RF_Flipper_Manual.pdf)
+ +{#larmor_noteJJXray} +##### Note: JJ X-Ray ##### +[7-axis sample stack](http://www.jjxray.dk/products/positioning/multi-axis-stages/sample-stacks)
+[Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR%20JJXRAY%20SAMPLE%20STACK.pdf) + +{#larmor_noteSampleStack} +##### Note: SampleStack ##### + +{#larmor_noteMonitor} +##### Note: Monitor ##### +In/Out positioning
+Driven by [Galils](#larmor_noteGalil) + +{#larmor_noteDAE} +##### Note: DAE ##### +Location: Screened Room
+DAE-3 + +{#larmor_noteCAEN} +##### Note: CAEN HT ##### +Location: Screened Room
+ +{#larmor_noteGalil} +##### Note: Galil ##### +Model: [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) + +{#larmor_noteOMRON} +##### Note: OMRON PLC ##### +ISIS Bench Air Line Control + +{#larmor_notePfeiffer} +##### Note: Pfeiffer ##### +Location: TBC - alongside PLCs
+[LARMOR](https://github.com/ISISComputingGroup/IBEX/wiki/LARMOR-Instrument-Details), [IMAT](https://github.com/ISISComputingGroup/IBEX/wiki/IMAT-Instrument-Details)
+Due: Day 1
+ +{#larmor_noteJulabo} +##### Note: Julabo ##### +Model: all models at ISIS
+ +{#larmor_noteOI_ITC503} +##### Note: Oxford Instruments Temperature Controller ##### +Model: ITC503
+Location: TBC
+ +{#larmor_noteOI_ILM200} +##### Note: Oxford Instruments Helium Level Meter ##### +Model: ILM200 series
+ +{#larmor_noteEurotherm} +##### Note: Eurotherm ##### +Model: all models at ISIS
+ +{#larmor_noteThurlby} +##### Note: Thurlby ##### +Model: all models at ISIS
+ +{#larmor_noteKEPCO} +##### Note: KEPCO ##### +1. Model: all models at ISIS +1. [Kepco BOP 100-10MG PSU](http://www.kepcopower.com/bophimod.htm).
See also [#3005](https://github.com/ISISComputingGroup/IBEX/issues/3005) + +{#larmor_noteTektronix} +##### Note: Tektronix ##### +1. Model: Oscilloscope [MSO 4104B](http://www.tek.com/oscilloscope/mso4000-dpo4000) +1. Model: Function Generator [AFG 3022B](http://www.tek.com/datasheet/signal-generator/afg3000-function-generator-arbitrary-function-generators) + +{#larmor_noteSampleChanger} +##### Note: SampleChanger ##### +Similar to that used on: SANS2D, translations done via sample stack
+See [Two-Tier Sample Changer](https://github.com/ISISComputingGroup/IBEX/wiki/TwoTierSampleChanger) + +{#larmor_notePI_Nano} +##### Note: Physik Instrument ##### +Model: Nano-positioning Rotation Stage
+[PI Software](http://www.physikinstrumente.com/en/products/prdetail.php?sortnr=1100018), [EPICS driver](http://www.aps.anl.gov/epics/modules/manufacturer.php#Physik%20Instrumente)
+ +{#larmor_noteBioLogic} +##### Note: BioLogic ##### +Model: Stopped Flow Cell
+ +{#larmor_noteSciMag} +##### Note: Scientific Magnetics ##### +1. [Scientific Magnetics](http://www.scientificmagnetics.co.uk/) +1. Model: 3 Axis Super conducting magnet & VTI (see [#1398](https://github.com/ISISComputingGroup/IBEX/issues/1398)). + +{#larmor_noteLakeshore} +##### Note: Lakeshore ##### +Model: all models at ISIS
+ +{#larmor_noteSyringePumps} +##### Note: Syringe Pumps ##### +Model: all models at ISIS
+ +{#larmor_noteHitachi} +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs) (may be obsolete). Check existing SECI VI for logic and manual: both VI and manual are located here: `C:\LabVIEW Modules\Drivers\Hitachi L-7100`.~ + * Update (27-06-2019): Hitachi pumps are no longer used. No longer any need to support them. See [#3780](https://github.com/ISISComputingGroup/IBEX/issues/3780). + + +{#larmor_noteMcLennan} +##### Note: McLennan ##### +Location: TBC
+As used on: Many instruments
+ +{#larmor_noteKnauer} +##### Note: Knauer Electric Valve Drive ##### +Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. + +{#larmor_noteLinkam} +##### Note: Linkam ##### +Location: TBC
+There are VIs for this.
+See also [Linkam T95 controller](http://www.linkam.co.uk/t95-system-controllers/) and tickets [#1106](https://github.com/ISISComputingGroup/IBEX/issues/1106), [#1496](https://github.com/ISISComputingGroup/IBEX/issues/1496), [#1509](https://github.com/ISISComputingGroup/IBEX/issues/1509). + +{#larmor_noteKeithley} +##### Note: Keithley ##### +Model: all models in use at ISIS
+There are VIs for this
diff --git a/doc/processes/instrument_details/LET-Instrument-Details.md b/doc/processes/instrument_details/LET-Instrument-Details.md new file mode 100644 index 000000000..f3c1f6f66 --- /dev/null +++ b/doc/processes/instrument_details/LET-Instrument-Details.md @@ -0,0 +1,129 @@ +# LET + +This page collects information that will be useful for the implementation of the IBEX control system on LET. + +## Background & Timeline ## +LET is a cold neutron multi-chopper spectrometer on TS2 at ISIS. The [LET](http://www.isis.stfc.ac.uk/instruments/let/let6414.html) web page describes the background to the instrument. + +## Control System ## +It is proposed to migrate from the SECI control system to the IBEX control system in time for Cycle 2018/04. +LET does currently use IBEX to control its SKF choppers (but that's all). + +## LET Equipment ## +The equipment listed below is used on LET. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +SKF | MB4150-G5 | Disk Chopper | Ethernet | | [see High-Speed SKF Disk Chopper note](#note-high-speed-skf-disk-chopper) | +Astrium | TBD | Disk Chopper | TBD | .NET | [see High-Speed Astrium Disk Chopper note](#note-high-speed-astrium-disk-chopper) | +ISIS | Mk3 | Disk Chopper | Ethernet | .NET | [see ISIS Chopper note](#note-isis-disk-chopper) | +N/A | N/A | Jaws | | | [see Jaws note](#note-jaws) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) |[see Pfeiffer note](#note-pfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +ISIS | Orange Cryostat | Cryogenic System | RS-232 | | [see Orange Cryostat note](#note-orange-cryostat) +Oxford Instruments | Heliox | Dilution Fridge | RS-232 | [#3739](https://github.com/ISISComputingGroup/IBEX/issues/3739) | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | [#2744](https://github.com/ISISComputingGroup/IBEX/issues/2744) | [see Oxford Instruments note](#note-oxford-instruments) +ISIS | ??? | Mezei Flipper | | [#3738](https://github.com/ISISComputingGroup/IBEX/issues/3738) | [see Mezei Flipper Notes](#note-mezei-neutron-spin-flipper) +ISIS | Furnace | High temperature furnace | RS-232 | | [see Furnace note](#note-isis-high-temperature-furnace) +Oxford Instruments | 9T | Magnet | ??? | [#2765](https://github.com/ISISComputingGroup/IBEX/issues/2765) | [see 9T Magnet note](#note-isis-9t-chopper-magnet) +Oxford Instruments | 14T | Magnet | ??? | | [see 14T Magnet note](#note-isis-14t-magnet) +[Lakeshore](http://www.lakeshore.com/Pages/Home.aspx) | 340 | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see Lakeshore note](#note-lakeshore) +ISIS | ORC | Collimator | via Galil | [#2535](https://github.com/ISISComputingGroup/IBEX/issues/2535) | [see LET Notes](#let-notes) +ISIS | McLennan | Sample Changer | via Galil | | [see LET Notes](#let-notes) +ISIS | McLennan | Rotation Stage | via McLennan | | [see LET Notes](#let-notes) +ISIS | McLennan-Newport | Rotation Stage | via McLennan | | [see LET Notes](#let-notes) + +##### Note: DAE ##### +Main Detector banks + several fixed monitors. + +##### Note: High-Speed SKF Disk Chopper ##### +LET currently has 2 high-speed SKF choppers (same as IMAT). These are currently controlled using a mini-IBEX system. +1. SKF G5 choppers also used on IMAT (see [#1907](https://github.com/ISISComputingGroup/IBEX/issues/1907)). + * [SKF MB4150-G5 Chopper](http://www.skf.com/uk/products/magnetic-systems/magnetic-systems-applications/neutron-chopper-instrumentation/index.html) + * [MBScope](http://www.skf.com/uk/products/magnetic-systems/software-downloads/MBScope-Beamline-software.html) software for SKF magnetic bearing choppers. + +##### Note: High-Speed Astrium Disk Chopper ##### +LET currently has 2 high-speed Astrium choppers. These are due to be replaced by 2 new SKF G5 choppers ~within 9-12 months (from September 2017).~ Latest news: The Astrium choppers will be replaced in early 2020 ~(one might even be replaced as early as August 2019).~, but more likely is chopper 5 in January/February shutdown and chopper 1 in the long shutdown. + +The Astrium choppers are currently controlled by 2 VMs running on NDHLET (one VM per chopper). The VMs communicate with the choppers, via a MOXA, using a .NET connection. + +##### Note: ISIS Disk Chopper ##### +LET currently has 1 Mk3 ISIS chopper. + +##### Note: Jaws ##### +LET has no jaws. + +##### Note: Pfeiffer ##### +[Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used for vacuum system read-back. + +##### Note: Eurotherm ##### +Eurotherms in use on LET. Used to control temperature of all top-loading CCRs, Orange cryostats, heaters and furnaces. + +##### Note: Lakeshore ##### +
**NOTE:** this entry added 07-02-2019.
+LET uses a Lakeshore Model 340 cryogenic temperature controller. +1. [Model 340](https://www.lakeshore.com/products/Cryogenic-Temperature-Controllers/Model-340/Pages/Overview.aspx). + * According to the Lakeshore web-site, the Model 340 is now obsolete. It has been replaced by the models 336 and 350. + +##### Note: Orange Cryostat ##### +Controlled via Eurotherm. + +##### Note: Oxford Instruments ##### +1. Dilution fridges: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx). LET uses both Triton and Heliox dilution fridges + * Triton + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) + * Heliox + * The Heliox dilution fridge is currently (November 2018) broken and is being repaired at Oxford Instruments. There is no ETA on when it will be fixed. There are no experiments scheduled to use the Heliox dilution fridge. Obviously, that will change when the Heliox is repaired. + +##### Note: ISIS 9T Chopper Magnet ##### +[ISIS 9T Chopper Magnet](https://www.isis.stfc.ac.uk/Pages/9T-chopper-magnet.aspx) +See also:[IBEX Support for 9T chopper magnet](/specific_iocs/Cryogenics). + +##### Note: ISIS 14T Magnet ##### +LET has, in the past, used the [ISIS 14T Magnet](https://www.isis.stfc.ac.uk/Pages/14T-Diffraction-Magnet.aspx). This magnet is currently being repaired and there is no ETA on when it might return to service. + +##### Note: Mezei Neutron Spin Flipper ##### +The Mezei Neutron Spin Flipper is a collection of devices (power supplies, signal generators & control PC) housed in a mobile rack. The control PC runs a Python script to control the power supplies & signal generators. The Python script accepts a set of simple commands. The Mezei Neutron Spin Flipper is, at the time of writing, used on LET and POLREF. In future, it may be used on other instruments too. + +There is a VI to control the PSU, located here: `C:\LabVIEW Modules\Instruments\POLREF\POLREF Flipper`. It was originally developed by Gareth Howells, but has subsequently been modified by students. Consult with Goran Nilsson & Andrew Caruana for more details. + +**N.B.:** LET plans to use this device in Cycle 2018/04. + +##### Note: ISIS High temperature furnace ##### +Controlled via Eurotherm. Furnaces are used only very rarely on LET. + +## LET Notes ## +LET has the following specialist systems: +1. LET specific oscillating radial collimator. Controlled by special program downloaded into Galil. + 1. See [2535](https://github.com/ISISComputingGroup/IBEX/issues/2535) +1. LET sample changer + * A sample changer has not often been used on LET. From Cycle 2018/04 LET will be using a sample changer of the same design as the MERLIN sample changer (the two devices should be interchangeable). +1. LET rotation stage + * LET has two rotation stages; + 1. A McLennan rotation stage (same as MERLIN, used with CCRs & Orange Cryostats)) + 1. A McLennan-Newport rotation stage (used with the 9T Magnet) + +## LET SECI Configs ## +LET has a number of SECI configurations. Equivalent IBEX configs for the following combinations of equipment are required: + * Eurotherm (for Cryostat or CCR) + rotation stage + * Eurotherm (for Cryostat or CCR) + ORC + rotation stage + * Eurotherm (for Cryostat or CCR) + dilution fridge (Triton) + rotation stage + * Eurotherm (for Cryostat or CCR) + dilution fridge (Triton)+ ORC + rotation stage + * Eurotherm (for Cryostat or CCR) + dilution fridge (Heliox) + rotation stage + * Eurotherm (for Cryostat or CCR) + dilution fridge (Heliox)+ ORC + rotation stage + * 9T Magnet + McLennan-Newport rotation stage + +## LET Genie Scripts ## +LET has a number of instrument specific Genie scripts. The key scripts to be migrated to genie_python are: + +Script| Purpose | Notes | +------------ | ------------- | ------------------------------------------- +`park_choppers`| It parks the choppers, of course | Needed because LET has 4 choppers +`set_ei5` | Sets chopper speed/phase | This is the LET version of `set_ei`. It is **not** the same as `set_ei` used on other excitations instruments. It would be helpful if `set_ei5` could be extended to better support vetoes and wiring tables. Ignore the older versions `set_ei1` - `set_ei4`. +`mono` | | + +On LET user scripts are usually created by modifying an old user script. Create a template genie_python script (based on an old user script) which can be used in future. + diff --git a/doc/processes/instrument_details/LOQ-Instrument-Details.md b/doc/processes/instrument_details/LOQ-Instrument-Details.md new file mode 100644 index 000000000..ae7b5b4b4 --- /dev/null +++ b/doc/processes/instrument_details/LOQ-Instrument-Details.md @@ -0,0 +1,241 @@ +# LOQ + +This page collects information that will be useful for the implementation of the IBEX control system on LOQ. +## Background & Timeline ## +LOQ is a time-of-flight SANS instrument instrument at ISIS, on TS1. The [LOQ](https://www.isis.stfc.ac.uk/Pages/LOQ.aspx) web page describes the background to the instrument. + +It has 3 collimation apertures (only A2 is computer-controlled), 4 monitors (insertion/removal of M3 is computer-controlled), and 2 detectors (an ORDELA He-3 MWPC 'main' detector and a annular 4-module 'high-angle' scintillator detector, _neither of which move!_). _There are no jaw sets on LOQ._ + +## Control System ## +LOQ will migrate from the SECI control system to the IBEX control system in time for Cycle 2018/04. + +## LOQ Equipment ## +The equipment listed below is used on LOQ. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. as necessary). + +For a _minimally functional instrument_: + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- |------------| ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#loq_note-dae) +ISIS | n/a | LiveView | | | [see Live View note](#loq_note-live-view) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#loq_note-choppers) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS] | [see Galil note](#loq_note-galil) +ORDELA | 2661N | Ordela Detector | | | [see Ordela Detector note](#loq_note-ordela-detector) +NI | ??? | Fieldpoint | Ethernet | | [see Ordela Detector note](#loq_note-ordela-detector) +~Omega~ | ~iBTHX~ | ~Transmitter~ | ~Ethernet~ | | ~[see Omega note](#loq_noteOmega)~ +~Omega~ | ~PAC~ | ~Intelligent Controller~ | ~Modbus~ | | ~[see Omega PAC note](#loq_note-omega-pac)~ +MOXA | ioLogik | Remote I/O Controller | Ethernet | | [see MOXA ioLogik note](#loq_note-moxa-iologik) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#loq_note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#loq_note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#loq_note-eurotherm) +Julabo | FP-51 | Water Bath | RS-232 | | [see Water Bath note](#loq_note-water-baths) +ISIS | n/a | Julabo Valve | | | [see Water Bath note](#loq_note-julabo-valve) +ISIS | LOQ | Sample Changer | | | [see Sample Changer note](#loq_note-sample-changer) +ISIS | LOQ | Sample Changer Scan | | | [see Sample Changer Scan note](#loq_note-sample-changer-scan) + +There is then a wide range of alternative sample environment used on a much less frequent basis, listed below in (approximate) decreasing order of importance: + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +CNR-ISIS | n/a | DLS | RS232 & Ethernet | | [see DLS note](#loq_note-dls) - commissioning soon +Thar | ISIS | Pressure Cell | | | [see Pressure Cell note](#loq_note-pressure-cell) +Danfysik | 8000 | PSU | RS232 | DFKPS | [see Danfysik note](#loq_note-danfysik) +Anton-Paar | Physica MCR-501 | Rheometer | | | [see Rheometer note](#loq_note-rheometer) +ISIS | RAL | V/Nb Furnace | | | [see Furnace note](#loq_note-furnace) +ISIS | RAL | Muon Optical Furnace | | | [see Furnace note](#loq_note-furnace) +ISIS | | CCR | | | [see Cryo note](#loq_note-cryo) +ISIS | | Orange Cryostat | | | [see Cryo note](#loq_note-cryo) +ISIS | LOQ | Couette Cell | | | [see Couette Cell note](#loq_note-couette-cell) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#loq_note-mclennan) +Unilever| | Extensional-Flow Cell | | | +ISIS | LOQ | T-Jump Cell | | | [see T-Jump Cell note](#loq_note-t-jump-cell) +Keithley | 2400 | Source Meter | RS-232 | #1826 | [see Keithley note](#loq_note-keithley) +Biologic| ??? | Stop-Flow Cell | | | +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#loq_note-thurlby) +Huber| SMC 9000 | Motion Controller | Ethernet | | [see Huber note](#loq_note-huber) + +{#loq_note-dae} +##### Note: DAE ##### +Also see Ordela Detector note below. + +{#loq_note-live-view} +##### Note: Live View ##### +1. LOQ has a SECI VI for visualising the data being acquired on the ORDELA detector in real-time. This is essential to efficient instrument operations and will need implementing in Ibex. +2. If it were possible to extend the functionality to include the high-angle detector that would be very welcome. + +{#loq_note-choppers} +##### Note: Choppers ##### +LOQ has an ISIS Mk3 chopper.
+ +{#loq_note-galil} +##### Note: Galil ##### +LOQ Galil drives 4 axes: Sample Changer _height _ stage, Sample Changer _vertical_ stage, A2, and M3. + +{#loq_note-ordela-detector} +##### Note: Ordela Detector ##### +1. [ORDELA](http://www.ordela.com/) (Oak Ridge Detector Laboratory) is a manufacturer of area-detector devices. It is a spin-off from Oak Ridge National Laboratory. Detector installed is a Model 2661N He-3 MWPC. +2. This detector is currently controlled (HV on/off, setting discriminator levels) by proprietary W95-era software running on a dedicated PC. The same PC also features a NI DAQ card which can be used to record data independently of the LOQ DAE. This facility is only ever used for detector diagnostics. +3. **LOQ's reliance on an ageing PC/software combination to turn the detector on/off and set the discriminator levels is a significant single-point-of-failure which has been flagged for many years but only recently received any resource (ask DPK). As part of the Ibex migration this issue should be addressed.** +4. The NI Fieldpoint device measures the detector voltage, the discriminator voltage and the detector temperature on three separate channels on separate modules (see VI). The plan is to replace this obsolete device with an NI cDAQ (model 9185?) with the appropriate modules as this is currently supported in IBEX. +5. The ORDELA detector can be damaged as a result of the neutron count rate being too high. Software is required to detect this over count rate (via the ISISICP) and move an aperture into a position to block the beam (axis controlled by Galil motion controller). Settings should be available to enter the over count rate level and the duration of the over count rate before any action is deemed necessary. The aperture has three discrete positions for normal operation. To block the beam it needs to be moved to the closest mid point of these positions. A message needs to be reported to a contact list and a corresponding message displayed on the instrument screen to identify a problem has been detected. +(VI references : 1. C:\LabVIEW Modules\DAE\DAE Detector Check.vi 2. C:\LabVIEW Modules\Instruments\LOQ\LOQ Detector Protect\LOQ Detector Protect.llb\LOQ - Detector Protect - Front Panel.vi) + +{#loq_noteOmega} +##### ~Note: Omega~ ##### +~[OMEGA™ iBTHX transmitter](https://www.omega.co.uk/pptst/IBTX_IBTHX.html) is a device to monitor and record barometric pressure, temperature, relative humidity, and dew point over an Ethernet network or the LOQnet. It is attached to the rear of the ORDELA detector.~ + +{#loq_note-omega-pac} +##### ~Note: Omega PAC~ ##### +~The OMEGA™ iBTHX (see above) is not reliable. It has been decided to replace it with a [OMEGA™ PAC Controller](https://www.omega.co.uk/pptst/OME-WISE-7000_SERIES.html) device to monitor and record temperature.~ + +{#loq_note-moxa-iologik} +##### Note: MOXA ioLogik ##### +Both the OMEGA™ iBTHX and OMEGA™ PAC devices (see above) have proved unreliable. It has been decided to use [MOXA ioLogik 12XX](https://www.moxa.com/en/products/industrial-edge-connectivity/controllers-and-ios/universal-controllers-and-i-os/iologik-e1200-series) devices to monitor and record temperature & voltage on the ORDELA detector. + +{#loq_note-vacuum-system} +##### Note: Vacuum System ##### +1. There are 2 vacuum gauges on LOQ. Both are TPG300's. One is atop the Galil at the sample position monitoring the collimation, and the other is integrated into the detector tank pumping system. +2. Only the detector tank TPG300 is currently read back in SECI. There are two sensors, one at the pump, and one on the tank. +3. **It is highly desirable to have the collimation TPG300 integrated into Ibex too.** +4. TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#loq_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/LOQ/LOQ_Eurotherms.jpg) are used to control the temperature of Orange Cryostats, CCRs and Furnace devices. And also to measure voltages from other sensors. + +{#loq_note-water-baths} +##### Note: Water Baths ##### +LOQ routinely uses 2 Julabo Water Baths. Both are Model FP-51's. In an emergency (ie, a bath failure) we would use Julabos from the ISIS inventory which include Model FP-50's and Model FP-52's. + +> The LOQ Julabo baths act in a pair, and depend on the external/internal status. If the baths are not behaving as you would expect, check whether internal and external have been flipped. We will check out of cycle if this could be caused by IBEX rebooting. + +{#loq_note-julabo-valve} +##### Note: Julabo Valve ##### +**There is an electrically-operated solenoid valve to allow scripted control of which Julabo Water Bath is thermostating the Sample Changer. This is implemented in SECI and will need replicating in Ibex.**
+A similar device could also be used on SANS2D, so a driver for the Julabo Valve will be valuable elsewhere. + +{#loq_note-sample-changer} +##### Note: Sample Changer ##### +1. The LOQ Sample Changer is driven by a Galil. It sits on a Height Stage which is also driven by a Galil. This combination accounts for >90% of sample environment usage on LOQ. +2. In SECI the Sample Changer can be driven 'manually' from a VI, or 'autonomously' from GCL scripts. +3. **Sample Changer scripts are created by a VB application called LOQscript developed by MJC. An Ibex-compatible equivalent of LOQscript will need to be provided.** + +{#loq_note-sample-changer-scan} +##### Note: Sample Changer Scan ##### +1. A VI called LOQ SC Scan allows the sample positions on the Sample Changer to be 'scanned in' using the LOQ alignment laser and the output recorded on a photocell. It was written by GDH. +2. The photocell is connected to a [Thor Labs Model PM100A power meter](https://www.thorlabs.com/thorproduct.cfm?partnumber=PM100A). + +{#loq_note-dls} +##### Note: Dynamic Light Scattering ##### +1. **This apparatus should be ready to commence commissioning in Autumn 2018.** Commissioning will take place on LOQ but the apparatus is intended to be available on LOQ, SANS2D, LARMOR and ZOOM. +2. The apparatus contains a 100 mW Coherent OBIS visible laser with the facility for remote control. +3. The scattered laser light will be processed by an LS Instruments multi-channel, multi-tau, correlator. +4. For further information see ticket [#3361](https://github.com/ISISComputingGroup/IBEX/issues/3361). + +{#loq_note-pressure-cell} +##### Note: Pressure Cell ##### +The Thar Pressure Cell normally just integrates with the Eurotherm controller. + +{#loq_note-danfysik} +##### Note: Danfysik ##### +1. : Currently a Danfysik Model 8000 controller is integrated into an ageing GEC power supply, but the GEC unit is due to be replaced in the near future as part of ISIS-wide obsolescence. It will most likely be replaced by a Danfysik power supply. Richard Hale may be able to advise what model. + 1. There is a possibility that the GEC power supply will be replaced prior to December 2018. The replacement power supply will be a Danfysik 8500 Model 859. +2. [Danfysik Power Supplies](http://www.danfysik.com/en/products/power-supplies/) +3. [User and Software Manuals](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FPower%20Supplies&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence) for System 8500. +4. See [#1208](https://github.com/ISISComputingGroup/IBEX/issues/1208) for comms settings. + +{#loq_note-rheometer} +##### Note: Rheometer ##### +1. This device is controlled by proprietary software (on a dedicated PC). Manufacturer is reluctant to allow it to be integrated with ISIS software (i.e. SECI or IBEX). +2. We have in the past used a Eurotherm controller to send a trigger pulse to initiate actions by the proprietary software. +3. Manuals describing the use of the Anton-Paar Physica MCR-501 Rheometer are located here: `\\isis\shares\ISIS_Experiment_Controls\AntonPaar_Physica_MCR501`. + +{#loq_note-furnace} +##### Note: Furnaces ##### +Furnaces normally just integrate with the Eurotherm controller. + +{#loq_note-cryo} +##### Note: CCRs and Cryostats ##### +1. CCR's and Cryostats normally just integrate with the Eurotherm controller. +2. _LOQ does not use Cryomagnets. Ever!_ + +{#loq_note-couette-cell} +##### Note: Couette Cell ##### +1. The Couette Cell is driven by a [McLennan PM1000](#loq_note-mclennan) (compatible with a PM600). +2. See also [#3105](https://github.com/ISISComputingGroup/IBEX/issues/3105) + +{#loq_note-mclennan} +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +{#loq_note-t-jump-cell} +##### Note: T-Jump Cell ##### +1. The T-Jump Cell is driven by a Keithley 2400. +2. See also [#3175](https://github.com/ISISComputingGroup/IBEX/issues/3175) + +{#loq_note-keithley} +##### Note: Keithley ##### +1. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter) +2. See also [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176) + +{#loq_note-thurlby} +##### Note: Thurlby ##### +1. Thurlby EX355P PSU - see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). +2. [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + +{#loq_note-huber} +##### Note: Huber ##### +1. [Huber](https://www.xhuber.com/en/). +2. According to Huber's web-site, a Huber SMC9000 is a motion controller. See [SMC 9300](https://www.xhuber.com/en/products/3-control/smc-9300/) which appears to be the latest of the 9000-series and [SMC 9000](https://www.xhuber.com/fileadmin/user_upload/downloads/usermanuals/9000_1103.pdf) +3. The Huber device has not been used on LOQ for some time:
+_I believe this device was a one-off use by a former member of the instrument team. SMK._
+It should not be considered a priority for support on LOQ. See also [#3502](https://github.com/ISISComputingGroup/IBEX/issues/3502) + +## LOQ Notes ## +LOQ has the following specialist panels: +1. + +LOQ has the following devices under motion control: +1. Aperture (A2) + * The aperture device is a plate with 3 different sized aperture holes. The Galil drives the plate to one of 3 predefined positions (each corresponding to one of the aperture holes). It would be useful to create a 4th position, corresponding to a solid portion of the aperture plate (so that it can be used to block the beam). +2. Sample Changer (translation & vertical) + * The sample changer moves in a grid fashion - left/right translation across the beam and up/down vertical translation. Sample changer positions can be set directly in mm, or via preset positions (defined in a file, which is edited regularly). The preset positions use several naming conventions, according to the type of sample rack in use. +3. Transmission Monitor (M3) + * moves in/out of the beam. + +## LOQ SECI Configs ## +Document information about LOQ SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +LOQ_NORMAL_25Hz_Cryostat.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_DurhamRack | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_Flowcell.conf | - | dd/mm/yyyy | see note 1 | +LOQ_NORMAL_25Hz_Furnace.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_Huber_stages.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_Magnet.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_PressuceCell.conf | - | dd/mm/yyyy | N | +LOQ_NORMAL_25Hz_PressureCell.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_Rheometer.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_SampleChanger.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_SampleChanger_eurotherm_controller.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_SampleChanger_eurotherm_controller_2.conf | - | dd/mm/yyyy | ? | +LOQ_NORMAL_25Hz_SampleChanger_NOLIVEVIEW.conf | - | dd/mm/yyyy | see note 2 | +LOQ_NORMAL_25Hz_Shear.conf | - | dd/mm/yyyy | see note 1 | +LOQ_NORMAL_25Hz_StopFlow.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_TJump.conf | - | dd/mm/yyyy | Y | +LOQ_NORMAL_25Hz_UnileverCell.conf | - | dd/mm/yyyy | Y | +LOQ_QUIET.conf | - | dd/mm/yyyy | see note 3 | + +##### Note: 1 ##### +Need to establish what devices these relate to! + +##### Note: 2 ##### +This is LOQ_NORMAL_25Hz_SampleChanger.conf without the second time regime. Occasionally used with non-standard detector mapping. + +##### Note: 3 ##### +Configures LOQ for running on internal clock. + +## LOQ Genie Scripts ## +Similarly, Document information about LOQ SECI Genie scripts here. + diff --git a/doc/processes/instrument_details/MAPS-Instrument-Details.md b/doc/processes/instrument_details/MAPS-Instrument-Details.md new file mode 100644 index 000000000..d71dd4150 --- /dev/null +++ b/doc/processes/instrument_details/MAPS-Instrument-Details.md @@ -0,0 +1,87 @@ +# MAPS + +This page collects information that will be useful for the implementation of the IBEX control system on MAPS. + +## Background & Timeline ## +MAPS is a chopper spectrometer on TS1 at ISIS. The [MAPS](http://www.isis.stfc.ac.uk/instruments/maps/) web page describes the background to the instrument. + +## Control System ## +MAPS will migrate from the SECI control system to the IBEX control system. + +## MAPS Equipment ## +The equipment listed below is used on MAPS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#maps_noteDAE) +ISIS | MK3| T0 Chopper | N/C | | [see T0 Chopper note](#maps_noteT0Chopper) | +ISIS | MK3 | Disk Chopper | Ethernet | EPICS | [see Disk Chopper note](#maps_noteDiskChopper) | +FZJ | Digital Drive | Fermi Chopper | Ethernet | TBD | [see Fermi Chopper note](#maps_noteFermiChopper) | +??? | TBD | Fermi Chopper Lift | TBD | | [see Fermi Chopper Lift note](#maps_noteFermiLift) | +LINMOT Jaws | TBD | Motion Controller | RS232 | EPICS | [see LinMot note](#maps_noteLinMot) | +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) |[see Pfeiffer note](#maps_notePfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#maps_noteEurotherm) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#maps_noteMcLennan) +ISIS| | Sample Changer | via McLennan | | [see Sample Changer note](#maps_noteMcLennan) +??? | ??? | Heater | RS-232 | | controlled via Eurotherm +ISIS | RAL | Furnaces | N/A | | controlled via Eurotherm +Neocera | LTC-21 | Temperature Controller | RS-232 | [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) | [see Neocera note](#maps_noteNeocera) + +{#maps_noteDAE} +##### Note: DAE ##### +Main Detector banks + several fixed monitors. + +{#maps_noteT0Chopper} +##### Note: MAPS T0 Chopper ##### +T0 chopper is generally operated in read-only mode (i.e. visiting scientists should **_never_** change its settings).
+T0 is an ISIS MK3 chopper. + +{#maps_noteDiskChopper} +##### Note: MAPS Disk Chopper ##### +User needs the ability to change speed & phase settings.
+Disk Chopper is an ISIS MK3 chopper. + +{#maps_noteFermiChopper} +##### Note: MAPS Fermi Chopper ##### +The fermi chopper controller is a Jülich model (FZJ - Forschungszentrum Jülich) - custom built for ISIS and believed to be a prototype being developed by both institutions. A copy of the manual (PDF) is in the shared area.
+The control program will need to include various safety-critical condition monitoring routines (e.g. overspeed, bearing voltage out-of-range). See manual for more details.
+The Fermi chopper on MAPS is primarily controlled via scripts. Scripts are used to calculate the speed & phase, based on the energy and frequency (this calculation could be folded into IOC). The OPI should merely display the energy, frequency, speed & phase, not allow the user to change them.
+Ideally, the Fermi Chopper should have a hierarchy of OPIs - at least, Simple UI (user) & Complex UI (manager). Talk with the instrument scientist about what is important when we design the IOC & OPI(s). + +{#maps_noteFermiLift} +##### Note: MAPS Fermi Chopper Lift ##### +A lift is used to raise/lower the Fermi chopper. Lowered means the Fermi chopper is in the beam; raised means it is out of the beam. It would be helpful if the lift can report its status to the IBEX GUI.
+Lift is controlled by a PLC, therefore IBEX can report status of lift by interrogating the PLC. IBEX will not be able to change the lift position - that will have to be done on the PLC control panel. Make/type of PLC - to be determined.
+Given the above, the requirement that "The Fermi chopper must not be operated (i.e. it must not be spinning) when it is in the raised position. Lift controls should not be accessible to the user." is automatically satisfied.
+ +{#maps_noteLinMot} +##### Note: LinMot ##### +LinMots are used to drive the jaws. There are two 4-blade sets: one named "fermi" and the other "sample" which relate to their positions in the beamline. + +{#maps_notePfeiffer} +##### Note: Pfeiffer ##### +[Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used for vacuum system read-back. May actually be read back via PLC (same PLC that controls the Fermi Chopper Lift). + +{#maps_noteEurotherm} +##### Note: Eurotherm ##### +Eurotherms in use on MAPS. Used to control temperature of all top-loading CCRs, Orange cryostats, heaters and furnaces. + +{#maps_noteMcLennan} +##### Note: McLennan ##### +1. One McLennan is used to control the rotating centre-stick, McLennan-Newport Rotation Stage + 1. Typically used to rotate sample (about vertical axis) into position and hold for periods of several minutes to several hours before moving on to next measurement.
+ +{#maps_noteNeocera} +##### Note: Neocera ##### +Neocera is used only occasionally. [Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +## MAPS SECI Configs ## +MAPS has a number of SECI configurations, which need to be migrated. + +## MAPS Genie Scripts ## +Critical OpenGenie scripts for initialisation and focussing. + +MAPS has a number of instrument specific Genie scripts. These need to be migrated to genie_python. + +## MAPS Notes ## +Add any notes about special items of equipment, setup or conditions on MAPS that might impact the deployment and configuration of IBEX. diff --git a/doc/processes/instrument_details/MARI-Instrument-Details.md b/doc/processes/instrument_details/MARI-Instrument-Details.md new file mode 100644 index 000000000..98e562e7c --- /dev/null +++ b/doc/processes/instrument_details/MARI-Instrument-Details.md @@ -0,0 +1,92 @@ +# MARI + +This page collects information that will be useful for the implementation of the IBEX control system on MARI. + +## Background & Timeline ## +MARI is a chopper spectrometer on TS1 at ISIS. The [MARI](https://www.isis.stfc.ac.uk/Pages/mari.aspx) web page describes the background to the instrument. + +## Control System ## +It is proposed that MARI will migrate from the SECI control system to the IBEX control system in time for Cycle 2018/04. + +## MARI Equipment ## +The equipment listed below is used on MARI. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 3 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | MK3| T0 Chopper | N/C | | [see T0 Chopper note](#note-mari-t0-chopper) | +ISIS | MK3 | Disk Chopper | Ethernet | EPICS | [see Disk Chopper note](#note-mari-disk-chopper) | +FZJ | Analogue Drive | Fermi Chopper | RS232 | EPICS | [see Fermi Chopper note](#note-mari-fermi-chopper) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#note-galil) | +??? | ??? | 1 x 4-blade jaws | | | [see Jaws note](#note-jaws) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) |[see Pfeiffer note](#note-pfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +Oxford Instruments | Dilution Fridge | Cryogenic System | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Triton | Cryogen-Free Dilution Fridge | Ethernet | | [see Oxford Instruments note](#note-oxford-instruments) +ISIS| | Transmission Monitor | via McLennan | | [see Transmission Monitor note](#note-transmission-monitor) +Neocera | LTC-21 | Temperature Controller | RS-232 | [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) | [see Neocera note](#note-neocera) +ISIS | One-Off | Cryogenic Sample Changer | ENET & RS232 | | [see Sample Changer note](#note-sample-changer) + +##### Note: DAE ##### +Main Detector banks + several fixed monitors. MARI uses DAE-3. + +##### Note: MARI T0 Chopper ##### +T0 chopper is generally operated in read-only mode (i.e. visiting scientists should **_never_** change its settings).
+T0 is an ISIS MK3 chopper. + +##### Note: MARI Disk Chopper ##### +MARI has two ISIS MK3 disk choppers. User needs the ability to change speed & phase settings.
+Ideally, the two choppers should seen as a single pair by visiting scientists; instrument scientists need the ability to control the two choppers independently (e.g. for diagnostic purposes). + +##### Note: MARI Fermi Chopper ##### +The fermi chopper is a Jülich model (FZJ - Forschungszentrum Jülich) - custom built for ISIS (therefore unique, although very similar to models installed on MAPS and MERLIN). A copy of the communications manual (PDF) is [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/MERLIN/Fermi%20Chopper%20Communications%20Protocol.pdf) and the operating manual [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/MERLIN/Fermi%20Chopper%20Operating%20Manual.pdf).
+The control program will need to include various safety-critical condition monitoring routines (e.g. overspeed, bearing voltage out-of-range). There are some aspects of the control system which are specific to MARI and will need to be incorporated into the IOC. Ideally, they would be modular (e.g. macro, configuration file) so that the main program would be compatible between the three systems (MERLIN, MAPS, MARI).
+David Keymer wrote the original VIs, which are a useful reference.
+The Fermi chopper on MARI is primarily controlled via scripts. Scripts are used to calculate the frequency & phase, based on the energy (longer-term this calculation could be folded into IOC). The OPI should merely display the frequency & phase, not allow the user to change them.
+Ideally, the Fermi Chopper should have a hierarchy of OPIs - at least, Simple UI (user) & Complex UI (manager). The current VI has too many details on it - instrument scientists would like to hide these. Talk with the instrument scientist about what is important when we design the IOC & OPI(s).
+ + +##### Note: Galil ##### +A Galil is used to drive the jaws. + + +##### Note: Jaws ##### +Single 4-blade jaw-set. Jaws are driven by a Galil. + +##### Note: Pfeiffer ##### +[Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used for vacuum system read-back. May actually be read back via PLC. + +##### Note: Eurotherm ##### +Eurotherms are in use on MARI. Used to control temperature of all top-loading CCRs, Orange cryostats, heaters and furnaces. + +##### Note: Oxford Instruments ##### +1. Dilution fridge: Triton: Cryogen-Free Dilution Fridge +1. Historically, MARI uses dilution fridges and Blue cryostats only very rarely. Most low temperature experiments use CCRs and Orange cryostats. + +##### Note: Transmission Monitor ##### +MARI has a single, fixed transmission monitor. + +##### Note: Neocera ##### +Neocera is used only occasionally. [Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +##### Note: Sample Changer ##### +This is a single-axis cryogenic sample changer with up to 4 positions (sample cans).
+The changer is now in use and runs with a Mclennan controller via RS232/MOXI. +There is still a plan to use a Beckhoff controller in future.
+The changer is driven by an instrument script which has the rotation angle corresponding to each position hard-coded in.
+The script will also perform a home scan on request. This is usual done when samples are mounted.
+The changer should only be rotated in one direction, otherwise there is a risk the head will unscrew itself. The instrument script knows and enforces this condition.
+The temperature control will be handled by a Eurotherm.
+[Project Details](http://www.facilities.rl.ac.uk/isis/projects/_layouts/listform.aspx?PageType=4&ListId={EB0AB6DD-0B50-4DBD-8C95-E45C95069C26}&ID=541&ContentTypeID=0x01000DF6FB717D3FBD47809FF94DE29BBBC4) and +[Project Plan](https://tasks.office.com/stfc365.onmicrosoft.com/en-US/Home/PlanViews/8bdu_4UTBUeZwF_PB8SLIpYAESGr) + +## MARI SECI Configs ## +MARI has a number of SECI configurations, which need to be migrated. + +## MARI Genie Scripts ## +Critical OpenGenie scripts for initialisation and focussing. + +MARI has a number of instrument specific Genie scripts. These have now been migrated to genie_python. + +## MARI Notes ## +1. MARI does not currently use a script generator, but there is interest in having one in the future. diff --git a/doc/processes/instrument_details/MERLIN-Instrument-Details.md b/doc/processes/instrument_details/MERLIN-Instrument-Details.md new file mode 100644 index 000000000..1446b3408 --- /dev/null +++ b/doc/processes/instrument_details/MERLIN-Instrument-Details.md @@ -0,0 +1,125 @@ +# MERLIN + +This page collects information that will be useful for the implementation of the IBEX control system on MERLIN. + +## Background & Timeline ## +Merlin is a high count rate, medium energy resolution, direct geometry chopper spectrometer on TS1 at ISIS. The [MERLIN](https://www.isis.stfc.ac.uk/Pages/Merlin.aspx) web page describes the background to the instrument. + +## Control System ## +MERLIN will migrate from the SECI control system to the IBEX control system. + +## MERLIN Equipment ## +The equipment listed below is used on MERLIN. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#merlin_noteDAE) +ISIS | TBD | T0 Chopper | TBD | | [see T0 Chopper note](#merlin_noteT0Chopper) | +ISIS | TBD | Disk Chopper | TBD | | [see Disk Chopper note](#merlin_noteDiskChopper) | +ISIS | TBD | Fermi Chopper | TBD | | [see Fermi Chopper note](#merlin_noteFermiChopper) | +ISIS | TBD | Fermi Chopper Lift | TBD | | [see Fermi Chopper Lift note](#merlin_noteFermiLift) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#merlin_noteGalil) | +??? | ??? | 1 x 4-blade jaws | | | [see Jaws note](#merlin_noteJaws) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) |[see Pfeiffer note](#merlin_notePfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#merlin_noteEurotherm) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#merlin_noteMcLennan) +ISIS| | Sample Changer | via McLennan | | [see Sample Changer note](#merlin_noteSampleChanger) +ISIS| | Collimator | via Galil| | [see Oscillating Radial Collimator note](#merlin_noteORCollimator) +Sumitomo | 4K 100mm TL GM | Top-Loading Closed Cycle Refrigerator | | | [see Sumitomo note](#merlin_noteSumitomo) +ILL?| Orange Cryostat | Cryogenic System | RS-232 | | [see Orange Cryostat note](#merlin_noteOrangeCryostat) +??? | ??? | Heater | RS-232 | | controlled via Eurotherm +ISIS | RAL | Furnaces | N/A | | controlled via Eurotherm +Neocera | LTC-21 | Temperature Controller | RS-232 | [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) | [see Neocera note](#merlin_noteNeocera) +McWhan | TIZR | Pressure Cell | N/A | | controlled manually? [see McWhan note](#merlin_noteMcWhan) + +{#merlin_noteDAE} +##### Note: DAE ##### +Main Detector banks + several fixed monitors. + +{#merlin_noteT0Chopper} +##### Note: MERLIN T0 Chopper ##### +T0 chopper is generally operated in read-only mode (i.e. visiting scientists should **_never_** change its settings).
+T0 is an ISIS Mk2 chopper. + +{#merlin_noteDiskChopper} +##### Note: MERLIN Disk Chopper ##### +User needs the ability to change speed & phase settings. Also provide the ability to park the disc.
+Disk Chopper is an ISIS Mk2 chopper.
+At present, scientists often have to press a button on the chopper electronics crate before they can change the settings. It would be helpful if this was no longer the case (i.e. design IOC to eliminate this extra step, if possible). +We believe this "button" is the local vs computer-control switch. If set to local, the chopper can only be controlled by the control panel on the crate. If set to computer-control, IBEX can control the chopper. Unfortunately, the chopper control system does not serve up the state of the local vs computer-control switch. + +{#merlin_noteFermiChopper} +##### Note: MERLIN Fermi Chopper ##### +The fermi chopper is a Jülich model (FZJ - Forschungszentrum Jülich) - custom built for ISIS (therefore unique, although very similar to models installed on MAPS and MARI). A copy of the communications manual (PDF) is [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/MERLIN/Fermi%20Chopper%20Communications%20Protocol.pdf) and the operating manual [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/MERLIN/Fermi%20Chopper%20Operating%20Manual.pdf).
+The control program will need to include various safety-critical condition monitoring routines (e.g. overspeed, bearing voltage out-of-range). There are some aspects of the control system which are specific to MERLIN and will need to be incorporated into the IOC. Ideally, they would be modular (e.g. macro, configuration file) so that the main program would be compatible between the three systems (MERLIN, MAPS, MARI). David Keymer wrote the original VIs, which are a useful reference.
+The Fermi chopper on MERLIN is primarily controlled via scripts. Scripts are used to calculate the frequency & phase, based on the energy (longer-term this calculation could be folded into IOC). The OPI should merely display the frequency & phase, not allow the user to change them.
+Current system exhibits odd behaviour at 600Hz - see if we can fix this when migrating to IBEX.
+Ideally, the Fermi Chopper should have a hierarchy of OPIs - at least, Simple UI (user) & Complex UI (manager). The current VI has too many details on it - instrument scientists would like to hide these. Talk with the instrument scientist about what is important when we design the IOC & OPI(s).
The 600Hz issue occurs when resending the speed setpoint or "switch drive on and run" +The fermi chopper originally exhibited strange behaviour (Ticket 2628) because the high and low words of the delay setpoint were being sent in the wrong order. The order should be the low word followed by the high word. The device appeared to "take" the setpoint and the setpoint readback was correct, but the actual delay readback would go towards another value instead. + +{#merlin_noteFermiLift} +##### Note: MERLIN Fermi Chopper Lift ##### +A lift is used to raise/lower the Fermi chopper. Lowered means the Fermi chopper is in the beam; raised means it is out of the beam. It would be helpful if the lift can report its status to the IBEX GUI.
+Lift is controlled by a PLC, therefore IBEX can report status of lift by interrogating the PLC. IBEX will not be able to change the lift position - that will have to be done on the PLC control panel. Make/type of PLC - to be determined.
+Given the above, the requirement that "The Fermi chopper must not be operated (i.e. it must not be spinning) when it is in the raised position. Lift controls should not be accessible to the user." is automatically satisfied.
+ +{#merlin_noteGalil} +##### Note: Galil ##### +Galils are used to drive the jaws. + +{#merlin_noteJaws} +##### Note: Jaws ##### +Single 4-blade jaw-set. Jaws are driven by Galils.
+Apparently, the position of these jaws is reported via a potentiometer (rather than an encoder), which might also have implications for working with limit switches (as in, it won't be possible). + +{#merlin_notePfeiffer} +##### Note: Pfeiffer ##### +[Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used for vacuum system read-back. May actually be read back via PLC (same PLC that controls the Fermi Chopper Lift). + +{#merlin_noteEurotherm} +##### Note: Eurotherm ##### +Three Eurotherms in use on MERLIN. Used to control temperature of all top-loading CCRs, Orange cryostats, heaters and furnaces. + +{#merlin_noteMcLennan} +##### Note: McLennan ##### +1. One McLennan is used to control the rotating centre-stick, McLennan-Newport Rotation Stage + 1. Typically used to rotate sample (about vertical axis) into position and hold for periods of several minutes to several hours before moving on to next measurement.
+ 1. MERLIN scientists would also like to be able to oscillate the rotating centre-stick (i.e. slowly rotate back & forth between two fixed angles). +2. One McLennan is used to control the Sample Changer. + +{#merlin_noteSampleChanger} +##### Note: Sample Changer ##### +The Sample Changer is driven by a McLennan motor. It is similar (but not identical) to the sample changer used on IRIS. + +{#merlin_noteORCollimator} +##### Note: Oscillating Radial Collimator ##### +The Oscillating Radial Collimator (presumably) is driven by a Galil motor controller. Oscillates at a fixed, low frequency - 0.2Hz. A similar system is used on LET. The mounting radius is 284.00 mm
+**Note: 31-07-2017:** The currently proposed design for the Oscillating Radial Collimator on MERLIN has been scrapped. A new design will be created. No need to progress this item until a new design is ready. + +{#merlin_noteSumitomo} +##### Note: Sumitomo ##### +MERLIN has a dedicated SUMITOMO 4K 100mm TL GM CCR: [CCR-62](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/ccrs/ccr-parameters---pdf9787.pdf). Not directly computer-controlled - controlled via Eurotherm. + +{#merlin_noteOrangeCryostat} +##### Note: Orange Cryostat ##### +MERLIN will soon have a dedicated Orange Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. Eurotherm controlled. + +{#merlin_noteNeocera} +##### Note: Neocera ##### +Neocera is used only occasionally. [Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +{#merlin_noteMcWhan} +##### Note: McWhan ##### +[McWhan pressure cell](http://www.isis.stfc.ac.uk/sample-environment/high-pressure-and-gas-handling-/clamped-cells/mcwhan-clamped-cell-8653.html). McWhan refers to the design of the cell (not to the manufacturer). Cell cannot be controlled directly from IBEX. It may be possible to report the current pressure via a device such as a Chell pressure transducer. +See also [General Clamped Cells](http://www.isis.stfc.ac.uk/sample-environment/high-pressure/clamped-cells/general-clamped-cells/general-clamped-cells14180.html) and [High Pressure Gas Cells](http://www.isis.stfc.ac.uk/sample-environment/sample-containers/high-pressure-gas-cells/high-pressure-gas-cells8936.html). + +## MERLIN SECI Configs ## +MERLIN has a number of SECI configurations, which need to be migrated. + +## MERLIN Genie Scripts ## +The critical OpenGenie scripts, for initialisation and focussing, are in `C:\\OG` (duplicated, more or less, in `C:\\scripts\OG`) + +MERLIN has a number of instrument specific Genie scripts. These need to be migrated to genie_python. + +## MERLIN Notes ## +Add any notes about special items of equipment, setup or conditions on MERLIN that might impact the deployment and configuration of IBEX. diff --git a/doc/processes/instrument_details/MUON-Front-End-Instrument-Details.md b/doc/processes/instrument_details/MUON-Front-End-Instrument-Details.md new file mode 100644 index 000000000..0018d3081 --- /dev/null +++ b/doc/processes/instrument_details/MUON-Front-End-Instrument-Details.md @@ -0,0 +1,32 @@ +# MUONFE + +This page contains information pertinent to the automation of the Muon Front End Beamline Control at ISIS. + +## Background & Timeline ## +The system is due on line September 2016. + +## Control System ## +Whilst the Front End control will be undertaken under IBEX, it must integrate with SECI as well as part of the system is to be used on existing instruments. Some of the IOCs will only be controlled from instrument PCs, whilst in some situations only monitoring will be required. + +## Equipment ## +| Manufacturer | Model | Type | Connection | Driver | Notes | +| --- | --- | --- | --- | --- | --- | +| Danfysik | 8000 | PSU | RS232 | DFKPS | Created, some outstanding changes needed. See #1208 for comms settings. | +| Danfysik | 9100 | PSU | RS232 | DFKPS | Created, some outstanding changes needed. See #1208 for comms settings. | +| Danfysik | 8800 | PSU | RS232 | DFKPS | Created, some outstanding changes needed. See #1208 for comms settings. | +| TDK | Lambda Genesys | PSU | RS232 | TDK_ LAMBDA_ GENESYS | Created, setup required | +| Glassman | PG200P2.5 | PSU | via PLC | | | +| | | Kicker | via PLC | | This is monitoring of a relay state | +| | | Momentum Slits | via Galil | Galil | This is a separate project, and the same Galil controls the individual beamline jaws. See [see Barndoors and Momentum Slits note](#muonfe_noteMomentumSlits) | +| Pfeiffer | TPG300 | Vacuum Monitor | RS232 | TPG300 | | +| | | Valve Control | via PLC | | This is the monitoring of various valves | +| Galil | DMC2280 | Motor Controller | Ethernet | EPICS | | +| | | PLC | | | This will be a modbus connection, as it is an existing PLC. The details are still unconfirmed | + +{#muonfe_noteMomentumSlits} +## Barndoors and Momentum Slits ## +The barn doors and momentum slits control the muon beam delivered to the 3 muon instruments: HIFI, muSR and EMU. The control of these devices is described on the [Barndoors & Momentum-Slits](/specific_iocs/motor_extensions/jaws/Barndoors-and-Momentum-Slits-on-MUON-Front-End) page. + +{#muonfe_noteMotion} +##### Note: Motion ##### +There is a project relating to the motion on the South Side Muons in progress, this information may be out of date \ No newline at end of file diff --git a/doc/processes/instrument_details/MUSR-Instrument-Details.md b/doc/processes/instrument_details/MUSR-Instrument-Details.md new file mode 100644 index 000000000..159c045b8 --- /dev/null +++ b/doc/processes/instrument_details/MUSR-Instrument-Details.md @@ -0,0 +1,146 @@ +# MUSR + +This page collects information that will be useful for the implementation of the IBEX control system on MUSR. +## Background & Timeline ## +MUSR is a general purpose muon spin rotation spectrometer at ISIS, on TS1. However, the emphasis of the experimental work conducted is investigating magnetism and superconductivity. The [MUSR](https://www.isis.stfc.ac.uk/Pages/MUSR.aspx) web page describes the background to the instrument. + +## Control System ## +MUSR will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## MUSR Equipment ## +The equipment listed below is used on MUSR. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#musr_noteDAE) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see motion note](#musr_note-motion) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#musr_note-eurotherm) +ICE | ICECube Dilution Fridge | Oxford ICE Cube | RS-232 | | [see Oxford ICE Cube note](#musr_note-Oxford-ICE-Cube) +[CAEN](http://www.caen.it/csite/HomePage.jsp) | V895 | Discriminator | ??? | |[see CAEN note](#musr_note-cAEN) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | | [see Oxford Instruments note](#musr_note-oxford-instruments) +Oxford Instruments | Mercury | Temperature Controller | RS-232 | | [see Oxford Instruments note](#musr_note-oxford-instruments) +Oxford Instruments | Heliox | He3 Refrigerator | | | [see Oxford Instruments note](#musr_note-oxford-instruments) +Lakeshore | 372 | Temperature Controller | | | [see Lakeshore note](#musr_note-lakeshore) +LakeShore | 340 | | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#musr_note-lakeshore ) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#musr_note-thurlby) +Danfysik | 8000 | PSU | RS232 | DFKPS | [see Danfysik note](#musr_note-danfysik) +~Gossen~ | ~64D 42P~ | ~PSU~ | ~???~ | | [see Gossen note](#musr_note-gossen) +MetroLab | PT2025 | Teslameter | ??? | | [see MetroLab note](#musr_note-metrolab) + +{#musr_noteDAE} +##### Note: DAE ##### +See multi-detector and single-detector below. + +{#musr_note-motion} +##### Note: Motion ##### +There is a project relating to the motion on the South Side Muons in progress, this information may be out of date + +##### Note: Choppers ##### +MUSR does not have a chopper. + +##### Note: Jaws ##### +Provide information about MUSR jaws. + +{#musr_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/MUSR/EMU_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#musr_note-Oxford-ICE-Cube} +##### Note: Oxford ICE Cube ##### +1. The ICE Fridge is a dilution fridge. + 1. **N.B.** It is **not** an Oxford Instruments device. It is from a different manufacturer. + 1. It cannot be compared to other models of dilution fridge. +1. The ICE control PC is referred to as an ICECube. + 1. This PC runs Windows XP. It should not be connected to the ISIS network. + 1. Communication with the equipment is via serial and a MOXA unit. +1. The dilution fridge temperature at low temperatures is controlled via a Lakeshore 370. +1. There is only one ICE dilution fridge which is used on MUSR mainly. It is available for EMU. +1. See ticket [#3876](https://github.com/ISISComputingGroup/IBEX/issues/3876) + +Documentation can be found at : `C:\LabVIEW Modules\Drivers\ICEOxford\ICECube\Documentation` +LabVIEW client can be found at : `C:\LabVIEW Modules\Drivers\ICEOxford\ICECube\ICECube - Client.llb\ICECube - Client.vi` + +{#musr_note-cAEN} +##### Note: CAEN ##### +The [CAEN V895](http://www.caen.it/csite/CaenProd.jsp?parent=11&idmod=49) is a 16-channel Leading Edge Discriminator. + +{#musr_note-oxford-instruments} +##### Note: Oxford Instruments ##### +1. Dilution fridge: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx) to be determined. + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) +1. Support for Mercury Temperature Controller was created as part of [#2840](https://github.com/ISISComputingGroup/IBEX/issues/2840) +1. Heliox devices are He3 Sorption Refrigerators. See [#3739](https://github.com/ISISComputingGroup/IBEX/issues/3739) and [#4549](https://github.com/ISISComputingGroup/IBEX/issues/4549). + +{#musr_note-lakeshore} +##### Note: LakeShore ##### +1. [Model 340](http://www.lakeshore.com/products/Cryogenic-Temperature-Controllers/Model-340/Pages/Overview.aspx): this model is now obsolete, having been replaced by the 336 and 350 models. +1. On muon instruments, the Triton fridges have an additional Lakeshore controller (the 372) to allow control and continuous readout of the sample temperature (the OI software doesn't allow this using the in-built Lakeshore, despite the unit having a dedicated channel for the sample thermometry). +1. [Model 372](https://www.lakeshore.com/products/categories/overview/temperature-products/ac-resistance-bridges/model-372-ac-resistance-bridge-temperature-controller) temperature controller. +1. There are [EPICS drivers](https://epics-controls.org/resources-and-support/modules/hardware-support/) for a variety of Lakeshore temperature controllers. + +{#musr_note-thurlby} +##### Note: Thurlby ##### +1. Thurlby EX355P PSU - see [#3784](https://github.com/ISISComputingGroup/IBEX/issues/3784) +1. [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + +{#musr_note-danfysik} +##### Note: Danfysik ##### +Need to check if MUSR does use Danfysik 8000 or a different model of PSU. +1. [Danfysik Power Supplies](http://www.danfysik.com/en/products/power-supplies/): model 8000 appears to have been superseded by model 8500. +1. [User and Software Manuals](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FPower%20Supplies&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence) for System 8500. +1. See [#1208](https://github.com/ISISComputingGroup/IBEX/issues/1208) for comms settings. + +{#musr_note-gossen} +##### Note: Gossen ##### +1. ~[Gossen 64D 42P PSU](https://www.sagatron-shop.de/Gossen-Metrawatt/Laboratory-power-supply/MSP-64D-KONSTANTER/GOSSEN-MSP-64D-Lab-Power-Supply.html)~ +1. Gossen PSU is no longer used on MuSR (superseded by [Thurlby](#musr_note-thurlby)) + +{#musr_note-metrolab} +##### Note: MetroLab ##### +1. [MetroLab PT2025](https://www.metrolab.com/products/pt2025/) + + +## MUSR Notes ## +MUSR has the following specialist panels: +1. MUON Front End Magnet Monitoring (IOC) +1. MUON Front End Control (IOC) +1. MUSR Field Viewer (IOC) +1. MUSR Lakeshore 340 (Custom Panel) + * See [see Lakeshore note](#musr_note-lakeshore). Check functionality against existing MUSR VI. +1. MUSR MUON Jaws (IOC) +1. MUSR CAEN (IOC) + * See [see CAEN note](#musr_note-cAEN). Check functionality against existing MUSR VI. +1. MUSR Transverse Field (TTI EX355P or ~Gossen 64D 42P PSU~) + * See [see Thurlby note](#musr_note-thurlby). Check functionality against existing MUSR VI. + * ~See [see Gossen note](#musr_note-gossen). Check functionality against existing MUSR VI.~ +1. HIFI MetroLab PT2025 + * See [see MetroLab note](#musr_note-metroLab). Check functionality against existing MUSR VI. +1. MUON Magnets - Danfysik + * See [see Danfysik note](#musr_note-danfysik). Check functionality against existing MUSR VI. +1. MUON Magnets - [MUON Rotation Control](/specific_iocs/daq/MuSR-Rotation-control) +1. MUON Magnets - [Zero Field Controller](/specific_iocs/magnets/Zero-field-controller) +1. ~MUON TRMC2~ (TRMC2 is obsolete, no longer used) +1. MUSR Steering - [Muon Steering magnets](/specific_iocs/magnets/MuSR-Steering-Magnets) + +MUSR has the following motion control items: +1. ~sample changer (2-axis)~ not needed (configuration item probably accidentally copied from another instrument) +1. ~transmission monitor~ not needed (configuration item probably accidentally copied from another instrument) + +MUSR also has the following system: +1. MUON Script Generator (muongui.exe) + +## MUSR SECI Configs ## +Document information about MUSR SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +EMU_place_holder1.conf | - | dd/mm/yyyy | - | +EMU_place_holder2.conf | - | dd/mm/yyyy | - | + +## MUSR Genie Scripts ## +Similarly, Document information about MUSR SECI Genie scripts here. + +## MUSR configs from EMU +Configurations have been removed that were migrated from SECI + EMU as they are being phased out or have not been in use. Commit `63303d94400cd0270b3c07357a08cbdf5945d8e8` should still contain the configurations before removal. \ No newline at end of file diff --git a/doc/processes/instrument_details/MUX-Instrument-Details.md b/doc/processes/instrument_details/MUX-Instrument-Details.md new file mode 100644 index 000000000..960cce53e --- /dev/null +++ b/doc/processes/instrument_details/MUX-Instrument-Details.md @@ -0,0 +1,3 @@ +# MUX + +MUX is the name of the system to run the Negative Muon items. More details to follow as the system becomes more operational. \ No newline at end of file diff --git a/doc/processes/instrument_details/NIMROD-Instrument-Details.md b/doc/processes/instrument_details/NIMROD-Instrument-Details.md new file mode 100644 index 000000000..92d28386f --- /dev/null +++ b/doc/processes/instrument_details/NIMROD-Instrument-Details.md @@ -0,0 +1,130 @@ +# NIMROD + +This page collects information that will be useful for the implementation of the IBEX control system on NIMROD. +## Background & Timeline ## +NIMROD is a diffractometer instrument at ISIS, on TS2. The [NIMROD](https://www.isis.stfc.ac.uk/Pages/NIMROD.aspx) web page describes the background to the instrument. + +## Control System ## +NIMROD will migrate from the SECI control system to the IBEX control system ahead of Cycle 2019/02. + +## NIMROD Equipment ## +The equipment listed below is used on NIMROD. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#nimrod_note-dae) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +??? | | 4-blade jaws | | | [see Jaws note](#nimrod_note-jaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#nimrod_note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#nimrod_note-vacuum-system) +Pfeiffer | TPG26x | Vacuum Gauge | RS-232 | EPICS | [see Vacuum Pump note](#nimrod_note-vacuum-pump) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#nimrod_note-eurotherm) +ISIS | NIMROD Sample Changer | Sample Changer | RS-232 | | [see NIMROD Sample Changer note](#nimrod_note-nimrod-sample-changer) +Julabo | FP-50 | Water Bath | RS-232 | | [see Water Bath note](#nimrod_note-water-bath) +Julabo | FP-52 | Water Bath | RS-232 | | [see Water Bath note](#nimrod_note-water-bath) +Oerlikon | Centre One | Pressure Gauge | RS-232 | EPICS | [see Oerlikon note](#nimrod_note-oerlikon) +MKS | PDR2000 | Pressure Transducer | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS) | [see MKS note](#nimrod_note-mks) +MKS | [PR4000B](https://www.mksinst.com/f/pr4000b-digital-power-supply) | Digital Pressure Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS) | [see MKS note](#nimrod_note-mks) +ISIS | [TiZr Cell](/specific_iocs/cells/TiZr-Sample-Can) | Eurotherms | RS-232 | | + +{#nimrod_note-dae} +##### Note: DAE ##### +Main Detector banks + fixed monitors. + +##### Note: Chopper ##### +NIMROD has no choppers. + +{#nimrod_note-jaws} +##### Note: Jaws ##### +NIMROD has 6 sets of 4-blade jaws controlled by Galils. +Motion of jaws is coordinated? (see NIMROD Jaws VI). + +{#nimrod_note-vacuum-system} +##### Note: Vacuum System ##### +The vacuum pressure on NIMROD is the pressure in the NIMROD tank/chamber itself. NIMROD only needs to monitor the vacuum pressure; scientists do not want to control the vacuum system from IBEX. Scientists also monitor the pressure of the guide, also using a TPG300. +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#nimrod_note-vacuum-pump} +##### Note: Vacuum Pump ##### +NIMROD uses a [vacuum pump](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/NIMROD/NIMROD_Turbo_Pump.jpg) (in conjunction with CCRs). A TPG26x is used to measure the pressure. +TPG26x support was originally implemented via [#1411](https://github.com/ISISComputingGroup/IBEX/issues/1411), [#2379](https://github.com/ISISComputingGroup/IBEX/issues/2379) and [#2578](https://github.com/ISISComputingGroup/IBEX/issues/2578) + +{#nimrod_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANS2D/SANS2D_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#nimrod_note-nimrod-sample-changer} +##### Note: NIMROD Sample Changer ##### +The NIMROD sample changer is a two-axis (vertical and rotation) sample changer controlled by a Galil. + +As of July 2023, the lower limit for the vertical axis is not physically working, so be careful when changing soft limits for the sample changer's vertical axis. The sample changer is set to home to the upper limit as the switch works. + +{#nimrod_note-water-bath} +##### Note: Water Bath ##### +NIMROD uses two types of water bath, both Julabo models: FP-50 and [FP-52](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/NIMROD/NIMROD_Julabo_FP52.jpg). +See [supported Julabo models](/specific_iocs/temp_controllers/Julabo) for more details. +NIMROD uses the FP-52 model of Julabo most frequently (+90%), used in conjunction with the [Sample Changer](#nimrod_note-nimrod-sample-changer). Sometimes NIMROD uses water as the coolant/heating medium; sometimes it uses oil or glycol as the coolant/heating medium (Julabo/Presto A40 device). + +According to the [Water Baths](https://www.isis.stfc.ac.uk/Pages/Water-Baths.aspx) page, NIMROD uses WB21 (which is a Julabo FB52-SL). This contradicts the above information - need to check this. The [Julabo Commands](/specific_iocs/temp_controllers/julabo_commands.xlsx) spreadsheet indicates that WB21 is configured to use the "small" Julabo command set. + +NIMROD also uses oil bath OB-1 (Presto A40 oil bath) from time to time. + +{#nimrod_note-oerlikon} +##### Note: Oerlikon ##### +Oerlikon Centre-One pressure gauges appear to obsolete.
+There is a manual here: `C:\LabVIEW Modules\Drivers\Oerlikon Centre One` + +{#nimrod_note-mks} +##### Note: MKS ##### +The [MKS PDR2000A](https://www.mksinst.com/f/pdr2000a-2-channel-digital-power-supply-and-display) Pressure Transducer provides power and readout up to two (2) Baratron® pressure transducers.
+The [MKS PR4000B](https://www.mksinst.com/f/pr4000b-digital-power-supply) Digital Power Supply and Display module is available with either one or two channels. The single-channel version controls one pressure transducer or one mass flow controller. The two-channel version supports up to two pressure transducers or two mass flow controllers, or one of each. + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS%20Instruments) for a variety of MKS devices (although there is a driver for a PR4000B pressure transducer, there does not appear to be one for a PDR2000). +The manual for the MKS PR4000B is here: `\\isis\shares\ISIS_Experiment_Controls\MKS_PR4000B_Controller`.
+The SECI VI for the MKS PDR2000 is here: `C:\LabVIEW Modules\Drivers\MKS PDR2000 Pressure Gauge`. + +**Note:** Please refer to ticket [#3962](https://github.com/ISISComputingGroup/IBEX/issues/3692) when implementing support for the PDR2000 on NIMROD. + +## NIMROD Specialist UI Panels ## +NIMROD has the following specialist panels/systems: +1. NIMROD Jaws - calculates jaw settings based on moderator projection. + +## NIMROD SECI Configs ## +Document information about NIMROD SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +NIMROD_temp.conf | - | dd/mm/yyyy | - | +NIMROD_SuperStick.conf | - | dd/mm/yyyy | - | +NIMROD_SC.conf | - | dd/mm/yyyy | - | +NIMROD_RB1410637.conf | - | dd/mm/yyyy | - | +NIMROD_RB1410416.conf | - | dd/mm/yyyy | - | +NIMROD_CCR.conf | - | dd/mm/yyyy | - | +NIMROD_Basic.conf | - | dd/mm/yyyy | - | +NiceIceSECI.conf | - | dd/mm/yyyy | - | + +## NIMROD Genie Scripts ## +On NIMROD, normal practice is to adapt an earlier script. There is no library of scripts. Most scripts are similar to those used on SANDALS. It might be useful to create some genie_python scripts to serve as examples. One script that could be converted (to serve as an example) is `fraser10_depwarmup.gcl`. + +Script Name | Notes | Last Accessed | Required | +--------------------------|---------------------------------------------------------|---------------|----------| +`fraser10_depwarmup.gcl` | Convert to use an a template/example for future scripts | dd/mm/yyyy | - | + +The "dry_run" feature used on SANDALS scripts could be handy for NIMROD too. + +## NIMROD Notes ## +#### Pumps #### +NIMROD tends to use a lot of different pumps (e.g. Oerlikon, Edwards, Pfeiffer). Worth having support for a variety of pumps to make switching between pumps easy. + +#### Gas Panel #### +NIMROD will be getting a new gas panel system, which will use two MKS PR4000-B pressure controllers. IBEX will need to display pressure readings (up to 4, 2 from each). + +#### Neutron Camera #### +There is a plan to install a neutron camera on NIMROD. The camera would be a Neutron Optics device. It will be mounted on a simple axis to move it in/out of the beam. + +This has been installed on Galil 4 axis 5 (D) on NIMROD. Motion setpoints have been added under the name `CAMERA` for this, with positions for in/out of the beam. + +![](NIMROD__Oxford_MCP_Camera.jpg) + +#### Electrochemical Cell Positioner #### +A device is being built to position an electrochemical cell in the beam. This is a single linear axis with continuous positioning. We need not concern ourselves with the cell itself as this will be controlled by the users. \ No newline at end of file diff --git a/doc/processes/instrument_details/NIMROD__Oxford_MCP_Camera.jpg b/doc/processes/instrument_details/NIMROD__Oxford_MCP_Camera.jpg new file mode 100644 index 000000000..ae5d55d53 Binary files /dev/null and b/doc/processes/instrument_details/NIMROD__Oxford_MCP_Camera.jpg differ diff --git a/doc/processes/instrument_details/OSIRIS-Instrument-Details.md b/doc/processes/instrument_details/OSIRIS-Instrument-Details.md new file mode 100644 index 000000000..4a616e390 --- /dev/null +++ b/doc/processes/instrument_details/OSIRIS-Instrument-Details.md @@ -0,0 +1,89 @@ +# OSIRIS + +This page collects information that will be useful for the implementation of the new control system on OSIRIS. +## Background & Timeline ## +OSIRIS is a long established instrument at ISIS, on TS1. The [OSIRIS](https://www.isis.stfc.ac.uk/Pages/osiris.aspx) web page describes the background to the instrument. A schematic layout of OSIRIS is shown in ​[this diagram](https://www.isis.stfc.ac.uk/Gallery/OSIRIS_3D_BW_Labelled.JPG).
+OSIRIS shares a port with [IRIS](https://www.isis.stfc.ac.uk/Pages/iris.aspx). We may need to have a way of sharing information between the IRIS and OSIRIS control systems. It might also be worth migrating IRIS and ​OSIRIS to IBEX at the same time. + +## Control System ## +OSIRIS will migrate from the SECI control system to the IBEX control system. + +## OSIRIS Equipment ## +The equipment listed below is used on OSIRIS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | #169 | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Motion note](#motion-control) | +[McLennan](http://www.mclennan.co.uk/) | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#note-mclennan) +[Pfeiffer](https://www.pfeiffer-vacuum.com/en/products/) | TPG 300 | ISIS Vacuum System | RS232 | #216 | [see Pfeiffer note](#note-pfeiffer) +Oxford Instruments | Triton | Cryogen-Free Dilution Fridge | Ethernet | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | 7.5T Magnet | Superconducting Magnet | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Variox | (Blue) Cryostat | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +[LakeShore](http://www.lakeshore.com/Pages/Home.aspx) | 218 | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#note-lakeshore) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#note-neocera) +ISIS | CryoValve Controller | All models at ISIS | RS-232 | | [see CryoValve Controller note](#note-cryovalve-controller) + +##### Note: DAE ##### +Main Detector banks + 2-3 monitors. + +##### Note: Pfeiffer ##### +Two controllers each with four active channels. +1. [Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407) +1. TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +##### Note: McLennan ##### +1. Controls Rotating centre-stick, McLennan-Newport Rotation Stage +1. Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +##### Note: Oxford Instruments ##### +1. Dilution fridge: Triton. + * See [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) +1. 7.5T [Cryomagnet](https://www.isis.stfc.ac.uk/Pages/75T-Magnet.aspx) (aka the Teslatron) + * See [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) +1. [Variox Cryostat](https://www.isis.stfc.ac.uk/Pages/Oxford-Variox-Cryostats.aspx) (OXF-08) + * See [#1389](https://github.com/ISISComputingGroup/IBEX/issues/1389), [#1390](https://github.com/ISISComputingGroup/IBEX/issues/1390), [#1391](https://github.com/ISISComputingGroup/IBEX/issues/1391) + +##### Note: Eurotherm ##### +Used to control temperature of top-loading CCRs, Furnace, Orange Cryostat devices. +OSIRIS has 3 Eurotherm devices, arranged in a single crate, as illustrated in the photograph of the [Eurotherm Crate](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/OSIRIS/OSIRIS_3x_Eurotherm.jpg). + +##### Note: LakeShore ##### +Three model 218 temperature controllers in total, each with 8 active channels. Two monitor the temperature of the Analyser and one the beryllium filter. +1. [Model 218](http://www.lakeshore.com/products/Cryogenic-Temperature-Monitors/Model-218/Pages/Overview.aspx). The [3 LakeShore devices](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/OSIRIS/OSIRIS_3x_LakeShore_218.jpg), are mounted in the OSIRIS rack. + +##### Note: Neocera ##### +1. [Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) +1. See also [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) + +##### Note: CryoValve Controller ##### +1. Implemented on IRIS - see [#1405](https://github.com/ISISComputingGroup/IBEX/issues/1405). The [CryoValve Controller](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/OSIRIS/OSIRIS_CryoValve.jpg) sits on the OSIRIS rack. + +## OSIRIS Notes ## +#### Specialist Panels/Systems #### +OSIRIS has the following specialist panels/systems: +1. IRIS CryoValve + * see [CryoValve Controller note](#note-cryovalve-controller) + +#### Motion Control #### +OSIRIS has the following devices under motion control: +1. Jaws + * 2-axis, North and South blades only. I.e. only vertical gap and centre can be controlled. +1. Beryllium Filter + * single axis, "IN/OUT" device moving between two setpoints. +1. Sample Changer + * **N.B.** For the avoidance of doubt: OSIRIS does not currently (September 2018) have a sample changer. There have been discussions about building one but, for the foreseeable future, there is no sample changer. + + +## OSIRIS SECI Configs ## +Over the years, OSIRIS has built up a significant collection of SECI configs. Not all of them are current (some may no longer be used, or used only infrequently). We will need to investigate how to convert them to IBEX configs. The config files are located in the folder `SECI\Configurations\` on the control server and have the extension `.conf`. Sub-configuration files have the extension `.comp` (component). The files with numbers as extensions (`.1`,`.2`,`.3`,etc.) are backups of the configuration and sub-configuration files. The current list of OSIRIS configs is listed on [trac](https://trac.isis.rl.ac.uk/ICP/wiki/OSIRIS). + + +## OSIRIS Genie Scripts ## +Similarly, OSIRIS has built up a significant collection of genie scripts over the years. Again, many scripts are old and may no longer be used regularly. Most runs on OSIRIS are controlled via scripts. There will be a need to convert some of these scripts to genie-python, but probably not all. + +UPDATE (11/01/2016) Franz has explained how the scripts are used: Usually, each time a user arrives to perform an experiment, they would create a new script by modifying an existing one and alter the appropriate parameters. There is NO common set of OSIRIS specific scripts and/or routines that are included in every new script. Therefore a conversion of OSIRIS scripts to Genie-Python couldn't be done en-masse. Although a few examples could be carefully chosen for users to work from. + +The current list of OSIRIS genie scripts is listed on [trac](https://trac.isis.rl.ac.uk/ICP/wiki/OSIRIS). diff --git a/doc/processes/instrument_details/PEARL-Instrument-Details.md b/doc/processes/instrument_details/PEARL-Instrument-Details.md new file mode 100644 index 000000000..93167c4be --- /dev/null +++ b/doc/processes/instrument_details/PEARL-Instrument-Details.md @@ -0,0 +1,153 @@ +# PEARL + +This page collects information that will be useful for the implementation of the IBEX control system on PEARL. + +## Background & Timeline ## +PEARL is a powder diffraction instrument on TS1 at ISIS, dedicated to high-pressure studies. The [PEARL](https://www.isis.stfc.ac.uk/Pages/pearl.aspx) web page describes the background to the instrument. + +### PEARL networking + +PEARL has a slightly complicated networking situation, where they have some patch panels that propagate through to next to their sample prep/setup cabin. + +To summarise: + +Ports `SC0-SC10` are patched through to the top patch panel in the setup cabin. `SC6-9` are actually on the outside wall of the setup cabin, near where their vacuum tanks live. `W269-272` should patch from the setup cabin to the patch panel at the back of the main cabin. + +Galils, choppers etc are fed through `DS31-DS35` which lives in the little server room down the stairs. + +There is a large patch panel in the main PEARL cabin which links to the blockhouse ports, sat above some Eurotherm racks. The confusing bit - there are also ports that live outside the setup cabin, on the left side above the beamline itself. Ports 1-5 here ALSO route through to the large rack-mounted patch panel which lives in the main cabin. For example, port 3 outside the setup cabin, commonly used for the `PACE 5000` pressure controller, routes to port 3 on the large patch panel (in the rack in the PEARL cabin). This is then fed to the Moxa which is sat above. Each of the ports on the large patch panel have a code that looks like `9MC4180` which means nothing to us - we only care about the patch panel number which is underneath. + +## Control System ## +PEARL will migrate from the SECI control system to the IBEX control system. + +## PEARL Equipment ## +The equipment listed below is used on PEARL. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | #169 | [see Mk3 Chopper note](#pearl_note-mk3-chopper) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#note-galil) | +[LINMOT](http://www.linmot.com/) | [P0x-23](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) | Linear Motors and Motion Controller | RS-232 | [#2098](https://github.com/ISISComputingGroup/IBEX/issues/2098) | [see LinMot note](#note-linmot) | +??? | ??? | 1 x 4-blade jaws | | | [see Jaws note](#note-jaws) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | #216 |[see Pfeiffer note](#note-pfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +LakeShore | 336| Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#note-lakeshore ) +~Oxford Instruments~ | ~ITC503~ | ~Cryogenic System~ | | | [see Oxford Instruments note](#note-oxford-instruments) +Sorensen | High-PT | PSU | | | [see Sorensen note](#note-sorensen) +Pace | 5000 | Pressure Controller | RS-232 | | [see Pace note](#note-pace) +ISIS | PEARL | Pressure Cell Controller | | | [see Pressure Cell Controller note](#note-pressure-cell-controller) +ISIS | PEARL | MMP ~200~ or 210 Pressure Transducer | | | [see PEARL Pressure Transducer note](#note-pearl-pressure-transducer) +ISIS | PEARL | Temperature Controller | | | [see PEARL Temperature Controller note](#note-pearl-temperature-controller) +ISIS | ??? | Cold Valve| | | [see Cold Valve note](#note-cold-valve) + +##### Note: DAE ##### +Main Detector banks + one working fixed monitor upstream of sample position. + +##### Note: Galil ##### +PEARL uses Galil controllers. + +##### Note: LinMot ##### +PEARL uses LinMot P0x-23 motors, controlled by LinMot drives.
+[LinMot User Manual](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) +1. Alignment Mirror: Driven either IN or OUT by a single LinMot axis. +1. Also jaw sets mentioned below + +##### Note: Jaws ##### +1. Two sets of jaws are driven by LinMot P0x-23 motors... +1. ... and one by Galil +1. PEARL has custom screens for controlling the jaws + * See `C:\LabVIEW Modules\Instruments\PEARL\PEARL Jaws\Screens` + +{#pearl_note-mk3-chopper} +##### Note: ISIS Mk3 Choppers ##### +PEARL has two ISIS Mk3 choppers. + +##### Note: Pfeiffer ##### +1. [Model TPG 26x](https://www.pfeiffer-vacuum.com/en/products/measurement/activeline/controllers/?detailPdoId=5724), used on the furnace vacuum system +2. [Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407), used on the tank/guide vacuum system + +##### Note: Eurotherm ##### +Three Eurotherms in use on PEARL. Used to control temperature of all top-loading CCR/ He-cryostats and furnaces. + +##### Note: LakeShore ##### +1. [Model 336](http://www.lakeshore.com/products/cryogenic-temperature-controllers/model-336/Pages/Overview.aspx): + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) for a variety of Lakeshore temperature controllers. +Documentation on the Lakeshore 336 IOC setup can be found [here](/specific_iocs/temp_controllers/Lakeshore336). + +##### Note: Sorensen ##### +Sorensen appear to have been taken over by [AMETEK](http://www.programmablepower.com/brands/sorensen.htm).
+1. VI for Sorensen PSU is here: `C:\LabVIEW Modules\Instruments\PEARL\PEARL High-PT PSU Controller` + +##### Note: Oxford Instruments ##### +1. Support for OI ITC503 created as part of [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) +1. No longer used on PEARL. + +##### Note: Pace ##### +PACE appear to have been taken over by [Baker Hughes Digital Solutions](https://www.bakerhughesds.com/) - specifically, Baker Hughes [Druck](https://www.bakerhughesds.com/measurement-sensing/druck-pressure-measurement) subsidiary.
+1. [Pace 5000 Pneumatic Modular Pressure Controller](https://www.bakerhughesds.com/measurement-sensing/pressure-measurement-and-calibration/test-and-calibration/pressure-controllers-pace) +1. Documentation: `\\isis\shares\ISIS_Experiment_Controls\PACE5000 Pressure Controller` +1. VI for Pace 5000 is located here: `C:\LabVIEW Modules\Drivers\PACE 5000` + +##### Note: PEARL Pressure Transducer ##### +1. VI for PEARL Pressure Transducer is here: `C:\LabVIEW Modules\Instruments\PEARL\PEARL Pressure Transducer` +1. **MMP 200/210 device is no longer used on PEARL** +1. Superseded by M905 (currently connected to Pressure Cell Controller, but would like to be read independently in other experiments) + +##### Note: Pressure Cell Controller ##### +1. Pressure Controller Repositories +- [Support](https://github.com/ISISComputingGroup/EPICS_PearlPressure) +- [IOC](https://github.com/ISISComputingGroup/EPICS-ioc/tree/master/PEARLPC) +1. VI for PEARL Pressure Cell Controller is here: `C:\LabVIEW Modules\Instruments\PEARL\PEARL Pressure Cell Controller` + +##### Note: PEARL Temperature Controller ##### +1. Based on a [NI cDAQ 9181](http://www.ni.com/en-gb/support/model.cdaq-9181.html) device using 9213 Thermocouple module. +1. VI for PEARL Temperature Controller is here: `C:\LabVIEW Modules\Instruments\PEARL\PEARL Temperature` + +##### Note: Cold Valve ##### +1. The PEARL ColdValve is a motor driven needle valve that controls the flow of liquid nitrogen through their press. The motor is driven by a standard Mclennan drive crate. +1. VI for the ISIS Cold Valve is located here: `C:\LabVIEW Modules\Drivers\ISIS Cold Valve` +1. They current model will (at some point in the future) be replaced by a new one which the Cryogenics team are making. Both the old and new models will use the same controller. +1. For the avoidance of doubt, the PEARL ColdValve is _**entirely unrelated**_ to the IRIS CryoValve. + +##### Note: Camera ##### +PEARL's camera is a blackfly model which we communicate with through the `areaDetector` module. There is a generic "webcam" device screen which needs to be set up for this pointing at the `PEARLCAM_01` IOC which lets them start acquiring image data with region of interest functionality. + +## PEARL Notes ## + +PEARL has the following specialist panels: +1. PEARL Sample Alignment + * This is a method to move the sample into position, using 'nudges' rather than setpoints + * see `C:\LabVIEW Modules\Instruments\PEARL\PEARL Alignment` +1. PEARL Jaws & Beam Dimensions + * PEARL Beam dimensions.vi (controls all jaw sets 1x Galil & 2x LinMot) + +Compare with similar systems on GEM and POLARIS. + +PEARL has the following devices under motion control: +1. Jaws + * see `C:\LabVIEW Modules\Instruments\PEARL\PEARL Jaws\Screens` +1. Rotation stage +1. XYZ stage +1. Diamond Anvil Cell(DAC): uses XYZ stage, rotation stage. All driven by Galil. + +This is a weird `Servostep` box which contains a Galil 4000 series inside. It looks like this: + +![IMG_5182-1](PEARL-ServoStep.jpg) + + +Compare with similar systems on GEM and POLARIS. + +There are some inaccessible front end jaws on PEARL controlled by the Galil, these are not used but must be kept open for operation. They are homed to the forward limit and seem to typically be kept about 4/5mm inside that position for running. This should be kept in mind when migrating the Galil. + +## PEARL SECI Configs ## +Document information about PEARL SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +PEARL_place_holder1.conf | - | dd/mm/yyyy | - | +PEARL_place_holder2.conf | - | dd/mm/yyyy | - | + +## PEARL Genie Scripts ## +Similarly, Document information about PEARL SECI Genie scripts here. diff --git a/doc/processes/instrument_details/PEARL-ServoStep.jpg b/doc/processes/instrument_details/PEARL-ServoStep.jpg new file mode 100644 index 000000000..32293c747 Binary files /dev/null and b/doc/processes/instrument_details/PEARL-ServoStep.jpg differ diff --git a/doc/processes/instrument_details/POLARIS-Instrument-Details.md b/doc/processes/instrument_details/POLARIS-Instrument-Details.md new file mode 100644 index 000000000..eb95f86f7 --- /dev/null +++ b/doc/processes/instrument_details/POLARIS-Instrument-Details.md @@ -0,0 +1,135 @@ +# POLARIS + +This page collects information that will be useful for the implementation of the IBEX control system on POLARIS. +## Background & Timeline ## +POLARIS is a diffractometer instrument at ISIS, on TS1. The [POLARIS](http://www.isis.stfc.ac.uk/instruments/polaris/polaris4643.html) web page describes the background to the instrument. + +## Control System ## +POLARIS will migrate from the SECI control system to the IBEX control system. + +## POLARIS Equipment ## +The equipment listed below is used on POLARIS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#polaris_noteDAE) +ISIS | Mk2 Chopper | Chopper | RS-232 | | [see Chopper note](#polaris_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#polaris_noteGalil) | +[JJ-XRAY](http://www.jjxray.dk/) | | 4 x 4-blade jaws | | | Blades driven by Galils. [see JJ X-Ray note](#polaris_noteJJXray) + | | 1 x 4-blade jaws | | | Blades driven by Galils. Mounted inside sample tank +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | #169 | +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS Vacuum System | RS232 | #216 |[see Pfeiffer note](#polaris_notePfeiffer) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG26x | Gas Handling System | RS232 | #1411 |[see Pfeiffer note](#polaris_notePfeiffer) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#polaris_noteEurotherm) +Chell | CCD100 | Pressure Transducer | RS-232 | #1827 | [see Chell note](#polaris_noteChell) +Keithley | 2400 | Source Meter | RS-232 | #1826 | [see Keithley note](#polaris_noteKeithley) +Oxford Instruments | Orange Cryostat| Cryogenic System | | | [see Oxford Instruments note](#polaris_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | | | [see Oxford Instruments note](#polaris_noteOxfordInstruments) +Oxford Instruments | Dilution Fridge (Kelvinox) | Cryogenic System | | | [see Oxford Instruments note](#polaris_noteOxfordInstruments) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#polaris_noteMcLennan) +ISIS | Eurotherm & Keithley | Resistivity Measurements | RS-232 | | Only an offline experiment to date [see Resistivity note](#polaris_noteResistivity) +ISIS | POLARIS Sample Changer | Sample Changer | RS-232 | | Copy of GEM device +Applied Measurements Ltd | INT2-L | Pressure Transducer | RS-232 | | Used in the Hydrothermal Reaction Cell [see Hydrothermal Reaction Cell note](#polaris_noteHydrothermal) | +ISIS | POLARIS Pressure Cell Controller | Pressure Cell Controller | RS-232 | | [see Pressure Cell Controller note](#polaris_notePressureCell) +Spellman| SL600 | high-voltage power supply | N/A | | [see Spellman note](#polaris_noteSpellman) +ISIS | CS155 centre stick | high voltage centre stick | N/A | | [see HV Centre Stick note](#polaris_noteHVCentreStick) + + +{#polaris_noteDAE} +##### Note: DAE ##### +Main Detector banks + fixed monitors. + +{#polaris_noteChopper} +##### Note: Choppers ##### +One T0 chopper.
+Chopper is ISIS Mk2 chopper. Mk2 choppers have a serial interface (not Ethernet like Mk3). + +{#polaris_noteJJXray} +##### Note: JJ X-Ray ##### +[JJ X-Ray 4-blade jaws](http://www.jjxray.dk/products/jj-x-ray-slit-systems/neutron-slits)
+All blades are driven by Galils.
+POLARIS has two screens for managing all 5 sets of jaws: + +1. One screen to view & edit settings for jaw sets 1 -4 and display settings for jaw set 5 +1. A separate screen (Jaws Manager) to view & edit settings for all 5 jaw sets + +{#polaris_notePfeiffer} +##### Note: Pfeiffer ##### + +1. [Model TPG 26x](https://www.pfeiffer-vacuum.com/en/products/measurement/activeline/controllers/?detailPdoId=5724) +2. [Model TPG 300](https://www.pfeiffer-vacuum.com/en/products/measurement/modulline/controllers/?detailPdoId=3407) + +{#polaris_noteChell} +##### Note: Chell ##### +Experiments requiring pressure transducer. [Chell CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100) + +{#polaris_notePressureCell} +##### Note: Pressure Cell Controller ##### +The pressure cell controller on POLARIS is a copy of the same device on PEARL. At the current time (April 2017), it has not been used on POLARIS for some time. However, it may be used at some future point (but probably not for another 12-18 months, by which time we should have implemented support for the pressure cell controller on PEARL). + +{#polaris_noteKeithley} +##### Note: Keithley ##### +Experiments requiring source meter. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter) + +{#polaris_noteEurotherm} +##### Note: Eurotherm ##### +Used to control temperature Orange Cryostat devices. + +{#polaris_noteOxfordInstruments} +##### Note: Oxford Instruments ##### + +1. Orange Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. Orange cryostats are controlled via a temperature controller (e.g. Eurotherm), so nothing on the cryostat itself for IBEX to control. +1. Blue Cryostat: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/cryostats/cryostats8445.html) to be determined. + 1. Are these the Heliox and ITC devices? + 1. The Heliox device has a sorption insert (is that relevant to the control system?) +1. Dilution fridge: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/dilution-refrigerators8825.html) to be determined. + 1. [Kelvinox dilution fridge](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/kelvinox-dilution-fridge/kelvinox-dilution-fridge13981.html) + +{#polaris_noteMcLennan} +##### Note: McLennan ##### +1. One McLennan is used raise/lower the sample-stick +2. Another single-axis McLennan is used to rotate the sample-stick + +{#polaris_noteGalil} +##### Note: Galil ##### +GALIL_02 is currently set up to use QR instead of DR mode because DR mode stopped working. This is the same issue as documented in Ticket #2191. + +{#polaris_noteHydrothermal} +##### Note: Hydrothermal Cell ##### +Uses the INT2-L pressure transducer. + +{#polaris_noteResistivity} +##### Note: Resistivity Measurements ##### +An evacuated furnace cell for measuring resistivity. Controlled by Eurotherms and measured by Keithleys (models 2000, 2400?) + +{#polaris_noteSpellman} +##### Note: Spellman High Voltage Power Supply ##### +6kV, 100mA, 600W, DC power supply. Not possible to control this device via software. + +{#polaris_noteHVCentreStick} +##### Note: CS155 Centre Stick ##### +Not possible to control this device via software. It connects to the [Spellman](#polaris_noteSpellman) HV-PS. + +## POLARIS SECI Configs ## +Document information about POLARIS SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +----------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|---------------|----------| + POLARIS Base + Hydrothermal | Chopper, Vacuum, Jaws, Eurotherms Test | 03/02/2017 | - + POLARIS Base + Dilution Fridge | Chopper, Vacuum, Jaws, Oxford - Dilution Fridge | 15/12/2016 | - + POLARIS Base + Sample Changer + Eurotherms_matt | Chopper, Vacuum, Jaws, Eurotherms Test, Sample Changer | 12/12/2016 | - + POLARIS Base + Sample Changer + Eurotherms + Heliox | Chopper, Vacuum, Jaws, Eurotherms Test, Sample Changer, Heliox | 19/09/2016 | - + POLARIS Base | Chopper, Vacuum, Jaws | 30/11/2015 | - + POLARIS Base + Sample Changer + Eurotherms | Chopper, Vacuum, Jaws, Eurotherms Test, Sample Changer | 18/11/2015 | - + POLARIS Base + Sample Changer + Eurotherms + ITC_matt | Chopper, Vacuum, Jaws, Eurotherms Test, Sample Changer | 25/06/2015 | - + POLARIS Base + Eurotherms | Chopper, Vacuum, Jaws, Eurotherms Test | 05/06/2014 | - + POLARIS Base + Keithley (Voltage) + Eurotherms | Chopper, Vacuum, Jaws, Eurotherms Test, Keithley (Voltage) | 02/06/2014 | - + POLARIS Base + Keithley + Eurotherms | Chopper, Vacuum, Jaws, Eurotherms Test, Keithley | 25/03/2013 | - + delete | - | 01/10/2012 | - + POLARIS Base + Sample Changer | Chopper, Vacuum, Jaws, Sample Changer | 14/03/2012 | - + test | - | 10/11/2011 | - + +## POLARIS Genie Scripts ## +Similarly, document information about SECI Genie scripts currently used on POLARIS here. + +## POLARIS Notes ## \ No newline at end of file diff --git a/doc/processes/instrument_details/POLREF-Instrument-Details.md b/doc/processes/instrument_details/POLREF-Instrument-Details.md new file mode 100644 index 000000000..596171406 --- /dev/null +++ b/doc/processes/instrument_details/POLREF-Instrument-Details.md @@ -0,0 +1,163 @@ +# POLREF + +This page collects information that will be useful for the implementation of the IBEX control system on POLREF. + +## Background & Timeline ## +POLREF is a general purpose polarised neutron reflectometer, on TS2. The [POLREF](https://www.isis.stfc.ac.uk/Pages/polref.aspx) web page describes the background to the instrument. + +## Control System ## +POLREF will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## POLREF Equipment ## +The equipment listed below is used on POLREF. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#note-choppers) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | Be careful of homing routines | +Huber| | Sample Stack | Ethernet | | | [see Huber note](#note-huber) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS] | [see McLennan note](#note-mclennan) +Newport | SMC100 | Motion Controller | RS-232 | [EPICS] | [see Newport note](#note-newport) +??? | | 4-blade jaws | | | [see Jaws note](#note-jaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +Julabo | FP-50 | Water Bath | RS-232 | | [see Julabo note](#note-water-baths) +Julabo | FP-52 | Water Bath | RS-232 | | [see Julabo note](#note-water-baths) +~Haake~ | ~N6~ | ~Water Bath~ | ~RS-232~ | | [see Haake note](#note-water-baths) +Keithley | 2400 | Source Meter | RS-232 | | [see Keithley note](#note-keithley) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#note-keithley) +[Knauer](http://www.knauer.net/) | K6 | Electric valve drive | RS232 | |[see Knauer note](#note-knauer) +[Knauer](http://www.knauer.net/) | 1050 | HPLC pump | ??? | | [see Knauer HPLC note](#note-knauer-hplc) +~Hitachi~ | ~L-7100~ | ~HPLC pump~ | ~???~ | | [see Hitachi note](#note-hitachi) +JASCO | PU-4180 | HPLC pump| ??? | | [see JASCO HPLC note](#note-jasco-hplc-pump) +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 100-10MG | Bi-Polar Power Supply | RS232 | #187 |[see Kepco note](#note-kepco) +[KEPCO](http://www.kepcopower.com/bop.htm) | BIT 4886 | I/F card | GPIB, RS-232 | |[see Kepco note](#note-kepco) +LakeShore | 327 | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#note-lakeshore) +LakeShore | 460 | Gaussmeter | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#note-lakeshore) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#note-neocera) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Mercury | Temperature Controller | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | | Cryogenic Equipment | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Scientific Magnetics | 17T Cryomag | Super conducting magnet | RS-232 | 3 Axis Super conducting magnet & VTI | see [Scientific Magnetics note](#note-scientific-magnetics) +WPI | Aladdin-1000 | Syringe Pump | ??? | | [see Syringe Pumps note](#note-syringe-pumps) + +##### Note: DAE ##### +See multi-detector and single-detector below. + +##### Note: Choppers ##### +POLREF has a Mk3 chopper.
+ +##### Note: Jaws ##### +Provide information about POLREF jaws. + +##### Note: Huber ##### +1. [Huber](http://www.xhuber.de/en/home/). +1. According to Huber's web-site, a Huber SMC9000 is a motion controller. See [SMC 9300](http://www.xhuber.de/en/product-groups/3-control-systems/smc-9300/) which appears to be the latest of the 9000-series and [SMC 9000](http://www.xhuber.de/fileadmin/user_upload/downloads/usermanuals/9000_1103.pdf) + * However, in terms of POLREF, Huber seems to refer to a sample stack. Need to get to the bottom of this confusion. I guess it is a Huber sample stack (how many axes?) controlled by a SMC 9000 controller. + +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +##### Note: Newport ##### +1. [Newport SMC100](https://www.newport.com/f/smc100-single-axis-dc-or-stepper-motion-controller) +1. EPICS drivers for [Newport devices](https://epics.anl.gov/modules/manufacturer.php#Newport) +1. See also [#824](https://github.com/ISISComputingGroup/IBEX/issues/824) + +{#noteVacuum} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/POLREF/POLREF_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +##### Note: Water Baths ##### +1. Julabo Water Bath (apparently the following Julabos: WB23 ) + * consult the [Julabo commands](/specific_iocs/temp_controllers/julabo_commands.xlsx) spreadsheet for specific details of which set of commands is used by these water baths. +1. Haake Water Bath (no longer supported at ISIS) + 1. ~Haake N6 Water Bath. Haake has now been taken over by [ThermoFisher](https://www.thermofisher.com/uk/en/home/life-science/lab-equipment/water-baths-circulators-chillers.html). N6 model may be obsolete. Check existing SECI VI for logic and manual~. +1. ~See also [Water Baths](https://www.isis.stfc.ac.uk/Pages/Water-Baths.aspx) list~ + +##### Note: KEPCO ##### +1. [Kepco BOP 100-10MG PSU](http://www.kepcopower.com/bophimod.htm). + * See also [#3005](https://github.com/ISISComputingGroup/IBEX/issues/3005) +1. [Kepco BIT 4886](http://www.kepcopower.com/bit.htm) is an interface card for Kepco bi-polar ([BOP](http://www.kepcopower.com/bop.htm)) power supplies. + +##### Note: Keithley ##### +1. [Keithley 2400 Series Source Meter](https://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter).
+See also tickets [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#2695](https://github.com/ISISComputingGroup/IBEX/issues/2695), [#2801](https://github.com/ISISComputingGroup/IBEX/issues/2801) and [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176). + +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs (may be obsolete). Check existing SECI VI for logic and manual.~ + +##### Note: JASCO HPLC Pump ##### +[JASCO HPLC pump](https://jascoinc.com/products/chromatography/hplc/modules/hplc-pumps/) is a new (for ISIS) model of HPLC pump. Jasco pumps are replacing the old Hitachi pumps. + * See `C:\LabVIEW Modules\Drivers\Jasco PU-4180 HPLC Pump\Documentation` for documentation. + * See also [#3743](https://github.com/ISISComputingGroup/IBEX/issues/3743) and [#3923](https://github.com/ISISComputingGroup/IBEX/issues/3923) + +##### Note: Knauer ##### +1. Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+ * K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. + +##### Note: Knauer HPLC ##### +Knauer HPLC 1050 [Knauer HPLC 1050 is discontinued.](https://www.knauer.net/en/discontinued-smartline-pump-1050-successor-azura-p-61l/p14161). Check existing SECI VI for logic and manual. + +##### Note: LakeShore ##### +1. [Model 327](https://www.lakeshore.com/products/Cryogenic-Temperature-Controllers/Pages/default.aspx). Model 327 is not listed. Is it an obsolete model, or should it be Model-372? +1. [Model 460](https://www.lakeshore.com/products/Gaussmeters/Model-460-3-Channel-Gaussmeter/Pages/Overview.aspx). + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) for a variety of Lakeshore temperature controllers. + +##### Note: Neocera ##### +Low temperature experiments?[Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +##### Note: Oxford Instruments ##### +1. Support for OI ITC503 created as part of [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) +1. Support for Mercury Temperature Controller was created as part of [#2840](https://github.com/ISISComputingGroup/IBEX/issues/2840) +1. Dilution fridge: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx) to be determined. + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) + +##### Note: Scientific Magnetics ##### +1. [Scientific Magnetics](http://www.scientificmagnetics.co.uk/) +1. Model: 3 Axis Super conducting magnet & VTI +1. Implemented via LabVIEW, lvDCOM - see [#1398](https://github.com/ISISComputingGroup/IBEX/issues/1398)). + +##### Note: Syringe Pumps ##### +1. [WPI Aladdin-1000 Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps/al1000-220.aspx) + +## POLREF Notes ## +POLREF has the following specialist panels/systems: +1. POLREF Danfysik +1. POLREF Flipper +1. POLREF Vector Network Analyser +1. SURF Galil DIO + +POLREF has the following devices under motion control: +1. benches +1. coarse jaws +1. FOMs (Frame Overlap Mirror) +1. Huber +1. jaws +1. laser gimbal +1. point detector +1. Polarizer +1. reflectometer sample stack +1. rotation stage +1. transmission monitor + +**Note:** (from original POLREF notes (may now be obsolete - check with Kathryn)) +Note for when the time comes - check with the instrument scientist if the calibration function for the Danfysik is still used (allows for specific hysteresis curves to use between the limits in use), if it is, make sure we capture this functionality again - likely to be some SNL or a genie-python script might be able to do it instead. + +## POLREF SECI Configs ## +Document information about POLREF SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +POLREF_place_holder1.conf | - | dd/mm/yyyy | - | +POLREF_place_holder2.conf | - | dd/mm/yyyy | - | + +## POLREF Genie Scripts ## +Similarly, Document information about POLREF SECI Genie scripts here. diff --git a/doc/processes/instrument_details/RIKEN-Front-End-Instrument-Details.md b/doc/processes/instrument_details/RIKEN-Front-End-Instrument-Details.md new file mode 100644 index 000000000..02a690844 --- /dev/null +++ b/doc/processes/instrument_details/RIKEN-Front-End-Instrument-Details.md @@ -0,0 +1,38 @@ +# RIKENFE + +This page contains information pertinent to the automation of the RIKEN Front End Beamline Control at ISIS. + +## Background ## +ISIS have taken over management of the RIKEN facility at ISIS. Various systems on the RIKEN facility are being upgraded over time. + +## Control System ## +Control of the RIKEN Front End hardware will be migrated to IBEX in a series of phases. Therefore, there will be a period of mixed operation, in which some devices will be controlled via the existing RIKEN control system and others are controlled by IBEX. + +## Timeline ## +| Task/Event | Date | Notes | +| --- | --- | --- | +| PSU control | Summer 2018 | Upgrade PSU control. See [#2813](https://github.com/ISISComputingGroup/IBEX/issues/2813), [#3150](https://github.com/ISISComputingGroup/IBEX/issues/3150). | +| ARGUS Magnets | Summer 2018 | Changes to ARGUS magnets (no further details). | +| Replace PLC | Easter 2019 | Preferred replacement PLC is a [Schneider Modicon M580 - ePac](https://www.schneider-electric.co.uk/en/product-range/62098-modicon-m580---epac). | +| Mitsubishi PSU | No date yet | Part of cryogenics system. May be controlled via its own dedicated PC. | + +## Equipment ## +The table below captures what the Experiment Controls Group knows about the RIKEN Front End hardware (as of September 2018). + +| Manufacturer | Model | Type | Connection | Driver | Notes | +| --- | --- | --- | --- | --- | --- | +| Danfysik | XXXX | PSU | RS232 | DFKPS | [see PSUs note](#note-psus) | +| GEC | XXXX | PSU | RS232 | ??? | [see PSUs note](#note-psus) | +| Schneider | ???? | PLC | RS232 | ModbusRTU | PLC is, or is similar to, a [Schneider Electric Quantum PLC](https://www.schneider-electric.co.uk/en/product-range-download/538-modicon-quantum). | +| | | Kicker | ??? | | Currently not functional. No immediate plans to make it so. | +| | | Separator | ??? | | Separator PSUs are due to be replaced (no timescale yet) | +| Pfeiffer | TPG300 | Vacuum Monitor | RS232 | TPG300 | There are ~7 of these. Assumed to have RS232 cards. | +| Pfeiffer | TCP350 | Turbo Pump | TCP/IP | TCP350 | | +| National Instruments| DAQmx| Data Acquisition | TCP/IP | | Used to communicate with PLC & PSUs | + +##### Note: PSUs ##### +1. The RIKEN PSUs are physically a mixture of Danfysik, GEC, and Japanese power supplies. However, they are all fitted with Danfysik control boards. So as far as IBEX is concerned they are all Danfysik devices. +1. The Danfysik control boards talk a slightly different protocol than the other Danfysiks (as used on Muon-FE, EMU, LOQ and other instruments). For this reason, they require their own IOC. All of the power supplies on RIKEN-FE are controlled by a single IOC. +1. We have no record of the actual Danfysik model number to which these boards correspond (it is quite possible that they don't correspond to any standard Danfysik), so we have no Danfysik manuals for these particular boards. The best we information we have is the documents stored in the `RIKEN_power_supplies` sub-folder in the Experiment Controls Group's Manuals share. + + diff --git a/doc/processes/instrument_details/SANDALS-Instrument-Details.md b/doc/processes/instrument_details/SANDALS-Instrument-Details.md new file mode 100644 index 000000000..c374ac136 --- /dev/null +++ b/doc/processes/instrument_details/SANDALS-Instrument-Details.md @@ -0,0 +1,123 @@ +# SANDALS + +This page collects information that will be useful for the implementation of the IBEX control system on SANDALS. +## Background & Timeline ## +SANDALS is a diffractometer instrument at ISIS, on TS1. The [SANDALS](https://www.isis.stfc.ac.uk/Pages/sandals.aspx) web page describes the background to the instrument. + +## Control System ## +SANDALS will migrate from the SECI control system to the IBEX control system in January 2018. + +## SANDALS Equipment ## +The equipment listed below is used on SANDALS. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#sandals_noteDAE) +ISIS | N/A | Chopper | N/A | | [see Chopper note](#sandals_noteChopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +??? | | 2-axis jaws | | | [see Jaws note](#sandals_noteJaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#sandals_noteVacuum) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#sandals_noteVacuum) +Pfeiffer | TPG26x | Vacuum Gauge | RS-232 | EPICS | [see Vacuum Pump note](#sandals_noteVacuumPump) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#sandals_noteEurotherm) +Oxford Instruments | Orange Cryostat| Cryogenic System | | | [see Oxford Instruments note](#sandals_noteOxfordInstruments) +Oxford Instruments | Blue Cryostat| Cryogenic System | | | [see Oxford Instruments note](#sandals_noteOxfordInstruments) + | | Closed Cycle Refrigerator | | | | [see CCR note](#sandals_noteCCR) +ISIS | Furnace | Furnace | | | [see ISIS Furnaces note](#sandals_noteISISFurnaces) +ISIS | SANDALS Sample Changer | Sample Changer | RS-232 | | [see SANDALS Sample Changer note](#sandals_noteSampleChanger) +Julabo | FP-50 | Water Bath | RS-232 | | [see Water Bath note](#sandals_noteWaterBath) +Julabo | FP-52 | Water Bath | RS-232 | | [see Water Bath note](#sandals_noteWaterBath) +~Haake~ | ~???~ | ~Water Bath~ | ~RS-232~ | | [see Water Bath note](#sandals_noteWaterBath) +Chell | CCD100 | Pressure Transducer | RS-232 | | [see Chell note](#sandals_noteChell) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#sandals_noteKeithley) + +{#sandals_noteDAE} +##### Note: DAE ##### +Main Detector banks + fixed monitors. + +{#sandals_noteChopper} +##### Note: Choppers ##### +SANDALS has no choppers.
+A T0 chopper _might_ be installed on SANDALS at some point in the distant future (but there are no definite plans, so for all practical purposes there are no choppers on SANDALS).
+_**Note:**_ NIMROD does not have choppers either. + +**Note:** There is a proposal to fit a chopper to SANDALS in February 2020. The chopper would be a SKF G5 chopper (as used on IMAT and LET - see also [#1907](https://github.com/ISISComputingGroup/IBEX/issues/1907). + +{#sandals_noteJaws} +##### Note: Jaws ##### +SANDALS has a single set of jaws controlled by a [Parker controller](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_Parker_Controller.jpg), which is not currently controlled by SECI. There is no requirement to support the Parker system in IBEX, although it would be very convenient if such a thing were possible. + +It is proposed to replace the Parker controller with a Galil DMC4040 controller (see [#3259](https://github.com/ISISComputingGroup/IBEX/issues/3259)). + +{#sandals_noteVacuum} +##### Note: Vacuum System ##### +The vacuum pressure on SANDALS is the pressure in the SANDALS tank/chamber itself. SANDALS only needs to monitor the vacuum pressure; scientists do not want to control the vacuum system from IBEX. The vacuum pressure is also displayed on [a gauge mounted on the "fence"](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_fence.jpg) that surrounds SANDALS. +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#sandals_noteVacuumPump} +##### Note: Vacuum Pump ##### +SANDALS uses a [vacuum pump](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_Turbo_Pump.jpg) (in conjunction with CCRs). A TPG26x is used to measure the pressure. +TPG26x support was originally implemented via [#1411](https://github.com/ISISComputingGroup/IBEX/issues/1411), [#2379](https://github.com/ISISComputingGroup/IBEX/issues/2379) and [#2578](https://github.com/ISISComputingGroup/IBEX/issues/2578) + +{#sandals_notePLC} +##### Note: Gate Valve PLC ##### +SANDALS has an [Omron PLC](/specific_iocs/plcs/Omron-FINS) to control two gate valves, one of which has not been set up yet. `GV2` is currently in use and has been set up in SANDALS' base component. The PV for the V1 valve already exists so is ready to create blocks for when it is set up correctly, but currently nothing is wired up on the PLC end. We can still read the status for it. + +{#sandals_noteEurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#sandals_noteOxfordInstruments} +##### Note: Oxford Instruments ##### + +1. Orange Cryostat: [models](https://www.isis.stfc.ac.uk/Pages/Low-Temperature-Cryostats.aspx) to be determined. Orange cryostats are controlled via a temperature controller (e.g. Eurotherm), so nothing on the cryostat itself for IBEX to control. + 1. Orange cryostat is used occasionally on SANDALS. +1. Blue Cryostat: [models](https://www.isis.stfc.ac.uk/Pages/Low-Temperature-Cryostats.aspx) to be determined. + 1. Blue cryostat is very rarely used on SANDALS. + 1. Are these the Heliox and ITC devices? + 1. The Heliox device has a sorption insert (is that relevant to the control system?) +1. SANDALS does not use Dilution fridges. + +{#sandals_noteCCR} +##### Note: Closed Cycle Refrigerators ##### + 1. [CCR](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_CCR.jpg) is the most frequently used low-temperature device used in on SANDALS. +[CCRs](https://www.isis.stfc.ac.uk/Pages/Closed-Cycle-refrigerators,-CCRs.aspx). Not directly computer-controlled - controlled via Eurotherm. + +{#sandals_noteISISFurnaces} +##### Note: ISIS Furnaces ##### +More information on [IRIS Furnaces](https://www.isis.stfc.ac.uk/Pages/High-temperature.aspx). Furnaces are controlled via a temperature controller (e.g. Eurotherm), so nothing on the furnace itself for IBEX to control. + +{#sandals_noteSampleChanger} +##### Note: SANDALS Sample Changer ##### +The [SANDALS sample changer](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_Sample_Changer_bottom.jpg) can hold up to 15 samples. It has recently been upgraded to use a [Beckhoff PLC](/specific_iocs/motors/Beckhoff) which controls the jaws as well. + +{#sandals_noteWaterBath} +##### Note: Water Bath ##### +SANDALS uses two types of water bath, both Julabo models: FP-50 and [FP-52](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANDALS/SANDALS_Julabo_FP52.jpg). +See [supported Julabo models](/specific_iocs/temp_controllers/Julabo) for more details. +SANDALS uses the FP-52 model of Julabo most frequently (used in conjunction with the [Sample Changer](#sandals_noteSampleChanger)). Sometimes SANDALS uses water as the coolant/heating medium; sometimes it uses oil or glycol as the coolant/heating medium (Julabo/Presto A40 device). SANDALS has also used Haake water bath in the past, but none of the current instrument scientists know anything about this (so it might have been a one-off). + +{#sandals_noteChell} +##### Note: Chell ##### +Experiments requiring pressure transducer. [Chell CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100). See also [#1827](https://github.com/ISISComputingGroup/IBEX/issues/1827). +**Note:** also ask about Gas Panel/Baratron + +{#sandals_noteKeithley} +##### Note: Keithley ##### +Experiments requiring source meter. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter).
+See also [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826). + +## SANDALS SECI Configs ## +Document information about SANDALS SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +SANDALS_place_holder1.conf | - | dd/mm/yyyy | - | +SANDALS_place_holder2.conf | - | dd/mm/yyyy | - | + +## SANDALS Genie Scripts ## +Similarly, Document information about SANDALS SECI Genie scripts here. + +## SANDALS Notes ## +Add any notes about special items of equipment, setup or conditions on SANDALS that might impact the deployment and configuration of IBEX. + diff --git a/doc/processes/instrument_details/SANS2D-Instrument-Details.md b/doc/processes/instrument_details/SANS2D-Instrument-Details.md new file mode 100644 index 000000000..554a24972 --- /dev/null +++ b/doc/processes/instrument_details/SANS2D-Instrument-Details.md @@ -0,0 +1,305 @@ +# SANS2D + +This page collects information that will be useful for the implementation of the IBEX control system on SANS2D. +## Background & Timeline ## +SANS2D is a Time-of-Flight Small-Angle Neutron Scattering instrument, on TS2. The [SANS2D](https://www.isis.stfc.ac.uk/Pages/SANS2D.aspx) web page describes the background to the instrument. Information about the [construction of SANS2D](https://www.facilities.rl.ac.uk/isis/projects/ts2/ts2construction/Instruments/Forms/AllItems.aspx?Paged=TRUE&PagedPrev=TRUE&p_SortBehavior=0&p_FileLeafRef=SaP%20warrant%203%20%28sans2d%20e2-e3%20platform%29%2edoc&p_ID=1829&RootFolder=%2fisis%2fprojects%2fts2%2fts2construction%2fInstruments%2fSANS2D&PageFirstRow=181&&View={776499AB-B00F-4B98-9619-3125F112A0C8}), which is useful background for understanding the operation of the instrument, is also available. + +Images of SANS2D equipment can be found here [SANS2D equipment](https://github.com/ISISComputingGroup/IBEX/wiki/SANS2D-equipment-Images-GIF). + +The SANS2D instrument scientists have provided a description of the [instrument beamline](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANS2D/Beam_line_descriptors(Sarah%20and%20Najet).docx) and some [schematic drawings](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANS2D/Sans2d_drawing_for%20_IBEX(Sarah%20and%20Najet).pptx). + +## Mini-IBEX on SANS2D ## +This section contains relating to the mini-IBEX server on SANS2D. This information can be deleted once SANS2D has migrated to a full IBEX server. + +SANS2D is a SECI instrument, but uses the IBEX interactions for dealing with the CAEN HV. +As such, it has some setup which is not automated, but is needed to function correctly. + +If NDXSANS2D is restarted, then, you will need to run start_inst (the version of IBEX is still currently that old, as the CAEN hasn't changed in that time). This should run the reduced version of the start_inst code, but it does kill SECI as well as everything else. After this SECI can be restarted. + +If the HVCAEN_01 IOC doesn't automatically start, then either use the console to connect and start the IOC, or use the reset comms button on the CAEN VI which will start the IOC via the console in the background. + + +## Control System ## +SANS2D will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## SANS2D Equipment ## +The equipment listed below is used on SANS2D. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +See also [SANS2D Migration Notes](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANS2D/SANS2D%20Migration%20to%20IBEX%20Notes.docx) + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#note-chopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +ISIS | | Aperture | N/A | | [see Aperture note](#note-aperture-plates) +ISIS | | Guide | N/A | | [see Guide note](#note-guides) +ISIS | | Gate Valve | N/A | | [see Gate Valve note](#note-gate-valve) +ISIS | | Snout Valve | N/A | | [see Snout Valve note](#note-snout-valve) +Huber| | Sample Stack | Ethernet | | [see Huber note](#note-huber) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS] | [see McLennan note](#note-mclennan) +Newport | SMC100 | Motion Controller | RS-232 | [EPICS] | [see Newport note](#note-newport) +ISIS | "big" jaws | 4-blade jaws | | | [see Jaws note](#note-jaws) +JJ-XRay| "little" jaws | 4-blade jaws | | | [see Jaws note](#note-jaws) +FINS | ??? | PLC | | | [see Vacuum System note](#note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +Julabo | FPW55-SL | Water Bath | RS-232 | | [see Water Baths note](#note-water-baths) +Anton Paar | Viscotherm VT2 | Water Bath | RS-232 | | [see Water Baths note](#note-water-baths) +[Knauer](http://www.knauer.net/) | K6 | Electric valve drive | RS232 | |[see Knauer note](#note-knauer) +[Knauer](http://www.knauer.net/) | 1050 | HPLC pump | ??? | | [see Knauer HPLC note](#note-knauer-hplc) +~Hitachi~ | ~L-7100~ | ~HPLC pump~ | ~???~ | | [see Hitachi note](#note-hitachi) +[Jasco](https://jascoinc.com/products/chromatography/)| LC-4000 | HPLC pump| ??? | | [see Jasco note](#note-jasco) +[KEPCO](http://www.kepcopower.com/bop.htm) | BOP 100-10MG | Bi-Polar Power Supply | RS232 | #187 |[see Kepco note](#note-kepco) +[KEPCO](http://www.kepcopower.com/bop.htm) | BIT 4886 | I/F card | GPIB, RS-232 | |[see Kepco note](#note-kepco) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#note-thurlby) +Keithley | 2400 | Source Meter | RS-232 | #1826 | [see Keithley note](#note-keithley) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#note-neocera) +[Linkam](http://www.linkam.co.uk/) | T95 | | RS232 | |[see Linkam note](#note-linkam) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | Mercury | Temperature Controller | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Oxford Instruments | | Cryogenic Equipment | RS-232 | | [see Oxford Instruments note](#note-oxford-instruments) +Birmingham | 17T Magnet | Super conducting magnet | RS-232 | | [see Birmingham Magnet note](#note-birmingham-magnet) +Goudsmit | 2T Magnet | Electromagnet | RS-232 | | [see Goudsmit Magnet note](#note-goudsmit-magnet) +~Omega~ | ~iBTHX~ | ~Transmitter~ | ~Ethernet~ | | [see Omega note](#note-omega) +Superlogics| 8018R | Thermocouple | RS-485 | | [see Superlogics note](#note-superlogics) +Superlogics| 8520 | Converter | RS-232| | [see Superlogics note](#note-superlogics) +~ORDELA~ | ~2100N~ | ~Ordela Detector~ | | | [see Ordela Detector note](#note-ordela-detector) +ISIS | LOQ | Couette Cell | | | [see Couette Cell note](#note-couette-cell) +ISIS | LOQ | T-Jump Cell | | | [see T-Jump Cell note](#note-t-jump-cell) +WPI | Aladdin-1000 | Syringe Pump | ??? | | [see Syringe Pumps note](#note-syringe-pumps) +ISIS | | Detector Trolleys | | | [see Detector Trolleys note](#note-detector-trolleys) +ISIS | | Baffle Trolleys | | | [see Baffle Trolleys note](#note-baffle-trolleys) +ISIS | | Beamstops | | | [see Beamstops note](#note-beamstops) + +##### Note: DAE ##### +See multi-detector and single-detector below. + +##### Note: Chopper ##### +SANS2D has a Mk3 double-disk chopper.
+ +##### Note: Aperture Plates ##### +SANS2D has 5 moveable aperture plates: +1. All plates are controlled by Galils. +1. Plate positions (UP, CENTRE, DOWN) are defined as motion set-points. +1. Plates can also be driven to any position in their range (but this is not usual). Plates are sometimes scanned, to check the motion set-points are still valid. +1. Plates are homed by driving the plate to a top limit switch, then driving down to a home switch. +1. Motors must remain energised after moving to prevent plates from falling under their own gravity. +1. Plates **MUST NOT** be moved when the vacuum on (there is an inhibit signal from the vacuum PLC to prevent motion). +1. There is a 6th aperture plate (between moveable plates 1 and 2), but its position is fixed. + +##### Note: Guides ##### +SANS2D has 5 moveable guides. Each guide contains two channels - a collimation channel and a guide (super-mirror) channel: +1. All guides are driven by a [Beckhoff](/specific_iocs/motors/Beckhoff) which in turn drives Stober drives. + * The guides are very heavy (> 3 tonnes) - that's why they need Stober drives to move them. +1. Guide positions are defined as motion set-points. +1. Guides can also be driven to any position in their range (but this is not usual). Plates are sometimes scanned, to check the motion set-points are still valid. +1. Guides are homed by driving to a low limit switch, then driving up to a home switch. +1. Guides **MUST NOT** be moved when the vacuum is on (there is an inhibit signal from the vacuum PLC, forwarded to the Beckhoff to prevent motion). + * When the permit is re-established, there is a monitor in `custom_records.db` which forwards the PLC value to each of the axis enabled controls. + +##### Note: Gate Valve ##### +SANS2D has a gate valve. Its purpose is to allow the scientists to isolate the apertures/guide section of SANS2D from the "snout" section. +1. The gate valve can only be controlled manually. There is no requirement for it to be controlled by IBEX. + +##### Note: Snout Valve ##### +The snout valve (also known as the V8 valve) is used to vent the snout section of SANS2D (once it has been isolated from the apertures/guide section by the gate valve). +1. The snout valve can only be controlled manually. However, it is monitored by the PLC; IBEX should display the monitored values to the user. + +##### Note: Scraper Aperture ##### +SANS2D has a scraper aperture. +1. Driven by a Galil. +1. Positions are defined as motion set-points. +1. Scraper can also be driven to any position in its range (but this is not usual). Scraper is sometimes scanned, to check the motion set-points are still valid. + +##### Note: Jaws ##### +SANS2D has two sets of jaws: +1. One set of "small" or "little" JJ-XRay jaws, controlled by Galil. These are only used for GISANS experiments. When not in use they are not installed on the instrument. +2. One set of "large" or "big" jaws, which are located inside the tank. Also controlled by Galil. + * The "zero-position" of the big jaws is set to be on the centre line of the vacuum tank. However, because the detectors on SANS2D are normally offset to the left (looking towards the rear of the tank (i.e. in the direction that neutrons travel)), the big jaws are normally set to the left of the centre line of the tank. + +##### Note: Huber ##### +1. [Huber](http://www.xhuber.de/en/home/). +1. According to Huber's web-site, a Huber SMC9000 is a motion controller. + * However, at ISIS, the Huber is controlled via a Galil. See [#3502](https://github.com/ISISComputingGroup/IBEX/issues/3502), especially for the note about amplifier cards. + * See [SMC 9300](http://www.xhuber.de/en/product-groups/3-control-systems/smc-9300/) which appears to be the latest of the 9000-series and [SMC 9000](http://www.xhuber.de/fileadmin/user_upload/downloads/usermanuals/9000_1103.pdf). +1. SANS2D uses the Huber mini sample stack, which has its own controller (not Galil). + +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +##### Note: Newport ##### +1. [Newport SMC100](https://www.newport.com/f/smc100-single-axis-dc-or-stepper-motion-controller) +1. EPICS drivers for [Newport devices](https://epics.anl.gov/modules/manufacturer.php#Newport) +1. See also [#824](https://github.com/ISISComputingGroup/IBEX/issues/824) + +##### Note: Vacuum System ##### +The vacuum system on SANS2D is controlled by a FINS PLC. IBEX does not control the PLC, but it should display information from the PLC. + +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SANS2D/SANS2D_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +##### Note: Water Baths ##### +1. Julabo Water Bath (apparently the following Julabos: WB15, WB16 ) + * consult the [Julabo commands](/specific_iocs/temp_controllers/julabo_commands.xlsx) spreadsheet for specific details of which set of commands is used by these water baths. +1. Anton Paar Viscotherm VT2 (apparently the following: WB28). It appears the Viscotherm VT2 is no longer manufactured. +1. See also [Water Baths](https://www.isis.stfc.ac.uk/Pages/Water-Baths.aspx) list + +##### Note: KEPCO ##### +1. [Kepco BOP 100-10MG PSU](http://www.kepcopower.com/bophimod.htm).
See also [#3005](https://github.com/ISISComputingGroup/IBEX/issues/3005) + * LabVIEW VI is here: `C:\LabVIEW Modules\Drivers\KEPCO 100-10MG` +1. [Kepco BIT 4886](http://www.kepcopower.com/bit.htm) is an interface card for Kepco bi-polar ([BOP](http://www.kepcopower.com/bop.htm)) power supplies. + * LabVIEW VI is here: `C:\LabVIEW Modules\Drivers\Kepco 4886 Serial` + +##### Note: Thurlby ##### +[Thurlby Thandar Instruments](https://www.aimtti.com/) +1. [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + * see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). +1. See also [#3784](https://github.com/ISISComputingGroup/IBEX/issues/3784) + +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs (may be obsolete). Check existing SECI VI for logic and manual.~ + * Update (27-06-2019): Hitachi pumps are being retired. No longer any need to support them. See [#3780](https://github.com/ISISComputingGroup/IBEX/issues/3780) + * SANS2D will not be using Hitachi pumps in future. They will use the [Knauer](#note-knauer-hplc) or Jasco HPLC pumps. + +##### Note: Knauer ##### +1. Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+ * K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. +1. See also [#3781](https://github.com/ISISComputingGroup/IBEX/issues/3781) + +##### Note: Knauer HPLC ##### +Knauer HPLC 1050 [Knauer HPLC 1050 is discontinued.](https://www.knauer.net/en/discontinued-smartline-pump-1050-successor-azura-p-61l/p14161). +1. Check existing SECI VI for logic and manual. +1. Support for Knauer HPLC 1050 is implemented via the following tickets: +[#3262](https://github.com/ISISComputingGroup/IBEX/issues/3262), +[#3782](https://github.com/ISISComputingGroup/IBEX/issues/3782), +[#3881](https://github.com/ISISComputingGroup/IBEX/issues/3881), +[#4004](https://github.com/ISISComputingGroup/IBEX/issues/4004) + +##### Note: Jasco ##### +[JASCO HPLC pump](https://jascoinc.com/products/chromatography/hplc/modules/hplc-pumps/) is a new (for ISIS) model of HPLC pump. Jasco pumps are replacing the old Hitachi pumps. + * See [#3923](https://github.com/ISISComputingGroup/IBEX/issues/3923), [#4004](https://github.com/ISISComputingGroup/IBEX/issues/4004), [#4198](https://github.com/ISISComputingGroup/IBEX/issues/4198), [#4199](https://github.com/ISISComputingGroup/IBEX/issues/4199). + +##### Note: Keithley ##### +1. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter) +1. See also [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176) + +##### Note: Neocera ##### +* Low temperature experiments?[Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) +* An IOC & OPI for the Neocera were created in ticket [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) + +##### Note: Linkam ##### +There are VIs for this.
+See also [Linkam T95 controller](http://www.linkam.co.uk/t95-system-controllers/) and tickets [#1106](https://github.com/ISISComputingGroup/IBEX/issues/1106), [#1496](https://github.com/ISISComputingGroup/IBEX/issues/1496), [#1509](https://github.com/ISISComputingGroup/IBEX/issues/1509). + +##### Note: Oxford Instruments ##### +1. Support for OI ITC503 created as part of [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) +1. Support for Mercury Temperature Controller was created as part of [#2840](https://github.com/ISISComputingGroup/IBEX/issues/2840) +1. Dilution fridge: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx) to be determined. + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) + +##### Note: Birmingham Magnet ##### +The Birmingham 17T magnet is a superconducting magnet owned by the University of Birmingham. The magnet comes with its own control devices & PC, all mounted in a rack. It gets loaned out to various facilities around the country, including ISIS. IBEX only needs to communicate with the Birmingham control PC. +1. There is information about the Birmingham 17T magnet here: `C:\LabVIEW Modules\Drivers\Birmingham 17T Magnet`. +1. Further discussion of the Birmingham 17T magnet in [#4523](https://github.com/ISISComputingGroup/IBEX/issues/4523). + +##### Note: Goudsmit Magnet ##### +1. [Goudsmit Magnet](https://www.isis.stfc.ac.uk/Pages/Goudsmit-Electromagnet.aspx). +1. [Goudsmit Magnet User Manual](https://www.isis.stfc.ac.uk/Pages/goudsmit-electromagnet6531.pdf). +1. Used in conjunction with a Danfysik PSU. +1. Further information is available here: `C:\LabVIEW Modules\Large Scale Structures\Goudsmit_magnet`. + +##### ~Note: Omega~ ##### +~[OMEGA™ iBTHX transmitter](https://www.omega.co.uk/pptst/IBTX_IBTHX.html) is a device to monitor and record barometric pressure, temperature, relative humidity, and dew point over an Ethernet network or the LOQnet.~ + +~**NOTE:** LOQ has switched from the OMEGA™ iBTHX (because it is unreliable) to MOXA ioLogik devices. Should we do the same on SANS2D?~ + +##### Note: Superlogics ##### +A Superlogics thermocouple 8018R device is used to monitor temperatures on SANS2D. The 8018R device has a RS-485 interface. A Superlogics RS-485/RS-232 converter is used to convert the interface to RS-232. +1. [Superlogics 8018R Thermocouple](https://www.superlogics.com/data-acquisition-99/data-acq-sensor-specific/sensor-specific-rs485/8018r.html). The 8018R supports up to 8 input channels. +1. [Superlogics 8520 Converter](https://www.superlogics.com/data-acquisition-99/data-acq-converters/8520.html). + +##### Note: Ordela Detector ##### +1. [ORDELA](http://www.ordela.com/) (Oak Ridge Detector Laboratory) is a manufacturer of area-detector devices. It is a spin-off from Oak Ridge National Laboratory. +1. The ORDELA device is no longer used on SANS2D. + +##### Note: Couette Cell ##### +1. The Couette Cell is driven by a [McLennan PM1000](#note-mclennan) (compatible with a PM600). +1. See also [#3105](https://github.com/ISISComputingGroup/IBEX/issues/3105) + +##### Note: T-Jump Cell ##### +1. The T-Jump Cell is driven by a Keithley 2400. +1. See also [#3175](https://github.com/ISISComputingGroup/IBEX/issues/3175) + +##### Note: Syringe Pumps ##### +1. [WPI Aladdin-1000 Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps/al1000-220.aspx) +1. See also [#3787](https://github.com/ISISComputingGroup/IBEX/issues/3787) + +##### Note: Detector Trolleys ##### +SANS2D has two detector trolleys (in the vacuum tank). One for the front detector, one for the rear detector. The trolleys allow the detectors to be move longitudinally along the vacuum tank (z-direction in SANS2D geometry). The detector trolleys also support beamstops. + * driven by Galils + +##### Note: Baffle Trolleys ##### +SANS2D has two baffle trolleys (in the vacuum tank). One for the front baffle, one for the rear baffle. The baffles are both positioned between the front and rear detectors. The baffle trolleys allow the baffles to be move longitudinally along the vacuum tank (z-direction in SANS2D geometry). + * driven by Galils + +##### Note: Beamstops ##### +SANS2D has beamstops on the front and rear detectors +1. Front Detector: + * has a single beamstop, mounted on the detector trolley. Rotates in the plane of the detector array. The beamstop must not be moved when the detector is rotated (in SECI this requirement is implemented as a software inhibit). + * has a strip beamstop, mounted on the detector trolley. Moves linearly across the face of the detector array. Primarily used for commissioning purposes (usually "parked" out of the way). +1. Rear Detector: + * has three beamstops, mounted on a pillar attached to the detector trolley. Pillar moves linearly across the face of the detector array. The individual beam stops move independently up/down on the pillar. + * has a strip beamstop, mounted on the detector trolley. Moves linearly across the face of the detector array. Primarily used for commissioning purposes (usually "parked" out of the way). + +## SANS2D Notes ## +SANS2D has the following specialist panels/systems: +1. SANS2D CAEN +1. SANS2D Detector Temperature +1. SANS2D DLS +1. LOQ Couette Cell +1. LOQ T-Jump Cell +1. SANS2D Tank +1. SANS2D Guide +1. SANS2D Vacuum System + +SANS2D has the following devices under motion control: +1. goniometer +1. jaws +1. rotation stage +1. sample changer 2-axis +1. sample changer +1. stirring stages + * See also [ControlsWork #553](https://github.com/ISISComputingGroup/ControlsWork/issues/553) +1. XYZ stage +1. 5 guide stages - driven by [Baldor drives](#sans2d-baldor-drives). + +Specialist requirements: +1. There is a requirement within SANS2D to inhibit the movement of the detectors when they are powered (the CAEN HV is ON) + +## SANS2D SECI Configs ## +Document information about SANS2D SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +SANS2D_place_holder1.conf | - | dd/mm/yyyy | - | +SANS2D_place_holder2.conf | - | dd/mm/yyyy | - | + +## SANS2D Genie Scripts ## +Similarly, Document information about SANS2D SECI Genie scripts here. + +## SANS2D Baldor Drives ## +SANS2D has 5 guides on the front end. Each of these has a significant weight - ~3 tonnes. The motors used to lift the guides are servo motors. Galil controllers cannot provide sufficient power to motors capable of lifting such weights. As a result, some Baldor drives were introduced to provide the power required to lift these guides. They are digital motion controllers that can communicate with a control system and have settings in each controller to drive the motor. + +However, ISIS uses them as pseudo dumb power amplifiers only. A Galil drives the axes like a servo - it provides an analogue signal to the Baldor to ask it to drive, but no core power. The Baldor drives the axes. Therefore, from the point of view of SECI or IBEX, the Baldor drives appear as Galil axes. + +The Baldor drives do need to be set up – but this is done independently. Historically, this entire arrangement caused a lot of problems during commissioning. There is a bespoke homing routine in use with this setup (the need for such a thing has been questioned but, for now, a bespoke homing routine continues in use). + +**Important Note:** These are the only Baldor axes at ISIS. There is limited support (apparently Baldor has been bought out since SANS2D was commissioned). For SANS2D to operate, the axes must be operational. The motion control group are considering whether to convert these axes to a Beckhoff system that is supportable. At the present time (November 2018) no decision has been taken to replace the Baldor drives. However, people are aware that support for these is limited. + +## Additional Information ## +The backlash distance in IBEX converted from SECI was `-2` however(Front Det X), IBEX doesn't like it so when a motor is moved to a negative direction, it uses BVEL speed instead of VELO [5675](https://github.com/ISISComputingGroup/IBEX/issues/5657). diff --git a/doc/processes/instrument_details/SURF-Instrument-Details.md b/doc/processes/instrument_details/SURF-Instrument-Details.md new file mode 100644 index 000000000..24c886dcc --- /dev/null +++ b/doc/processes/instrument_details/SURF-Instrument-Details.md @@ -0,0 +1,157 @@ +# SURF + +This page collects information that will be useful for the implementation of the IBEX control system on SURF. +## Background & Timeline ## +SURF is a reflectometer on TS1, used primarily for liquid interface research. The [SURF](https://www.isis.stfc.ac.uk/Pages/SURF.aspx) web page describes the background to the instrument. + +## Control System ## +It is proposed that SURF will migrate from the SECI control system to the IBEX control system in time for Cycle 2018/04. + +## SURF Equipment ## +The equipment listed below is used on SURF. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#note-dae) +ISIS | Mk3 | Chopper | N/A | | [see Chopper note](#note-choppers) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +Huber| | Sample Stack | Ethernet | | | [see Huber note](#note-huber) +??? | | 4-blade jaws | | | [see Jaws note](#note-jaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#note-eurotherm) +Julabo | FP-50 | Water Bath | RS-232 | | [see Julabo note](#note-water-baths) +Julabo | FP-52 | Water Bath | RS-232 | | [see Julabo note](#note-water-baths) +~Grant Instruments~ | ??? | ~Water Bath~ | ~RS-232~ | | [see Grant Water Bath note](#note-water-baths) +~Haake~ | ~N6~ | ~Water Bath~ | ~RS-232~ | | [see Haake note](#note-water-baths) +[Knauer](http://www.knauer.net/) | K6 | Electric valve drive | RS232 | |[see Knauer note](#note-knauer) +[Knauer](http://www.knauer.net/) | 1050 | HPLC pump | ??? | | [see Knauer HPLC note](#note-knauer-hplc-pump) +~Hitachi~ | ~L-7100~ | ~HPLC pump~ | ??? | | [see Hitachi note](#note-hitachi) +JASCO | PU-4180 | HPLC pump| ??? | | [see JASCO HPLC note](#note-jasco-hplc-pump) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#note-thurlby) +Keithley | 2410 | Source Meter | RS-232 | #1826 | [see Keithley note](#note-keithley) +Keyence | LK-G | Positioning Sensor | RS-232 | | [see Keyence note](#note-keyence) +Nima Trough | ??? | Trough | ??? | | [see Nima Trough note](#note-nima-trough) +Watson Marlow | 323 | Peristaltic Pump | ??? | | [see Peristaltic Pumps note](#note-peristaltic-pumps) +WPI | Aladdin-1000 | Syringe Pump | ??? | | [see Syringe Pumps note](#note-syringe-pumps) +WPI | SP2xx | Syringe Pump | ??? | | [see Syringe Pumps note](#note-syringe-pumps) + +##### Note: DAE ##### +See multi-detector and single-detector below. + +##### Note: Choppers ##### +SURF has a Mk3 chopper.
+ +##### Note: Jaws ##### +Provide information about SURF jaws. + +##### Note: Huber ##### +1. [Huber](http://www.xhuber.de/en/home/). +1. According to Huber's web-site, a Huber SMC9000 is a motion controller. See [SMC 9300](http://www.xhuber.de/en/product-groups/3-control-systems/smc-9300/) which appears to be the latest of the 9000-series and [SMC 9000](http://www.xhuber.de/fileadmin/user_upload/downloads/usermanuals/9000_1103.pdf) + * However, in terms of SURF, Huber seems to refer to a sample stack. Need to get to the bottom of this confusion. I guess it is a Huber sample stack (how many axes?) controlled by a SMC 9000 controller. + * See also [#3502](https://github.com/ISISComputingGroup/IBEX/issues/3502) + +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SURF/SURF_Eurotherms.jpg) are used to control temperature. + +##### Note: Water Baths ##### +1. Julabo Water Bath (apparently the following Julabos: WB7, WB30 ) + * consult the [Julabo commands](/specific_iocs/temp_controllers/julabo_commands.xlsx) spreadsheet for specific details of which set of commands is used by these water baths. +1. Grant Water Bath (no longer supported at ISIS) ~(apparently the following Grant water bath: WB27)~ + * Update (25-06-2019): Grant water baths are no longer used. No longer any need to support them. See [#4457](https://github.com/ISISComputingGroup/IBEX/issues/4457) +1. Haake Water Bath (no longer supported at ISIS) + 1. ~Haake N6 Water Bath (apparently the following Haake water bath: WB9). Haake has now been taken over by [ThermoFisher](https://www.thermofisher.com/uk/en/home/life-science/lab-equipment/water-baths-circulators-chillers.html). N6 model may be obsolete. Check existing SECI VI for logic and manual.~ + * Update (25-06-2019): Haake water baths are no longer used. No longer any need to support them. See [#4456](https://github.com/ISISComputingGroup/IBEX/issues/4456) +1. See also [Water Baths](https://www.isis.stfc.ac.uk/Pages/Water-Baths.aspx) list + +##### Note: Thurlby ##### +[Thurlby Thandar Instruments](https://www.aimtti.com/) +1. [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + * see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). + * Often used with the `Analog_Digital_IO` script. + * IOC & OPI updated (18-07-2019). See [#3784](https://github.com/ISISComputingGroup/IBEX/issues/3784) + +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs) (may be obsolete). Check existing SECI VI for logic and manual: both VI and manual are located here: `C:\LabVIEW Modules\Drivers\Hitachi L-7100`.~ + * Update (27-06-2019): Hitachi pumps are no longer used. No longer any need to support them. See [#3780](https://github.com/ISISComputingGroup/IBEX/issues/3780). + +##### Note: Knauer ##### +1. Knauer K-6 Electric Valve Drive. [Knauer web site](https://www.knauer.net)
+ * K-6 model appears to have been superseded. Check existing SECI VI for logic and manual. + * See `C:\LabVIEW Modules\Drivers\Knauer Electric Valve Drive K-6\Documentation` for existing VI. + * The Knauer K-6 Electric Valve Drive is (nearly) always used with the [Knauer HPLC pump](#note-knauer-hplc-pump). + +##### Note: Knauer HPLC Pump ##### +Knauer HPLC 1050 pump [Knauer HPLC 1050 is discontinued.](https://www.knauer.net/en/discontinued-smartline-pump-1050-successor-azura-p-61l/p14161). Check existing SECI VI for logic and manual. + * See `C:\LabVIEW Modules\Drivers\Knauer HPLC 1050\Documentation` for existing VI. + * See also [#3262](https://github.com/ISISComputingGroup/IBEX/issues/3262) + * Knauer HPLC pumps are often used in pairs - so two IOCs will be needed. + +##### Note: JASCO HPLC Pump ##### +[JASCO HPLC pump](https://jascoinc.com/products/chromatography/hplc/modules/hplc-pumps/) is a new (for ISIS) model of HPLC pump. Jasco pumps are replacing the old Hitachi pumps. + * See `C:\LabVIEW Modules\Drivers\Jasco PU-4180 HPLC Pump\Documentation` for documentation. + * See also [#3743](https://github.com/ISISComputingGroup/IBEX/issues/3743) and [#3923](https://github.com/ISISComputingGroup/IBEX/issues/3923) + +##### Note: Keithley ##### +1. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter) +1. See also [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176) + +##### Note: Keyence ##### +1. [Keyence Web-site](https://www.keyence.co.uk/index.jsp) +1. The Keyence LK-G is a laser positioning sensor (possibly superseded by a newer model). It is used when adjusting the height of the sample stage. Check the existing VI. + 1. See `C:\LabVIEW Modules\Instruments\INTER\Keyence LK-G` for existing VI (yes, the same device is used on INTER). + 1. Current models of [Keyence LK-G laser sensors](https://www.keyence.co.uk/products/measure/laser-1d/index.jsp) + 1. The Keyence LK-G is always in position on the instrument, although not always used. + +##### Note: Nima Trough ##### +Nima Trough: SECI uses a manufacturer supplied VI (see `C:\LabVIEW Modules\Drivers\Nima Trough`). We may need to do the same in IBEX (via lvDCOM). + * **Note:** NIMA Technologies Ltd now seems to be part of [Biolin Scientific](https://www.biolinscientific.com/ksvnima).
+ * The NIMA trough is used regularly on SURF. The manufacturer supplied VI is used to view graphs showing information about thin films. + * The manufacturer has made additional software available for download. A copy of this software is located in `\\isis\shares\ISIS_Experiment_Controls\NIMA Trough\Nima_TR8.1.zip`. + +##### Note: Peristaltic Pumps ##### +1. [Watson Marlow 323 Peristaltic Pump](http://www.watson-marlow.com/gb-en/range/watson-marlow/300-tube-pumps/323d/) +1. Check existing SECI VI for logic and manual: VI and manuals are located here: + * `C:\LabVIEW Modules\Drivers\Watson Marlow 323 Pump` + +##### Note: Syringe Pumps ##### +1. [WPI Aladdin-1000 Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps/al1000-220.aspx) +1. [WPI SP2xx Syringe Pump](https://www.wpi-europe.com/products/pumps--microinjection/laboratory-syringe-pumps.aspx) - check specific model. See also [#3261](https://github.com/ISISComputingGroup/IBEX/issues/3261) +1. The Watson Marlow and WPI Aladdin-1000 syringe pumps are the ones used most frequently on SURF (usually depending on which one is available). +1. Check existing SECI VIs for logic and manual: VIs and manuals are located here: + * `C:\LabVIEW Modules\Drivers\WPI Aladdin-1000 Syringe Pump` + * `C:\LabVIEW Modules\Drivers\WPI SP2XX Pumps` + +## SURF Notes ## +SURF has the following specialist panels/systems: +1. INTER Keyence LK-G +1. SURF Galil AO +1. SURF Galil DIO +1. SURF Waterbath (Grant) +1. SURF Vacuum System +1. SURF Motion Control + +SURF has the following devices under motion control: +1. goniometer +1. jaws +1. jaws & height +1. multi detector +1. single detector +1. supermirror +1. transmission monitor +1. XYZ stage + + +## SURF SECI Configs ## +Document information about SURF SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +SURF_place_holder1.conf | - | dd/mm/yyyy | - | +SURF_place_holder2.conf | - | dd/mm/yyyy | - | + +## SURF Genie Scripts ## +Similarly, Document information about SURF SECI Genie scripts here. diff --git a/doc/processes/instrument_details/SXD-Instrument-Details.md b/doc/processes/instrument_details/SXD-Instrument-Details.md new file mode 100644 index 000000000..2b3b10fe7 --- /dev/null +++ b/doc/processes/instrument_details/SXD-Instrument-Details.md @@ -0,0 +1,131 @@ +# SXD + +This page collects information that will be useful for the implementation of the IBEX control system on SXD. +## Background & Timeline ## +SXD is a Single Crystal Diffractometer, on TS1, which uses the time-of-flight techniques. The [SXD](https://www.isis.stfc.ac.uk/Pages/SXD.aspx) web page describes the background to the instrument. + +## Control System ## +SXD will migrate from the SECI control system to the IBEX control system in MMMMMMMM YYYY. + +## SXD Equipment ## +The equipment listed below is used on SXD. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 / 3 | Detector Electronics | Ethernet | | [see DAE note](#sxd_note-dae) +N/A | N/A | N/A | N/A | | [see Chopper note](#sxd_note-chopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#sxd_note-mclennan) +[LINMOT](http://www.linmot.com/) | [P0x-23](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) | Linear Motors and Motion Controller | RS-232 | [#2098](https://github.com/ISISComputingGroup/IBEX/issues/2098) | [see LinMot note](#sxd_note-linmot) | +??? | | 4-blade jaws | | | [see Jaws note](#sxd_note-jaws) +ISIS | ??? | Mirror| | | [see Mirror note](#sxd_note-mirror) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#sxd_note-vacuum) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#sxd_note-vacuum) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#sxd_note-eurotherm) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#sxd_note-keithley) +Kammrath and Weiss | DDS 32 | Tensile Rig | RS-232 | | [see Kammrath and Weiss note](#sxd_note-kammrath-and-weiss) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#sxd_note-neocera) +ISIS | ??? | Helium Gauge | | |[see Helium Gauge note](#sxd_note-helium-gauge) +B&WTek | i-Raman Plus | Raman Spectrometer| | | [see mini-Raman Spectrometer note](#sxd_noteRamanSpect) + +{#sxd_note-dae} +##### Note: DAE ##### +Now using DAE-3. See multi-detector and single-detector below. + +{#sxd_note-chopper} +##### Note: Chopper ##### +SXD does not use a chopper. + +{#sxd_note-mclennan} +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +{#sxd_note-linmot} +##### Note: LinMot ##### +SXD uses LinMot P0x-23 motors to move its jaws, controlled by LinMot drives. SXD also uses LinMot motors to move the Mirror.
+[LinMot User Manual](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) + +{#sxd_note-mirror} +##### Note: Mirror ##### +The mirror on SXD is located upstream from the jaws. Its purpose is to direct laser light from an external laser to the sample position. The reason for the mirror is that we cannot have the laser in the vacuum of the flight path, but we can have a mirror. Most of the time it is parked on the side away from the direct beam. The mirror is positioned using LinMot motors. + +{#sxd_note-jaws} +##### Note: Jaws ##### +Provide information about SXD jaws. + +{#sxd_note-vacuum} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#sxd_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/SXD/SXD_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#sxd_note-keithley} +##### Note: Keithley ##### +1. [Keithley 2400 Series Source Meter](https://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter).
+See also tickets [#1826](https://github.com/ISISComputingGroup/IBEX/issues/1826), [#2695](https://github.com/ISISComputingGroup/IBEX/issues/2695), [#2801](https://github.com/ISISComputingGroup/IBEX/issues/2801) and [#3176](https://github.com/ISISComputingGroup/IBEX/issues/3176). + +{#sxd_note-kammrath-and-weiss} +##### Note: Kammrath and Weiss ##### +1. The Kammrath-Weiss tensile stress rig is controlled from a dedicated laptop running the manufacturer's software. SECI communicates with the PC to send/read values to/from the manufacturer's software. +1. [Kammrath and Weiss](https://www.kammrath-weiss.com/en/tensile-compression-modules/).
+ * See also ticket [#2681](https://github.com/ISISComputingGroup/IBEX/issues/2681) + +{#sxd_note-neocera} +##### Note: Neocera ##### +Low temperature experiments?[Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +{#sxd_noteAttoCube} +##### Note: Attocube ##### +1. [AttoCube](http://www.attocube.com/) - check existing VI. +1. There are existing [EPICS drivers](https://epics.anl.gov/modules/manufacturer.php#attocube) for Attocube devices. + +{#sxd_note-helium-gauge} +##### Note: Helium Gauge ##### +1. ISIS Helium Gauge - check existing VI. + +{#sxd_noteRamanSpect} +##### Note: mini-Raman Spectrometer ##### +1. [B&W Tek i-Raman Plus](http://bwtek.com/products/i-raman-plus/) SXD & HRPD wish to share a Raman Spectrometer (and centre-stick). + +## SXD Notes ## +SXD has the following specialist panels/systems: +1. SXD Atto Cube + * see [see Atto Cube note](#sxd_noteAttoCube) +1. SXD Jaws (Linmot) + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Jaws\Screens` +1. SXD Centre Stick (PM600) + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Centre Stick` +1. SXD Compact Focussing Device (PM600) + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Compact Focusing Device` + * Note from the instrument scientist: The compact focussing device (CFD) refers to a motorised mount that is permanently installed on SXD. The CFD is a compact supermirror guide, called a trumpet, with a length of 390mm and a rectangular cross section going from 15*15mm at the entrance down to 5*5mm at the exit. It was only used for testing and never since. The mount has x, y translation and horizontal and vertical angular adjustment. There was also translation along the beam but this was removed since it caused the entire setup to droop. +1. SXD Furnace (PM600) + * This implies that a Furnace is in use - check that it can be controlled via a Eurotherm. + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Furnace` +1. SXD CCR (PM600) +1. SXD PE CCR (PM600) + * These two items imply that CCRs are in use - check that they can be controlled via a Eurotherm. + * See `C:\LabVIEW Modules\Instruments\SXD\SXD CCR` and `C:\LabVIEW Modules\Instruments\SXD\SXD PE CCR` +1. Parma Sample Changer (PM600) + * What about `C:\LabVIEW Modules\Parma Sample Changer` and `C:\LabVIEW Modules\Parma Sample Changer (Axis 2)`? +1. SXD Keithley Source (Keithley 2410) + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Keithley Source` +1. SXD Foil Changer + * See `C:\LabVIEW Modules\Instruments\SXD\SXD Foil Changer` + * The instrument scientists don't know what this is. Assumed obsolete. + +SXD has the following devices under motion control: +1. goniometer +1. XYZ stage + +## SXD SECI Configs ## +Document information about SXD SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +SXD_place_holder1.conf | - | dd/mm/yyyy | - | +SXD_place_holder2.conf | - | dd/mm/yyyy | - | + +## SXD Genie Scripts ## +Similarly, Document information about SXD SECI Genie scripts here. diff --git a/doc/processes/instrument_details/TOSCA-Instrument-Details.md b/doc/processes/instrument_details/TOSCA-Instrument-Details.md new file mode 100644 index 000000000..66694ae43 --- /dev/null +++ b/doc/processes/instrument_details/TOSCA-Instrument-Details.md @@ -0,0 +1,92 @@ +# TOSCA + +This page collects information that will be useful for the implementation of the IBEX control system on TOSCA. +## Background & Timeline ## +TOSCA is an indirect geometry spectrometer, on TS1. The [TOSCA](https://www.isis.stfc.ac.uk/Pages/TOSCA.aspx) web page describes the background to the instrument. + +## Control System ## +TOSCA will migrate from the SECI control system to the IBEX control system in late October 2018 (prior to Cycle 2018/03). + +## TOSCA Equipment ## +The equipment listed below is used on TOSCA. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#tosca_noteDAE) +ISIS | Mk3 Chopper | Chopper | Ethernet/.NET | [#169](https://github.com/ISISComputingGroup/IBEX/issues/169) | [see Mk3 Chopper note](#tosca_noteMk3Chopper) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#tosca_noteMcLennan) +ISIS| | Sample Positioner | via McLennan | | [see Sample Positioner note](#tosca_noteSamplePositioner) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#tosca_noteVacuum) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#tosca_noteVacuum) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#tosca_noteEurotherm) +MKS | PDR2000 | Pressure Transducer | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#MKS) | +LakeShore | 218 | Temperature Monitor | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore)| [see LakeShore note](#tosca_noteLakeshore ) +ISIS | TOSCA | Helium Gauge | | |[see Helium Gauge note](#tosca_noteHeliumGauge) + +{#tosca_noteDAE} +##### Note: DAE ##### +See multi-detector and single-detector below. + +{#tosca_noteMk3Chopper} +##### Note: ISIS Mk3 Choppers ##### +TOSCA has an ISIS Mk3 double-disk chopper. + +{#tosca_noteMcLennan} +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +As of 2018-11-15, TOSCA uses only one McLennan motor, which is for the sample changer. The other motors were used for a rotating centre stick and for a rotating stage - both of these were used once a long time ago. The McLennan which drove the rotating centre stick was configured during the migration and then removed after migrating as the scientists do not use it. If it is needed in the future, look at the standard configuration before 2018-11-14 to see details of the motor configuration for the rotating centre stick. + +{#tosca_noteSamplePositioner} +##### Note: Sample Positioner ##### +The Sample Positioner is driven by a McLennan motor (i.e. it raises/lowers sample plates attached to a chain drive, driven by the McLennan). Here are 3 photograph of the Sample Positioner: [One](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/TOSCA/TOSCA_Sample_Changer_1.jpg), [Two](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/TOSCA/TOSCA_Sample_Changer_2.jpg), [Three](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/TOSCA/TOSCA_Sample_Changer_3.jpg). + +{#tosca_noteJaws} +##### Note: Jaws ##### +There are no jaws on TOSCA. + +{#tosca_noteVacuum} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063). The vacuum system is currently operated separately. + +{#tosca_noteEurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/TOSCA/TOSCA_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#tosca_noteLakeshore} +##### Note: LakeShore ##### + +TOSCA uses four Lakeshore 218s - not all of the channels are enabled on each of these. + +1. [Model 218](http://www.lakeshore.com/products/Cryogenic-Temperature-Monitors/Model-218/Pages/Overview.aspx) + * See also [#1097](https://github.com/ISISComputingGroup/IBEX/issues/1097), [#1098](https://github.com/ISISComputingGroup/IBEX/issues/1098) & [#3223](https://github.com/ISISComputingGroup/IBEX/issues/3223). + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) for a variety of Lakeshore temperature controllers. + +{#tosca_noteHeliumGauge} +##### Note: Helium Gauge ##### +1. The Helium Gauge is no longer used on TOSCA. + +## TOSCA Notes ## +TOSCA has the following specialist panels/systems: +1. ~MAPS CCR (PM600)~ + * See [McLennan note](#tosca_noteMcLennan). +1. ~TOSCA Helium Level Gauge~ + * See [Helium Gauge](#tosca_noteHeliumGauge) note +1. TOSCA New Sample Changer (PM600) +1. ~Rotating Centre Stick (PM600)~ + * See [McLennan note](#tosca_noteMcLennan). + +TOSCA has the following devices under motion control: +1. XY beam scanner (2 axis GALIL motor - this GALIL moves about and is not permanently on TOSCA). + +## TOSCA SECI Configs ## +Document information about TOSCA SECI configs here. + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +Standard + Rotation Stage.conf | - | 25/10/2018 | - | + +## TOSCA Genie Scripts ## +Similarly, Document information about TOSCA SECI Genie scripts here. diff --git a/doc/processes/instrument_details/VESUVIO-Instrument-Details.md b/doc/processes/instrument_details/VESUVIO-Instrument-Details.md new file mode 100644 index 000000000..f088a7080 --- /dev/null +++ b/doc/processes/instrument_details/VESUVIO-Instrument-Details.md @@ -0,0 +1,125 @@ +# VESUVIO + +This page collects information that will be useful for the implementation of the new control system on VESUVIO. +## Background & Timeline ## +VESUVIO (previously known as EVS) is a long established instrument at ISIS, on TS1. The [VESUVIO](http://www.isis.stfc.ac.uk/instruments/vesuvio/) web page describes the background to the instrument. An image of VESUVIO is shown in ​[this diagram](http://www.isis.stfc.ac.uk/instruments/vesuvio/vesuvio-configuration6163.jpg).
+ +## Control System ## +VESUVIO will migrate from the SECI control system to the IBEX control system. + +## VESUVIO Equipment ## +The equipment listed below is used on VESUVIO. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#vesuvio_noteDAE) +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | [see Galil note](#vesuvio_noteGalil) +[Pfeiffer](https://www.pfeiffer-vacuum.com/en/products/) | TPG 26x | ISIS Vacuum System | RS232 | EPICS | [see Pfeiffer note](#vesuvio_notePfeiffer) +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#vesuvio_noteMcLennan) +Oxford Instruments | Dilution Fridge | Cryogenic System | RS-232 | | [see Oxford Instruments note](#vesuvio_noteOxfordInstruments) +Eurotherm | All models at ISIS | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#vesuvio_noteEurotherm) +ISIS | RAL | Furnaces | N/A | | [see Furnace note](#vesuvio_noteFurnace) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#vesuvio_noteNeocera) +Julabo | TBD | Waterbath | RS-232 | EPICS | [see Julabo note](#vesuvio_noteJulabo) +Chell | CCD100 | Pressure Transducer | RS-232 | | [see Chell note](#vesuvio_noteChell) +Keithley | 2410 | Source Meter | RS-232 | | [see Keithley note](#vesuvio_noteKeithley) + +{#vesuvio_noteDAE} +##### Note: DAE ##### +Main Detector banks + +{#vesuvio_noteGalil} +##### Note: Galil ##### +Devices controlled by Galil: [Eulerian Cradle](http://www.xhuber.de/en/product-groups/1-positioning-devices/12-rotation/eulerian-cradles/). + +{#vesuvio_notePfeiffer} +##### Note: Pfeiffer ##### +1. [Model TPG 26x] (https://www.pfeiffer-vacuum.com/en/products/measurement/activeline/controllers/?detailPdoId=5724) + +{#vesuvio_noteMcLennan} +##### Note: McLennan ##### +1. The motors are set up in the following setup: + 1. MTR0101: Forward scattering foil, left + 1. MTR0102: Forward scattering foil, right + 1. MTR0201: Uranium foil + 1. MTR0202: Backward scattering foil +1. The single axis McLennan (currently Uranium foil) may be used to control one of the following other devices at a time: Rotating centre-stick, EVS Rotating centre-stick, McLennan-Newport Rotation Stage, MAPS CCR, PRISMA Omega Centre Stick, etc +1. The backward scattering foil is linked to hardware interlocks. If the cabin is open, the shutter can't be moved. In these cases, it will report being at either an upper or lower limit. + +{#vesuvio_noteOxfordInstruments} +##### Note: Oxford Instruments ##### +1. Dilution fridge: [models](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/dilution-refrigerators/dilution-refrigerators8825.html) to be determined. + +{#vesuvio_noteEurotherm} +##### Note: Eurotherm ##### +Used to control temperature of top-loading CCRs, Furnace, Orange Cryostat devices, etc. + +{#vesuvio_noteFurnace} +##### Note: Furnaces ##### +Vesuvio uses RAL designed furnaces - RAL5, vanadium and niobium furnaces (see [Furnaces](http://www.isis.stfc.ac.uk/sample-environment/high-temperature/standard-furnaces/standard-furnaces13745.html)). + +These are controlled via a Eurotherm temperature controller. + +{#vesuvio_noteNeocera} +##### Note: Neocera ##### +Low temperature experiments?[Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) + +{#vesuvio_noteJulabo} +##### Note: Julabo ##### +Experiments requiring waterbath. + +{#vesuvio_noteChell} +##### Note: Chell ##### +Experiments requiring pressure transducer. [Chell CCD100](http://www.chell.co.uk/product_details/flow-products/chell-ccd100) + +{#vesuvio_noteKeithley} +##### Note: Keithley ##### +Experiments requiring source meter. [Keithley 2400 Source Meter](http://uk.tek.com/keithley-source-measure-units/keithley-smu-2400-series-sourcemeter) + +## VESUVIO SECI Configs ## +Over the years, VESUVIO is likely to have accumulated a significant collection of SECI configs. Not all of them are current (some may no longer be used, or used only infrequently). We will need to investigate how to convert them to IBEX configs. +The config files are located in the folder `SECI\Configurations\` on the VESUVIO control server and have the extension `.conf`. Sub-configuration files have the extension `.comp` (component). The files with numbers as extensions (`.1`, `.2`, `.3`,etc.) are backups of the configuration and sub-configuration files. + + + +| Configuration Name | Sub-Configurations | Last Accessed | Required | +------------ |-------------------------------------| ------------- | ------------- | +VESUVIO_withTLCCR__31mar2015 | Sub_EVS_Base, Sub_EVS_Eurotherm_CCR | 16/09/2016 | - | +highTfurnace__oct2015 | - | 12/10/2015 | - | +EVS_Huber | Eurotherm | 09/09/2015 | - | +VESUVIO_Nbfurnace__15june2015 | Eurotherm | 15/06/2015 | - | +VESUVIO_emptyInstrument__26mar2015 | Eurotherm | 26/03/2015 | - | +VESUVIO_emptyInstrument__16mar2015 | Eurotherm | 16/03/2015 | - | +EVS Ambient | Eurotherm | 22/08/2014 | - | +tempalooza | Eurotherm | 13/06/2014 | - | +20130425 - EVS Ambient | - | 26/04/2013 | - | +rotating cs + cryo | - | 28/07/2011 | - | +heaters on sample 0 -100C | - | 28/07/2011 | - | +EVS Heated Can | - | 22/07/2011 | - | +EVS Cryostat | - | 22/07/2011 | - | +EVS Ambient + Platinums | - | 22/07/2011 | - | +Calibration | - | 31/03/2011 | - | +U Foil Calibration | - | 14/02/2011 | - | +rotating prisma cs | - | 14/02/2011 | - | +rotating prisma cs + cryo | - | 14/02/2011 | - | +rotating cs | - | 14/02/2011 | - | +rotating ccr | - | 14/02/2011 | - | +Neocera | - | 14/02/2011 | - | +Kelvinox | - | 14/02/2011 | - | +EVS Water Bath | - | 14/02/2011 | - | +EVS Kelvinox | - | 14/02/2011 | - | +EVS CCR | - | 14/02/2011 | - | +EVS Ambient + Cryostat | - | 14/02/2011 | - | +EVS 4K TLCCR | - | 14/02/2011 | - | +ChipIR2 | - | 14/02/2011 | - | +ChipIR | - | 14/02/2011 | - | + +## VESUVIO Genie Scripts ## +Similarly, VESUVIO has built up a significant collection of genie scripts over the years. Again, many scripts are old and may no longer be used regularly. There will be a need to convert some of these scripts to genie-python, but probably not all. + +## Tested IOCs ## + +| What | When | Notes | +| ---- | ---- | ----- | +| Device-X | dd/mm/yyyy| some notes | + diff --git a/doc/processes/instrument_details/WISH-Instrument-Details.md b/doc/processes/instrument_details/WISH-Instrument-Details.md new file mode 100644 index 000000000..ec1c6352e --- /dev/null +++ b/doc/processes/instrument_details/WISH-Instrument-Details.md @@ -0,0 +1,170 @@ +# WISH + +This page collects information that will be useful for the implementation of the IBEX control system on WISH. +## Background & Timeline ## +WISH is a long-wavelength diffractometer, on TS2. The [WISH](https://www.isis.stfc.ac.uk/Pages/WISH.aspx) web page describes the background to the instrument. + +## Control System ## +WISH has migrated to IBEX from SECI + +## WISH Equipment ## +The equipment listed below is used on WISH. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes | +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | DAE 2 | Detector Electronics | Ethernet | | [see DAE note](#wish_note-dAE) +ISIS | MK3 | Disk Chopper | Ethernet | EPICS | [see Disk Chopper note](#wish_note-disk-chopper) | +GALIL | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Galil%20Motion%20Control) | | +McLennan | PM600 | Motion Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#McLennan%20Servo%20Supplies) | [see McLennan note](#wish_note-mclennan) +??? | | 4-blade jaws | | | [see Jaws note](#wish_note-jaws) +ISIS | ??? | ISIS Vacuum System | | |[see Vacuum System note](#wish_note-vacuum-system) +Pfeiffer | TPG300 | Vacuum Gauge | RS-232 | EPICS | [see Vacuum System note](#wish_note-vacuum-system) +Eurotherm | Temperature Controller | All models at ISIS | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) | [see Eurotherm note](#wish_note-eurotherm) +LakeShore | 340 | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) | [see LakeShore note](#wish_note-lakeshore) +LakeShore | 350 | Temperature Controller | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore)| [see LakeShore note](#wish_note-lakeshore) +LakeShore | 370 | Resistance Bridge | RS-232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore)| [see LakeShore note](#wish_note-lakeshore) +Neocera | LTC-21 | Temperature Controller | RS-232 | | [see Neocera note](#wish_note-neocera) +Oxford Instruments | Triton | Dilution Fridge | RS-232 | | [see Oxford Instruments note](#wish_note-oxford-instruments) +Oxford Instruments | Mercury | Temperature Controller | RS-232 | | [see Oxford Instruments note](#wish_note-oxford-instruments) +Oxford Instruments | | Cryogenic Equipment | RS-232 | | [see Oxford Instruments note](#wish_note-oxford-instruments) +Oxford Instruments | Teslatron | | RS-232 | | [see Oxford Instruments note](#wish_note-oxford-instruments) +Thurlby | EX355P | PSU | ??? | | [see Thurlby note](#wish_note-thurlby) +[Stanford RS](http://www.thinksrs.com/) | SR850 | Amplifier | | | [see Stanford RS note](#wish_note-stanfordrs) +Tektronix | AFG 3021B | Function Generator | Ethernet | #237 |[see Tektronix note](#wish_note-tektronix) +Keithley | 6517B | Electrometer | RS-232 | | [see Keithley note](#wish_note-keithley) +Razorbill | RP100 | Power Supply | RS-232 | | [see Razorbill note](#wish_note-razorbill) + +{#wish_note-dAE} +##### Note: DAE ##### +See multi-detector and single-detector below. + +{#wish_note-disk-chopper} +##### Note: WISH Disk Chopper ##### +Disk Chopper is an ISIS MK3 chopper. + +{#wish_note-mclennan} +##### Note: McLennan ##### +Support for McLennan devices is well established (see [#1099](https://github.com/ISISComputingGroup/IBEX/issues/1099) and [#1100](https://github.com/ISISComputingGroup/IBEX/issues/1100) and subsequent tickets. + +##### Note: LinMot ##### +[LinMot User Manual](http://www.linmot.com/fileadmin//user_upload/Downloads/software-firmware/servo-drives/linmot-talk-1-3-x/UserManual_1r3_e_recent.pdf) + +{#wish_note-jaws} +##### Note: Jaws ##### +Provide information about WISH jaws. + +{#wish_note-vacuum-system} +##### Note: Vacuum System ##### +TPG300 support is implemented via [#216](https://github.com/ISISComputingGroup/IBEX/issues/216) and [#2063](https://github.com/ISISComputingGroup/IBEX/issues/2063) + +{#wish_note-eurotherm} +##### Note: Eurotherm ##### +[Eurotherms](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/WISH/HIFI_Eurotherms.jpg) are used to control temperature Orange Cryostat, CCR and Furnace devices. + +{#wish_note-lakeshore} +##### Note: LakeShore ##### +1. [Model 340](http://www.lakeshore.com/products/Cryogenic-Temperature-Controllers/Model-340/Pages/Overview.aspx): this model is now obsolete, having been replaced by the 336 and 350 models. +1. [Model 350](http://www.lakeshore.com/products/Cryogenic-Temperature-Monitors/Model-350/Pages/Overview.aspx): +1. [Model 370](https://www.lakeshore.com/products/AC-Resistance-Bridges/Model-370/Pages/Overview.aspx): this model is now obsolete, having been replaced by the 372 model. + +There are [EPICS drivers](http://www.aps.anl.gov/epics/modules/manufacturer.php#Lakeshore) for a variety of Lakeshore temperature controllers. + +{#wish_note-neocera} +##### Note: Neocera ##### +1. [Neocera LTC-21 Manual](http://www.submm.caltech.edu/~sharc/technical/LTC-21%20manual.pdf) +1. See also [#1828](https://github.com/ISISComputingGroup/IBEX/issues/1828) + +{#wish_note-oxford-instruments} +##### Note: Oxford Instruments ##### +1. Dilution fridge: [models](https://www.isis.stfc.ac.uk/Pages/Dilution-Refrigerators.aspx) to be determined. + * Triton LabVIEW Driver : C:\LabVIEW Modules\Drivers\Triton + * Triton Documentation : C:\LabVIEW Modules\Drivers\Triton\Documentation + * See also [#2915](https://github.com/ISISComputingGroup/IBEX/issues/2915) +1. Blue Cryostat: [models](https://www.isis.stfc.ac.uk/Pages/Oxford-Variox-Cryostats.aspx) to be determined. +1. 7.5T [Cryomagnet](https://www.isis.stfc.ac.uk/Pages/75T-Magnet.aspx) (aka the Teslatron) + * See [#2593](https://github.com/ISISComputingGroup/IBEX/issues/2593) + * Can also be 9T (see LET) or 14T magnets. + +{#wish_note-thurlby} +##### Note: Thurlby ##### +[Thurlby Thandar Instruments](https://www.aimtti.com/) +1. [Thurlby Thandar Instruments EX355P PSU](https://www.aimtti.com/product-category/dc-power-supplies/aim-ex-rseries) + * see [#155](https://github.com/ISISComputingGroup/IBEX/issues/155) and [#198](https://github.com/ISISComputingGroup/IBEX/issues/198). + +{#wish_note-tektronix} +##### Note: Tektronix ##### +1. Tektronix Function Generator: [AFG 3021B](https://www.tek.com/datasheet/afg3000-series) + * See also [#237](https://github.com/ISISComputingGroup/IBEX/issues/237) + +{#wish_note-stanfordrs} +##### Note: Stanford RS ##### +1. [SR850 lock-in amplifier](http://www.thinksrs.com/products/sr850.html) +1. There are existing [EPICS drivers](https://epics.anl.gov/modules/manufacturer.php#Stanford%20Research) for Stanford devices. + +{#wish_note-keithley} +##### Note: Keithley ##### +1. [Keithley 6500 Series Electrometers](https://uk.tek.com/keithley-low-level-sensitive-and-specialty-instruments/keithley-high-resistance-low-current-electrom). + * [Keithley 6517B DataSheet](https://uk.tek.com/datasheet/high-resistance-low-current-electrometers-series-6500-6430/model-6517b-electrometer-high-r) + * See also [Keithley 6517B Reference Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Forms/AllItems.aspx?RootFolder=%2Fisis%2Fcomputing%2FICPdiscussions%2FKeithley&FolderCTID=0x01200027AD8F05966A2748B3B04C98BB5B442B&View={F2C33C51-70E6-4343-B937-2C59A2568306}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence) +1. There are existing [EPICS drivers](https://epics-controls.org/resources-and-support/modules/hardware-support/) for similar Keithley devices (but not for model 6517B). + +{#wish_note-razorbill} +##### Note: Razorbill ##### +1. This is a small power supply controlling the WISH strain cell. + * [RP100 Manual](https://razorbillinstruments.com/wp-content/uploads/2018/10/RP100-Manual-v6.1-1.pdf) + +## WISH Notes ## +WISH has the following specialist panels/systems: +1. Rotating CCR (PM600) + * See `C:\LabVIEW Modules\Rotating CCR\Rotating CCR - Front Panel.vi` +1. GEM Beamline Jaws (PM341) + * check existing VI on WISH (GEM has been upgraded since original system was written) + * PM341 is a McLennan model + * Instrument scientist is not sure why GEM VIs are installed on WISH (they may not actually be used). +1. GEM Jaws (Linmot) + * check existing VI on WISH (GEM has been upgraded since original system was written) + * Instrument scientist is not sure why GEM VIs are installed on WISH (they may not actually be used). +1. GEM ORC + * check existing VI on WISH (GEM has been upgraded since original system was written) + * See also [#2409](https://github.com/ISISComputingGroup/IBEX/issues/2409) and [#2808](https://github.com/ISISComputingGroup/IBEX/issues/2808) + * WISH does have an ORC. Need to check if it the same as the GEM ORC. +1. MAPS CCR (PM600) + * See `C:\LabVIEW Modules\Instruments\MAPS\MAPS CCR\MAPS CCR - Front Panel.vi` +1. PEARL Pressure Transducer + * See `C:\LabVIEW Modules\Instruments\PEARL\PEARL Pressure Transducer\PEARL Pressure Transducer.vi` +1. PEARL Pressure Cell Controller + * See `C:\LabVIEW Modules\Instruments\PEARL\PEARL Pressure Cell Controller\PEARL Pressure Cell Controller - Front Panel.vi` +1. POLARIS Sample Changer + * See [#2173](https://github.com/ISISComputingGroup/IBEX/issues/2173) + * See also `C:\LabVIEW Modules\Instruments\POLARIS\POLARIS Sample Changer\POLARIS SC - Front Panel.vi` +1. WISH Jaws +1. WISH Lakeshore 370 +1. WISH Oscillating Radial Collimator (Galil) - see [here](/specific_iocs/motor_extensions/MERLIN,-LET-and-WISH-Oscillating-radial-collimators) for more info +1. WISH Vacuum PLC + +WISH has the following devices under motion control: +1. jaws +1. XYZ stage + * This is also referred to as/is controlling the "Alignment Flange" + + +WISH also tends to borrow things from other instruments. Commonly used equipment: + +1. PEARL pressure controller +1. Mercury-controlled Heliox cryostat insert +1. Blue Cryostat (ITC controlled) +1. IRIS Baratron Gas cell +1. Automatic needle valve - see note on [ticket #6695](https://github.com/ISISComputingGroup/IBEX/issues/6695) for investigation, [ticket #4240](https://github.com/ISISComputingGroup/IBEX/issues/4240) for IOC implementation and [ticket #6777](https://github.com/ISISComputingGroup/IBEX/issues/6777) for OPI. + +## WISH SECI Configs ## +Document information about WISH SECI configs here. + +Note: Most of WISH configs revolve around the WISH base config. This has been created as a component in IBEX and subsequent components for each thing that is added to the WISH base config should be added as components (i.e. the Teslatron, Strain Cell, Rotation stages etc.) + +Configuration Name | Sub-Configurations | Last Accessed | Required | +---------------------------------------|----------------------------------------------------|---------------|----------| +WISH_base.conf | WISH_base.comp | 01/07/2021 | - | +WISH_base + Oxford Cryostat.conf | WISH_base.comp, Oxford - teslatron.comp | "" | - | + +## WISH Genie Scripts ## +Similarly, Document information about WISH SECI Genie scripts here. diff --git a/doc/processes/instrument_details/ZOOM-Instrument-Details.md b/doc/processes/instrument_details/ZOOM-Instrument-Details.md new file mode 100644 index 000000000..d172425bb --- /dev/null +++ b/doc/processes/instrument_details/ZOOM-Instrument-Details.md @@ -0,0 +1,374 @@ +# ZOOM + +This page collects information that will be useful for the implementation of the new control system on ZOOM. +## Background & Timeline ## +The [ZOOM](http://www.isis.stfc.ac.uk/instruments/zoom/zoom8060.html) web page describes the background to the instrument. Additional material is also available on the [ZOOM sharepoint](http://www.facilities.rl.ac.uk/isis/projects/ts2/phase2instruments/ZOOM/Forms/AllItems.aspx). ZOOM is expected to have 36 motion axes. + +The ZOOM shutter is due to open in February 2017 (see Zoom plan 28th May 2016). + +## Control System ## +ZOOM will use the IBEX control system. + +## ZOOM Equipment ## +The equipment listed below is to be used on ZOOM. Please add new information (e.g. new items of equipment, new notes, information about drivers, etc. to this table). + +Manufacturer | Model | Type | Connection | Driver | Notes +------------ | ------------- | ------------- | ------------- | ------------- | ------------------------------------------- +ISIS | Fast Shutter | | OMRON | |[see Fast Shutter note](#zoom_noteFastShutter) +ISIS |Double Disc Chopper | CHOPPER | Ethernet/.NET | #170 |[see ISIS Double Disk Chopper note](#zoom_noteDDChopper) + | | JAWS | | #179, #180 |[see Jaws note](#zoom_noteJaws) + | Polariser, guide & collimation unit (PGC) | | | #1233 |[see Polariser, Guide & Collimator note](#zoom_notePGC) + | | JAWS | | #179, #180 | +ISIS | | Spin Flipper | | #1234 |[see SpinFlipper note](#zoom_noteSpinFlipper) + | | JAWS | | #179, #180 | + | | Alignment Laser | | |[see Alignment Laser note](#zoom_noteLaser) + | | JAWS | | #179, #180 | +[JJ XRAY](http://www.jjxray.dk/) | [7-axis Sample Stack](http://www.jjxray.dk/products/positioning/multi-axis-stages/sample-stacks) [Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR%20JJXRAY%20SAMPLE%20STACK.pdf) | Sample Stack | | #304, #1238 |[see JJ X-Ray note](#zoom_noteJJXray) + | Translation (across beam) | Sample Stack | | #1238 |[see SampleStack note](#zoom_noteSampleStack) +ISIS | Translation (along beam) | Sample Stack || #1238 |Driven by [Beckhoff](#zoom_noteBeckhoff) +ISIS | Two Tier sample changer | Sample Changer || |[see SampleChanger note](#zoom_noteSampleChanger) + | | Pneumatic Shutter | OMRON | |[see Pneumatic Shutter note](#zoom_notePneumaticShutter) +ISIS | Tank Translation | || |[see Tank Translation note](#zoom_noteTankTranslate)
Driven by [Beckhoff](#zoom_noteBeckhoff) + | | In/Out Monitor | | #267 |[see Monitor note](#zoom_noteMonitor) + | | JAWS | | #179, #180 | + | | Detector Motion System (DMS) | | #1236 |[see Detector Motion System (DMS) note](#zoom_noteDMS) +ISIS | DAE (2 or 3?) | Detector Electronics | Ethernet | #219 |[see DAE note](#zoom_noteDAE) +[CAEN](http://www.caen.it/csite/HomePage.jsp) | HT | Detector HV Control | Ethernet | #214 |[see CAEN HT note](#zoom_noteCAEN) +OMRON | | PLC | Ethernet | FINS, #217 |[see OMRON PLC note](#zoom_noteOMRON) +[Galil](http://www.galilmc.com/) | [DMC2280](http://www.galilmc.com/products/dmc-22x0.php) | Motion Controller | Ethernet | EPICS |[see Galil note](#zoom_noteGalil) +[Beckhoff](http://www.beckhoff.co.uk/) | | Motion Controller | TBC | |[see Beckhoff note](#zoom_noteBeckhoff) +[Pfeiffer](http://www.pfeiffer-vacuum.com/products/measurement/container.action) | TPG300 | ISIS VACUUM SYSTEM | RS232 | #218 |[see Pfeiffer note](#zoom_notePfeiffer) +[Julabo](http://www.julabo.com/us?p=1) | TBC | Water Bath | RS232 | #187, #188 |[see Julabo note](#zoom_noteJulabo) + | | PSU | TBC | |[see Power Supply Unit note](#zoom_notePSU) +[Oxford Instruments](http://www.oxford-instruments.com/products) | ITC503 | Temp Controller | RS232 | |[see Oxford Instruments Temperature Controller note](#zoom_noteOI_ITC503) +[Oxford Instruments](http://www.oxford-instruments.com/products) | ILM200 series | Helium Level Meter | RS232 | [EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Oxford%20Instruments) |[see Oxford Instruments Helium Level Meter note](#zoom_noteOI_ILM200) +[Eurotherm](http://www.eurotherm.co.uk/) | All models at ISIS | Temp Controller | RS232 | [ EPICS](http://www.aps.anl.gov/epics/modules/manufacturer.php#Eurotherm) |[see Eurotherm note](#zoom_noteEurotherm) +[Eurotherm](http://www.eurotherm.co.uk/) | Nanodac | Temp Controller | Ethernet| |[see Eurotherm note](#zoom_noteEurotherm) +[Thurlby](http://www.tti-test.com/) | [EX355P](http://www.tti-test.com/products-tti/text-pages/psu-ex355p.htm) | DC Power Supply | RS232 | #156 |[see Thurlby note](#zoom_noteThurlby) +[KEPCO](http://www.kepcopower.com/bop.htm) | TBC | Bi-Polar Power Supply | RS232 | #189 |[see KEPCO note](#zoom_noteKEPCO) +[Tektronix](http://www.tek.com/oscilloscope#all) | [MSO 4104B](http://www.tek.com/oscilloscope/mso4000-dpo4000) | Oscilloscope | Ethernet | #234 |[see Tektronix note](#zoom_noteTektronix) +[Tektronix](http://www.tek.com/function-generator) | [AFG 3022B](http://www.tek.com/datasheet/signal-generator/afg3000-function-generator-arbitrary-function-generators) | Function Generator | Ethernet | #237 |[see Tektronix note](#zoom_noteTektronix) +[PI](http://www.physikinstrumente.com/en/index.php) | Rotation Stage | Nano-positioning Rotation Stage | RS232 | |[see Physik Instrument note](#zoom_notePI_Nano) +[Bio-Logic](http://www.bio-logic.info/) | | Stopped Flow Cell | RS232 | |[see BioLogic note](#zoom_noteBioLogic) + | 17T Cryomag | Super conducting magnet | RS232 | | +Oxford Instruments | 7.5T Magnet | Super conducting magnet | RS232 | | [see 7.5T Magnet note](#zoom_note7p5TMagnet) +[Scientific Magnetics](http://www.scientificmagnetics.co.uk/) | | 3 Axis Super conducting magnet & VTI | RS232 | |[see Scientific Magnetics note](#zoom_noteSciMag) +[Lakeshore](http://www.lakeshore.com/Pages/Home.aspx) | All models at ISIS | | TBC | |[see Lakeshore note](#zoom_noteLakeshore) + | Syringe Pumps| | RS232 | |[see Syringe Pumps note](#zoom_noteSyringePumps) +~[Hitachi](http://www.hitachi-hightech.com/global/about/corporate/group/hhs/)~ | ~L-7100~ | ~HPLC Pump~ | ~RS232~ | |[see Hitachi note](#zoom_noteHitachi) +[TS-Haake](http://www.thermoscientific.com/en/products/baths.html) | | Water Bath | RS232 | |[see Thermo Scientific (Haake) note](#zoom_noteHaake) +[Huber](http://www.xhuber.de/en/home/) | | Sample Stack | RS232 | |[see Huber note](#zoom_noteHuber) +Thar | | Pressure Cell | TBC | |[see Thar note](#zoom_noteThar) +[Goudsmit](http://www.goudsmit-magnetics.nl/NL/) | 1T Magnet | Magnet | TBC | |[see Goudsmit note](#zoom_noteGoudsmit) +[Anton Paar](http://www.anton-paar.com/uk-en/products/group/rheometer/) | | Rheometer | | |[see Pfeiffer note](#zoom_noteAntonPaar) +ISIS | | Couette Cell || |[see ISIS Couette Cell note](#zoom_noteCouette) +[McLennan](http://www.mclennan.co.uk/) | | Motion Controller | RS232 | |[see McLennan note](#zoom_noteMcLennan) + | | T-Jump Cell | RS232 | |[see T-JumpCell note](#zoom_noteTJumpCell) + | Dynamic Light Scattering | DLS | TBC | |[see Dynamic Light Scattering (DLS) note](#zoom_noteDLS) + | | Raman Spectroscopy | TBC | |[see Raman Spectroscopy note](#zoom_noteRaman) +[Linkam](http://www.linkam.co.uk/) | | | RS232 | |[see Linkam note](#zoom_noteLinkam) +[Keithley](http://www.keithley.co.uk/) | All models in use at ISIS | | RS232, USB | |[see Keithley note](#zoom_noteKeithley) + | | Closed Cycle Refrigerator | via Eurotherm? | |[see CCR note](#zoom_noteCCR) + + +{#zoom_noteFastShutter} +##### Note: Fast Shutter ##### +Location: Front End
+Due: Day 1
+As used on: SANS2D
+ +{#zoom_noteDDChopper} +##### Note: ISIS Double Disk Chopper ##### +ISIS Mk3 Chopper
+Location: Front End
+Due: Day 1
+As used on: SANS2D
+ +{#zoom_noteJaws} +##### Note: Jaws ##### +Location: Front End Vacuum Tank
+Due: Day 1
+As used on: Many instruments
+Testing Summer 2015
+ +{#zoom_notePGC} +##### Note: Polariser, Guide & Collimator ##### +Location: Front End Vacuum Tank
+Due: Day 1
+1 horizontal motion axis controlling four 4m evacuated tubes
+ +{#zoom_noteSpinFlipper} +##### Note: SpinFlipper ##### +Location: Fixed Collimation
+Due: Day 1
+4m evacuated tube & Spin-flipper unit +Spin-flipper will be controlled by 2 Kepco Power Supplies
+ +{#zoom_noteLaser} +##### Note: Alignment Laser ##### +Location: Blockhouse
+Due: Day 1
+Single in/out axis
+No control of the alignment laser is required, other than the ability to move it in/out of the beam. It can treated like a monitor. + +{#zoom_noteJJXray} +##### Note: JJ X-Ray 7-axis Sample Stack ##### +Location: Blockhouse
+Due: Day 1
+[7-axis Sample Stack Manual](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Larmor/LARMOR%20JJXRAY%20SAMPLE%20STACK.pdf)
+See also [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteSampleStack} +##### Note: SampleStack ##### +Location: Blockhouse
+Due: Day 1
+The [JJ X-Ray sample stack](#zoom_noteJJXray) will sit on rails to allow the entire sample stack to be translated back & forth along the beam. The rails are not perfectly straight, so there needs to be an axis allowing motion perpendicular to the rails to correct for any slight misalignment in the rails. In other words, there are two axes for wholesale motion of the sample stack (in addition to the 7 internal axes of the sample stack itself). + + 1. Across beam (i.e. perpendicular to the rails): Alignment stage for reproducibility. + 2. Along beam (i.e. along the rails): Monitoring only, no control required.
+See [ZOOM_Sample_Stack_Schematic](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ZOOM/ZOOM_Sample_Stack_Schematic.pptx) + +{#zoom_notePneumaticShutter} +##### Note: Pneumatic Shutter ##### +Location: Blockhouse
+Due: Day 1
+As used on: SANS2D
+ +{#zoom_noteTankTranslate} +##### Note: Tank Translation ##### +Location: Blockhouse
+Due: Day 1
+The tank will be moved by a motor connected to a [Beckhoff](#zoom_noteBeckhoff) controller.
+Monitoring only, no control required. + +{#zoom_noteMonitor} +##### Note: Monitor ##### +Location: Tank
+Due: Day 1
+In/Out positioning - c.f. [LARMOR](LARMOR-Instrument-Details), [IMAT](IMAT-Instrument-Details),others
+ +{#zoom_noteDMS} +##### Note: Detector Motion System (DMS) ##### +Location: Tank
+Due: Day 1 + +7 axes consisting of: + 1. Baffle + 1. Detector translation + 1. Strip Beam Stop + 1. 2x Disk Beam stop - x and y movement (total: 4 axes) + +{#zoom_noteDAE} +##### Note: DAE ##### +Location: Screened Room
+Due: Day 1
+As used on: Many instruments
+ +{#zoom_noteCAEN} +##### Note: CAEN HT ##### +Location: Screened Room
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteOMRON} +##### Note: OMRON PLC ##### +Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details), [IMAT](IMAT-Instrument-Details)
+There are up to three functions for the PLC(s) - exactly which are to be included in the computing control is TBC. Current expectation is that the PLC will control: + 1. "Fast Shutter" (located at the front end of the beamline). SANS2D has a similar shutter. + 1. "Pneumatic Shutter" (located in the blockhouse). SANS2D has a similar shutter. + 1. [Pfeiffer](#zoom_notePfeiffer) TPG 300 pressure gauges + + +{#zoom_noteGalil} +##### Note: Galil ##### +Location: Blockhouse x 1, Mezzanine x 4
+Due: Day 1
+As used on: Many instruments
+There are at present 39 axes to be controlled via Galil
+ +{#zoom_noteBeckhoff} +##### Note: Beckhoff ##### +Location: TBC
+As used on: [IMAT](IMAT-Instrument-Details)
+The Beckhoff controller will control two axes: + 1. the motion of the JJ X-ray sample stack along the rails [see SampleStack](#zoom_noteSampleStack) + 1. the motion of the [tank](#zoom_noteTankTranslate) (enclosing the detectors & detector motion system). + +IBEX is **NOT** required to actually control these two axes (that will be done independently). IBEX is required to monitor the locations reported by the Beckhoff system - a read only driver will be required. + +{#zoom_notePfeiffer} +##### Note: Pfeiffer ##### +Location: TBC - alongside PLCs
+Due: Day 1
+As used on: [LARMOR](LARMOR-Instrument-Details), [IMAT](IMAT-Instrument-Details)
+ +{#zoom_noteJulabo} +##### Note: Julabo ##### +Location: TBC
+Due: Day 1
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_notePSU} +##### Note: Power Supply Unit ##### +Location: TBC
+Due: Day 1
+This is to control a magnet which will be linked to the interlocks, but there are no details at present
+ +{#zoom_noteOI_ITC503} +##### Note: Oxford Instruments Temperature Controller ##### +Model: ITC503
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteOI_ILM200} +##### Note: Oxford Instruments Helium Level Meter ##### +Model: ILM200 series
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteEurotherm} +##### Note: Eurotherm ##### +Model: all models at ISIS
+Due: Day 1
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+**Note:** ZOOM may also use a [Eurotherm Nanodac](http://www.eurotherm.co.uk/products/recorders/graphic/nanodac) to monitor detector temperatures. + +{#zoom_noteThurlby} +##### Note: Thurlby ##### +Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteKEPCO} +##### Note: KEPCO ##### +Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteTektronix} +##### Note: Tektronix ##### +1. Model: Oscilloscope +1. Model: Function Generator + +Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteSampleChanger} +##### Note: SampleChanger ##### +Location: Sample Area
+Due: Day 1
+Same as used on: SANS2D/[LARMOR](LARMOR-Instrument-Details), translations done via sample stack
+ +{#zoom_notePI_Nano} +##### Note: Physik Instrument ##### +Model: Nano-positioning Rotation Stage
+Location: TBC
+[PI Software](http://www.physikinstrumente.com/en/products/prdetail.php?sortnr=1100018), [EPICS driver](http://www.aps.anl.gov/epics/modules/manufacturer.php#Physik%20Instrumente)
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteBioLogic} +##### Note: BioLogic ##### +Model: Stopped Flow Cell
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteSciMag} +##### Note: Scientific Magnetics ##### +Model: 3 Axis Super conducting magnet & VTI
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteLakeshore} +##### Note: Lakeshore ##### +Model: all models at ISIS
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteSyringePumps} +##### Note: Pfeiffer ##### +Model: all models at ISIS
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteHitachi} +##### Note: Hitachi ##### +Hitachi L-7100 HPLC pump (no longer supported at ISIS). ~Can't find L-7100 on [Hitachi web-site](http://www.hitachi-hightech.com/global/about/corporate/group/hhs) (may be obsolete). Check existing SECI VI for logic and manual: both VI and manual are located here: `C:\LabVIEW Modules\Drivers\Hitachi L-7100`.~ + * Update (27-06-2019): Hitachi pumps are no longer used. No longer any need to support them. See [#3780](https://github.com/ISISComputingGroup/IBEX/issues/3780). + +{#zoom_noteHaake} +##### Note: Thermo Scientific (Haake) ##### +Model: Water bath
+Location: TBC
+As used on: [LARMOR](LARMOR-Instrument-Details)
+ +{#zoom_noteHuber} +##### Note: HUBER ##### +Model: Sample Stack
+As used on: SANS2D
+Location: TBC
+ +{#zoom_noteThar} +##### Note: Thar ##### +Model: Pressure Cell
+Location: TBC
+As used on: SANS2D, LOQ
+ +{#zoom_noteGoudsmit} +##### Note: Goudsmit ##### +Model: 1T Magnet
+Location: TBC
+As used on: SANS2D, LOQ
+ +{#zoom_noteAntonPaar} +##### Note: Pfeiffer ##### +Model: Rheometer
+Location: TBC
+As used on: SANS2D
+Currently controlled from a separate PC
+ +{#zoom_noteCouette} +##### Note: ISIS Couette Cell ##### +Location: TBC
+As used on LOQ
+ +{#zoom_noteMcLennan} +##### Note: McLennan ##### +Location: TBC
+Due: Day 1
+As used on: Many instruments
+ +{#zoom_noteTJumpCell} +##### Note: T-JumpCell ##### +Location: TBC
+As used on: LOQ
+ +{#zoom_noteDLS} +##### Note: Dynamic Light Scattering (DLS) ##### +Location: TBC
+As used on: SANS2D
+There is an interaction with a LabVIEW function that ought to be reproduced.
+ +{#zoom_noteRaman} +##### Note: Raman Spectroscopy ##### +Location: TBC
+As used on: INTER
+This may not have an existing interface, and may not need one
+ +{#zoom_noteLinkam} +##### Note: Linkam ##### +Location: TBC
+There are VIs for this
+ +{#zoom_noteKeithley} +##### Note: Keithley ##### +Model: all models in use at ISIS
+Location: TBC
+There are VIs for this
+ +{#zoom_noteCCR} +##### Note: CCR ##### +Model: TBC
+Location: TBC
+Details of [Closed-Cycle Refrigerators](http://www.isis.stfc.ac.uk/sample-environment/low-temperature/ccrs/) at ISIS. + +{#zoom_note7p5TMagnet} +##### Note: 7.5T Magnet ##### +Model: [7.5T Magnet](https://www.isis.stfc.ac.uk/Pages/75T-Magnet.aspx) \ No newline at end of file diff --git a/doc/processes/meetings/Backlog-Preparation.md b/doc/processes/meetings/Backlog-Preparation.md new file mode 100644 index 000000000..d92daf15a --- /dev/null +++ b/doc/processes/meetings/Backlog-Preparation.md @@ -0,0 +1,3 @@ +# Sprint pre-planning + +This meeting is often called pre-planning. This meeting is just a pre-runner to sprint planning and shares the same process see [Sprint Planning](Sprint-Planning). \ No newline at end of file diff --git a/doc/processes/meetings/Code-Chats-and-Lightning-Talks.md b/doc/processes/meetings/Code-Chats-and-Lightning-Talks.md new file mode 100644 index 000000000..f55acc4ae --- /dev/null +++ b/doc/processes/meetings/Code-Chats-and-Lightning-Talks.md @@ -0,0 +1,193 @@ +# Code chats + +Code chats occur on a semi-regular monthly basis within the group at ISIS. Although they were originally intended to be presentations focused on a specific part of the IBEX GUI they often cover other subjects that may be considered part of the backend, or talk about general coding practices. + +## Upcoming Code Chats + +Topic | Presenter | Date +----- | --------- | ---- + Basics of Epic DBs | TBD | TBD + + + +### Books and article ideas for technical book club + +- Peopleware +- Clean code +- The Pragmatic Programmer +- The Effective Engineer +- Design Patterns: Elements of Reusable Object-Oriented Software +- Refactoring +- The Agile Manifesto +- Fluent Python +- Effective Python + +### Proposed Future Code Chats + +Idea | Proposed Speaker +---- | --------------- +IBEX Project Management | Kathryn +LUA | ? +BDD/DDD | ? +Software design techniques in our processes | Aaron +The HIFI Cryomag | Lilith/Kathryn +Design Patterns in IBEX | ? +Datastreaming | Dolica/Jack/Matt +LVDCOM | Freddie +IBEX UI Design | Thomas Löhnert +Jenkins, testing and our build processes | ? +asyncio and asynchronous programming in Python 3 | ? +Script Server and Nicos | ? +CAEN v895 Discriminator | Freddie +MuSR conversion retrospective | Jack/Tom +BCC Cryogenics Course 2021 | ? +Java 17 LTS | TBC + +### Proposed Repeated Code Chats + +Idea | Proposed Speaker +---- | ---------------- +Observables and Writables in IBEX | ? +Dynamic Layouts in SWT | Thomas Löhnert +Databinding | ? +Mocking in Python and Java | ? +Checkstyle Quiz | ? +Maven, Tycho and Surefire | Tom Willemsen? +IOC Testing Framework & Emulators | Tom Willemsen? + +## Previous Code Chats + +The presentation slides for previous code chats are [available here](https://stfc365.sharepoint.com/sites/ISISExperimentControls/ICP%20Discussions/Forms/AllItems.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides&viewid=45d81cb9%2D9571%2D4a80%2Da5cc%2D2cb4871703a3). +
+ Previous code chats + + ### 2015 + +Topic | Presenter | Date +----- | --------- | ---- +Data Binding (two parts) (Contains info on MVVM) | Matt Clarke | Sept 2015 +Mockito | Ian Bush | Sept 2015 +Java Classes | Matt Clarke | Sept 2015 +Eclipse RCP (two parts) | Matt Clarke | Oct 2015 +Java Collections, Iterators and Streams | David Keymer | Nov 2015 +Design Patterns in IBEX | Dominic Oram | Dec 2015 +Eclipse Extensions | Dominic Oram | Dec 2015 +Observables and Writables in IBEX (two parts) | Ian Bush | Dec 2015 + +### 2016 +Topic | Presenter | Date +----- | --------- | ---- +Final | Matt Clarke | Jan 2016 +Java Quiz | Matt Clarke | Jan 2016 +Checkstyle Quiz | Dominic Oram | Jan 2016 +The SNS Scan Server | Dominic Oram | Jan 2016 +Testing with RCPTT (the basics) | Ian Bush | April 2016 +Maven and Tycho (two parts) | Matt Clarke | April 2016 +Javadoc | Kathryn Baker | Aug 2016 +Synchronized & Volatile | Thomas Lohnert | Aug 2016 +RCPTT - Tips and Tricks | Dominic Oram | Aug 2016 +User Interface Heuristics | Thomas Lohnert | Nov 2016 +How to Read ISIS LabVIEW Drivers | Kathryn Baker | Nov 2016 + +### 2017 +Topic | Presenter | Date +----- | --------- | ---- +MVVM Pattern | Dominic Oram | Feb 2017 +Collision Detection | Jon Elmer | Feb 2017 +IOC Testing Framework & Emulators | Tom Willemsen | Jul 2017 +Motion Control Axis Testing | Dominic Oram | Jul 2017 +Pythonic vs Non-Pythonic | Matt Clarke | Sep 2017 +Python Style Checking & Flake8 | Samuel Jackson | Sep 2017 +Eclipse 4 Migration | Adrian Potter | Sep 2017 +Alignment Automation on ALF | Thomas Löhnert | Sep 2017 +Code Reviews | Matt Clarke | Oct 2017 +Dynamic Layouts in SWT | Thomas Löhnert | Oct 2017 +The DAE | Freddie Akeroyd | Nov 2017 +The HIFI Cryomag | Eilidh Southren | Nov 2017 +Python 2 vs. Python 3 | David Keymer | Dec 2017 +The RB Numbering System | Matt Clarke | Dec 2017 + +### 2018 +Topic | Presenter | Date +----- | --------- | ---- +System Testing with Squish | Adrian Potter | Jan 2018 +Threading | Tom Willemsen | Jan 2018 +Galil Motors | Kathryn Baker | Feb 2018 +Grafana | Aidan McComb | Feb 2018 +AutoIt | Chris Moreton-Smith | Apr 2018 +Nicos | Dominic Oram | Apr 2018 +SScanSS | Stephen Nneji | Aug 2018 +Reflectometry | John Holt | Aug 2018 +Java 8 | Tom Willemsen | Sep 2018 +Introduction to Beckhoff Motors (Layers, Onions and Ogres) | Simon Cooper | Oct 2018 +Experience as a new starter | Aaron Long | Nov 2018 +Christmas Special: Bell ringing | Debbie Greenfield | Dec 2018 + +### 2019 +Topic | Presenter | Date +----- | --------- | ---- +Convert Record Changes | Liam Panchaud | Mar 2019 +Technical debt: Configurations | Thomas Löhnert | Mar 2019 +ACCU Conference Recap | Michal / Jack | April 2019 +Queued State Machine | Kathryn | April 2019 +Cryogenics | Tom | May 2019 +Beckhoff Code Camp Recap | Dom | May 2019 +EPICS Spring Meeting 2019 Recap | Aaron & David | June 2019 +Graduate Placement Retrospective | Sophie Kirkham | June 2019 +Controls at the ESS | Michael Hart | July 2019 +Datastreaming | Jack Harper | July 2019 +How data moves at ISIS | Chris | Sep 2019 +Migration to Python 3 | James | Sep 2019 +Java 11 | Tom | Oct 2019 +EPICS 7 | Freddie | Nov 2019 + +### 2020 +Topic | Presenter | Date +----- | --------- | ---- +The IBEX Script Generator | James | Jan '20 +NeXus | Freddie | 11th March '20 +System Testing With Squish | Dom | May '20 +How to Read ISIS LabVIEW Drivers | Kathryn Baker | 6th August '20 +Review techniques | Tom, Dom, James | 3rd September '20 +DAQMx and Moxa DIO devices | Alistair | 28th September '20 +Deploying EPICS with Ansible | CLF Controls team | 19th October '20 +Helium Level Monitoring | Kathryn/Mihai | 19th November '20 +Handover: Client Build System | Tom | 10th December '20 + +### 2021 +Topic | Presenter | Date +----- | --------- | ---- +Cryogenics handover | Tom Willemsen | 2nd February '21 +RIKEN refurbishment | Kevin | 8th of March 21 + +### 2022 +Topic | Presenter | Date | Recording +----- | --------- | ---- | ---------- +Github Universe | Jack Allen | 14/01/22 | Not Recorded +ISISICP/DAE | Freddie | 31/03/22 | [Recorded here.](https://stfc365-my.sharepoint.com/:v:/r/personal/jack_harper_stfc_ac_uk/Documents/Recordings/Code%20chat%20(ISISICP_DAE)-20220331_140431-Meeting%20Recording.mp4?csf=1&web=1&e=ccVfMJ) +Motion in Ibex| Jack Harper | 27/04/22 | Not Recorded +[Docusaurus](https://github.com/JackEAllen/Docusaurus_L_and_D) | Jack Allen | 24/06/22 | [Recorded here.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FDocusaurus%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) +SANS/TRANS Instrument Scripts | Thomas Cottee Meldrum | 21/07/22 | [Recorded here.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FSANS%20TRANS%20Instrument%20Scripts%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) +IOC Testing Framework & Emulators | Tom Willemsen | 23/08/22 | [Recorded here.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FEmulators%20and%20IOC%20Testing%20Framework%20Codechat%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) + Ibex Gui Build System (Maven) | Tom Willemsen | 22/11/22 | [Recorded here.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FIBEX%20GUI%20build%20System%20Code%20chat%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) + +### 2024 +Topic | Presenter | Date | Recording +----- | --------- | ---- | ---------- +Java Debug Option & the GUI Singleton | Sudeepta Chakraborty | 08/03/24|[Part 1.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FCode%5FChat%5FJava%5FDebug%5FOption%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) [Part 2.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FCode%5FChat%5FThe%5FGUI%5FSingleton%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview) +How Virtual Machines and VHDs can speed up IBEX deployment | Chris | 29/04/24 | [Recorded here.](https://stfc365.sharepoint.com/sites/ISISExperimentControls/_layouts/15/stream.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides%2FCode%5Fchat%5FHow%5FVirtual%5FMachines%5Fand%5FVHDs%5Fcan%5Fspeed%5Fup%5FIBEX%5Fdeployment%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview%2E5b2105eb%2D3914%2D4d48%2Da5db%2D1f77539e7b7b) + Bluesky | Tom Willemsen | 21/08 | [Slides](https://stfc365.sharepoint.com/sites/ISISExperimentControls/ICP%20Discussions/GUI_Chat_Slides/Bluesky.pptx) [talk](https://stfc365.sharepoint.com/:v:/r/sites/ISISExperimentControls/ICP%20Discussions/GUI_Chat_Slides/Bluesky.mp4?csf=1&web=1&e=IVGFha) | + + +
+ +## Lightning Talks +Lightning talks are a chance to share smaller bits of interesting information after standup at the end of a sprint. +Topic | Presenter | Date +----- | --------- | ---- +Rust | Jack Harper | 23/03/2022 +Git Interactive Staging| Lowri Jenkins | 10/08/2022 +Speeduino| Jack Harper | 10/08/2022 + +All presentations should be uploaded to the SharePoint site [here](https://stfc365.sharepoint.com/sites/ISISExperimentControls/ICP%20Discussions/Forms/AllItems.aspx?id=%2Fsites%2FISISExperimentControls%2FICP%20Discussions%2FGUI%5FChat%5FSlides&viewid=45d81cb9%2D9571%2D4a80%2Da5cc%2D2cb4871703a3) (There is a subfolder for lightning talks) and/or converted to a wiki page if more appropriate. It should also be reasonably assumed that the people who gave these presentations should know something about these subjects. + diff --git a/doc/processes/meetings/Meeting-Roles-and-Rotas.md b/doc/processes/meetings/Meeting-Roles-and-Rotas.md new file mode 100644 index 000000000..12d758304 --- /dev/null +++ b/doc/processes/meetings/Meeting-Roles-and-Rotas.md @@ -0,0 +1,108 @@ +# Meeting roles & rotas + +It was decided that we would set up a rota for the various roles in IBEX meetings, a rota is below. There is only a single rota for each type of role, although not all roles are needed at all meetings, there is a column then to indicate where in the rota we are. If the person scheduled on the rota isn't available, then a volunteer will be asked for at the beginning of the meeting and we will continue the rota next time. If you are scheduled for more than one role choose which one you do this time, and you get to do the other one next time. At the moment this should cover for 6 months, and it is not a repeating pattern, and it is subject to change. + +## Role descriptions: +### Chair +* Needed for Pre-Planning, Review/Retrospective, and Planning +* Make sure things happen as per the ticket workflow details for meetings, see the [processes](/Processes) page for more details (if they are out of date, update them) +* Keep the meetings on track (as much as possible) + +### Timekeeper in Planning: +* Limit discussion of tickets to a maximum of 5 minutes per ticket. +* Act as moderator for the voting site, deleting estimates between votes, show votes when everyone is ready, and giving brief reminders if some time has passed and some votes have not been cast. +* Keep an eye on the end time of the meeting - give everyone a five minute warning +* If the meeting is scheduled for more than an hour to indicate the halfway time to make sure we take a break + +### Timekeeper in Review +* In Review ensure there is enough time for everyone to speak in the hour - this would be an average of 5 minutes per person if everyone was planning to demo +* Keep an eye on the end time of the meeting - give everyone a five minute warning +* If the meeting is scheduled for more than an hour to indicate the halfway time to make sure we take a break + +### Note taker in Retrospective: +* Keep notes of the discussed points +* Add them to the wiki with a link on the [main retrospective notes page](../Retrospective-Notes) + +### Note taker in Planning: +* Make notes on tickets of any important information which gets discussed but is not already mentioned on the ticket + * This should include acceptance criteria discussed. +* Add a timestamp to tickets of when in the recording the ticket was discussed. + +### Stand up: +* This is a weekly role starting on the first working day in that week, with substitutes being decided daily if the named person is unavailable last minute - if you know you are unavailable for the week swap with someone +* Load and share the web pages as detailed in the tabbed note in the announce channel in the team +* Ensure everyone present has their chance to share progress, plans, and obstacles +* If you're able to, writing a squish test to replace one of the [Manual system tests](/deployment/Manual-System-Tests) would be great. If the person running the week before has done so and it hasn't been reviewed, now would be a good time to review that as well. + +## Rotas + + | Standup | Week Commencing | x for Done | + |--- | --- | --- | + |IG | 04/11/2024 |x | + |TW | 11/11/2024 |x | + |ES | 18/11/2024 |x | + |JD | 25/11/2024 |x | + |LC | 02/12/2024 |x | + |KB | 09/12/2024 |x | + |DK | 16/12/2024 |x | + |JH | 23/12/2024 |x | + |RB | 30/12/2024 |x | + |DM | 06/01/2025 |x | + |LJ | 13/01/2025 |x | + |CMS | 20/01/2025 |x | + |FA | 27/01/2025 |x | + |SC | 03/02/2025 |x | + |GR | 10/02/2025 |x | + |LC | 17/02/2025 |x | + |GR | 24/02/2025 |x | + |KB | 03/03/2025 |x | + |JD | 10/03/2025 |x | + |TW | 17/03/2025 |x | + |LJ | 24/03/2025 |x | + |ES | 31/03/2025 |x | + |DM | 07/04/2025 |x | + |IG | 14/04/2025 |x | + |CMS | 21/04/2025 |x | + |LC | 28/04/2025 |x | + |JH | 05/05/2025 | | + |GR | 12/05/2025 | | + |IG | 19/05/2025 | | + |KB | 26/05/2025 | | + |FA | 02/06/2025 | | + + + + +*** + +## Sprint Meetings +| Meeting| Chair | x for Done | Timekeeper | x for Done | Note taker | x for Done | +| ---| --- | --- | ---| --- | --- | --- | +| 2025_03_05_Planning| JH |x | TW|x | IG|x | +| 2025_03_05_Review/Retro| JD | x | DM| x | SC| x | +| 2025_04_10_Planning| TW | x | JD| x | DM| x | +| 2025_04_10_Review/Retro| FA | | KB| | DK| | +| 2025_05_08_Planning| JD | | LJ| | GR| | +| 2025_05_08_Review/Retro| KB | | LC| | ES| | +| 2025_06_12_Planning| LC | | CMS| | JH| | +| 2025_06_12_Review/Retro| DM | | IG| | LJ| | +| 2025_07_10_Planning| IG | | SC| | TW| | +| 2025_07_10_Review/Retro| SC | | ES| | DM| | +| 2025_07_31_Planning| ES | | GR| | SC| | +| 2025_07_31_Review/Retro| GR | | JH| | LC| | +| 2025_08_28_Planning| DM | | DK| | LJ| | +| 2025_08_28_Review/Retro| IH | | FA| | IG| | +| 2025_10_02_Planning| TW | | DM| | DK| | +| 2025_10_02_Review/Retro| CMS | | TW| | GR| | +| 2025_11_27_Planning| GR | | --| | ES| | +| 2025_11_27_Review/Retro| FA | | LJ| | --| | +| 2026_01_08_Planning| IG | | LC| | LJ| | +| 2026_01_08_Review/Retro| -- | | CMS| | TW| | +| 2026_02_05_Planning| JH | | IG| | SC| | +| 2026_02_05_Review/Retro| DK | | SC| | DM| | +| 2026_03_05_Planning| LC | | ES| | JH| | + + + + + diff --git a/doc/processes/meetings/Program-Increment-Planning.md b/doc/processes/meetings/Program-Increment-Planning.md new file mode 100644 index 000000000..e42ccb8e4 --- /dev/null +++ b/doc/processes/meetings/Program-Increment-Planning.md @@ -0,0 +1,30 @@ +# Program Increment Planning + +This meeting covers the overarching themes for the next few months. It will usually be held in the week or so before the first sprint of the Program Increment (PI) in question. This is an opportunity to ensure the whole team are aware of the more mid term goals of the team. + +A PI will consist of the sprints between this major release and the next one, these will typically be in January/February and August/September. + +In the first sprint in a PI we will create and deploy a release of the work we have just completed. The deploy should occur so that the instrument scientists have at least two full weeks of testing time available before the start of the user cycle. In the third sprint we will undertake dependency updates. + +This meeting will usually be chaired by the project manager for the group or other senior team member. + +### Preparation + +There is a [template](https://github.com/orgs/ISISComputingGroup/projects?query=is%3Aopen+is%3Atemplate) for the PI board into which can be brought the tickets that relate to the work to be undertaken. +Start by using that template to create a project for the coming PI. +Edit the setting to add the sprints to the appropriate field. Assign the pre-populated tickets to the appropriate sprint. +Add the items to be considered in the upcoming PI that are either a significant amount of work, or which the instrument scientists would appreciate visibility of. These can be added as either a link to a ticket in one of the repos, or a note on this board. These will typically have a classification of `Project/Feature`. It is wise to add sub-tickets of these umbrellas as well, as `To do for a Project/Feature`, as if there are dependencies between them they can be assigned to sprints as appropriate. +Items to consider may be carried over from the previous PI, or be new items. +Assign a priority order to the projects and features, and any sub-tickets which have been added so that they can be grouped easily. + +### During Meeting + +Review the PI about to end, see what is complete and what isn't, and discuss if appropriate. +Ask what has gone well or badly during the PI about to end, as a small retrospective. +Looking to the PI about to start, load each project/feature from the `Prioritising Table - Projects/Features` tab, so that the aims and purposes of each piece of work can be discussed and agreed. Care should be taken that the details of code and implementation should not be undertaken here too thoroughly. +Where items can be assigned to a specific sprint this should be done. If any priorities are wildly inappropriate that can also be discussed in this meeting, although care should be taken not to become too detailed. + +### Extra Notes +#### Preparation = Priority Ordering +Each project/feature should have a number which increments in the style 100, 200, etc. so that the sub-tickets (which could have further sub-tickets) can be numbered 110, 120, 130. If you have anything with more than 9 sub-tickets, or a third layer, increase at the highest level by an order of 10, so that the project would be 1000, the sub-tickets then become 1100, 1110, 1120, 1130 etc. Using this order makes it easier to rearrange, as if item 130 should fit between 110 and 120, simply setting it to 115 will move it to the right place. +After the meeting these numbers can be tidied up appropriately. \ No newline at end of file diff --git a/doc/processes/meetings/Sprint-Planning.md b/doc/processes/meetings/Sprint-Planning.md new file mode 100644 index 000000000..bedc8b712 --- /dev/null +++ b/doc/processes/meetings/Sprint-Planning.md @@ -0,0 +1,30 @@ +# Sprint Planning + +Sprint planning is the meeting where we decide what will appear in next sprint. We assign these roughly to high/medium/low priority tickets, and give them a `points` value to indicate a rough estimate of effort involved in those items. This is all done through the current Program Increment (PI) project on GitHub. + +### Preparation + +All team members should add the proposal label to tickets that should be considered whenever they are aware of this. These will automatically be added to the appropriate Program Increment project in GitHub. +The project manager (or an agreed alternative team member) should do the following prior to the meeting, either late the day before, or the morning of the meeting: +1. Ensure that all proposed items are in the appropriate `Sprint`, most easily done on the `PI Summary` tab. +1. On the `Breakdown by Sprint` board, check the items in the `No Sprint` column to see if they should be assigned to the next sprint - these are in priority order for the PI. +1. Update the `Planning Board` and `Planning Table` tabs to filter on the Sprint to be planned. +1. Any items from the current sprint in the backlog, but without a rework label, are assigned to the Sprint to be planned. +1. Create a support ticket based on the contents of the support channel (adding one tally for each thread, and for every multiple of 10 in the comments, so a thread with 23 comments would get 3 tally marks), and assign it to ISIS Builder with the appropriate number of points +1. Remove the automation from the `Review Complete` column of the IBEX board, rename that column to be the name of the sprint ending +1. Create a new `Review Complete` column which is automated to the `Done` type based on the closing of a ticket + +### During meeting +1. Post a link to the planning poker site and a reminder of the points estimates to the chat in zoom. This information is available in the `Planning` tab on the IBEX Developers MS Teams `announce` channel. +1. For each ticket in proposed (this should be kept to a maximum of 5 minutes per ticket if possible): + 1. Read the ticket and understand the issue + 1. Make sure the acceptance criteria makes sense + 1. Select the priority of the ticket high/medium/low and move to that column + 1. If a ticket is to be delayed to a later sprint in this PI, change the assignment of the ticket + 1. If a ticket should be delayed to the next PI, assign it to the appropriate Sprint + 1. If a ticket is put on hold for longer than that, simply remove it from the PI project + 1. Estimate the ticket using planning poker site; there is no need if it is already pointed; Points should be provided as a label at the minimum during the meeting, the field in the project is not in use as of May 2024 when this was updated + +### After the meeting + +1. Check with Kathryn, this process is currently being updated. diff --git a/doc/processes/meetings/Sprint-Review-and-Retro-slides.png b/doc/processes/meetings/Sprint-Review-and-Retro-slides.png new file mode 100644 index 000000000..677ccb13d Binary files /dev/null and b/doc/processes/meetings/Sprint-Review-and-Retro-slides.png differ diff --git a/doc/processes/meetings/Sprint-Review-and-Retro.md b/doc/processes/meetings/Sprint-Review-and-Retro.md new file mode 100644 index 000000000..832092cd4 --- /dev/null +++ b/doc/processes/meetings/Sprint-Review-and-Retro.md @@ -0,0 +1,29 @@ +# Sprint Review & Retrospective + +### Preparation + +![image](Sprint-Review-and-Retro-slides.png) + +1. Before the end of the sprint copy the template file in the the Files are of our General channel, update the name and details as appropriate and pin it to the tabs +1. Unpin the previous sprint review PowerPoint and move it into the `Past Sprint Reviews` folder +1. Let everyone know that the presentation has been created +1. On the morning of the review meeting, remind people to add slides to the presentation + +### During the Meeting + +1. Go through each person's slide and get them to demo and mention tickets they have done +1. Discuss any issues and goodness in the code +1. Review the [retrospective notes](../Retrospective-Notes) from the previous meeting, which will be found at the top of the page. The items to discuss will be located under 'Items from this retrospective:' and 'Other comments (Mad/Glad/Sad)' + +1. Open up Teams > IBEX Developers > retrospective and discuss any items below the relevant post: + + --------------------- retrospective stuff discussed above here --/--/---- ------------------------------ + +1. Discuss any Mad/Glad/Sad comments within the team +1. Discuss any final comments or thoughts that do not fit within the scope of Mad/Glad/Sad + + +### After the Meeting + +1. Follow up any issues that were found by creating new tickets if needed + diff --git a/doc/processes/meetings/Technical-Debt-Stand-down.md b/doc/processes/meetings/Technical-Debt-Stand-down.md new file mode 100644 index 000000000..fca8022bb --- /dev/null +++ b/doc/processes/meetings/Technical-Debt-Stand-down.md @@ -0,0 +1,58 @@ +# Technical Debt Stand-down + +A technical debt stand-down day is a day devoted to fixing technical debt in the system. The format is: + +1. Choose a day where most people are free +1. Choose a topic (See table below) +1. On the day, people pick up tickets when they get in, often working in pairs. There is an extra standup at 4pm to discuss what has been done. This is the moment to choose to either: + - stop working on a ticket (commit everything to a branch and add it to the ticket to be picked up later) + - move it into the current sprint + - ensure a ticket is reviewed + +Try to make sure all tickets by the end of the day are in a good state. + +### Ideas for Topics + + +- Fix script generator errors: https://github.com/ISISComputingGroup/IBEX/issues/5500 +- Emulator for the compressors +- Improve genie python unit tests coverage. It is lacking in several complicated pieces of code, such as the genie_change_cache module. +- Add to IOC numbers so that all IOCs have at least 2 +- Alarming + * Web does the same GUI + * Remove invalid alarm + * All IOCs should have an beast alarm +- Squish tests [#3707](https://github.com/ISISComputingGroup/IBEX/issues/3707) +- Eclipse/CSS interaction +- Tech debt tickets +- Block server exception handling +- Deployment/release process + - Automatically add release numbers based on the name of the release branch + - On install if only one component is release then fall back to later components e.g. if only 4.4.1 GUI is available install 4.4.0 server and genie_python + - Automate some manual steps +- Bugs +- Emulator/IOC tests +- Excess logging +- Dataweb/dashboard improvements +- Server submodule clean + - Add readmes to submodules in the server + - Remove unneeded submodules. + - Create automated dependency map + - Think about moving ISIS things to support +- EPICS Make clean uninstall can be successfully performed if old git files exist +- Fix IOC generator +- `genie_python` unit test coverage +- `dbchecker` vs `dbunitchecker` vs `dbchanges` +- Python 3 Compatibility +- User manual updates e.g. https://github.com/ISISComputingGroup/IBEX/issues/2763 and https://github.com/ISISComputingGroup/IBEX/issues/3814 +- Update the specific device IOC pages see [here](../retrospective-notes/Retrospective-Notes-2020.05.28) +- Tidy up code, remove unneeded parts + - Discuss what to do with pv sets +- Fix epics unit tests (https://github.com/ISISComputingGroup/IBEX/issues/5539) + +### Missing Documentation + +Section to document missing documentation. + +- Area detector [#4362](https://github.com/ISISComputingGroup/IBEX/issues/4362) + diff --git a/doc/processes/meetings_with_scientists/Demo-notes.md b/doc/processes/meetings_with_scientists/Demo-notes.md new file mode 100644 index 000000000..123afa216 --- /dev/null +++ b/doc/processes/meetings_with_scientists/Demo-notes.md @@ -0,0 +1,184 @@ +# First-time demos of IBEX to scientists + +```{note} +All instruments now have IBEX, so this process is unlikely to be needed for existing instruments. + +For existing instruments, use [regular demo documentation](Regular-Demos-of-IBEX-to-Scientists) +``` + +Anyone doing a demo of IBEX should use these notes, so that + +* You are familiar with the scope of the demo (i.e. what should be covered) +* You review it and rehearse it in advance (i.e. you are clear about what you will present) +* You don’t forget to mention any important aspects of the system + * keep a copy of these notes in front of you as you do the demo +* You don’t forget where key files (e.g. demo scripts) are located, etc. +* Instrument scientists get a consistent picture irrespective of where & when the demo is performed. +* If a question takes you “off piste”, you can easily find your way back to the main thread of the demo. +* We can enhance the script in the light of experience (e.g. better answers to questions) + * jot down any important points arising from any demos that you do and add them back into this document for the benefit of others. + +## Overview + +Provide the audience with a quick tour of the IBEX GUI + +* Explain that the GUI is a client application + * It displays data that is provided to it by server applications + * The server applications are called IOCs (Input-Output Controllers) + * IOCs are similar, in some respects, to LabVIEW VIs + * The IOCs run on the control PC (i.e. the NDX... machine) + * The GUI can run on the control PC. It can also run on other client PCs located elsewhere on the network. + +* Highlight the main areas of the GUI + * Dashboard - very similar to the SECI dashboard + * Blocks & Groups - again, very similar to SECI + * System Views - a number of different views of the system. Some views are similar to SECI; others are new. + * Clicking on the list of View buttons (on the left-hand side of the GUI window) changes the view + +* Highlight the IBEX menu + * IBEX - use to switch instrument, restore the default view or exit IBEX + * Configuration - use to edit configurations, load a new configuration or delete a configuration + * Synoptic - use to edit synoptics, load a new synoptic or delete a synoptic. + * IOC - use to start or stop IOCs + * Run Control - use to view and edit run-control settings + * Help - point out the About Box. User should use it to report the server & client version numbers if they encounter problems. + +## Dashboard + +The dashboard is very similar to the SECI dashboard +* It uses the same colour coding as the SECI dashboard +* It displays the same status messages as the SECI dashboard +* Information about the current experiment comes from the DAE set up (we'll touch on this later) +* Beam status information comes from the beam logger IOC + +## Beam Status + +Describe the Beam Status view +* Clicking on the Beam Status button displays the Beam Status view +* The Beam Status view always appears in the main display area +* The Beam Status view comprises 3 panes + * The main pane, taking up most of the display in the Beam Status graph. It shows the Synchrotron current and the currents to TS1 and TS2. You can toggle the display to show graphs for the last hour or last 24 hours. + * The top right pane shows the ISIS MCR news. It is just a simple scrolling window showing the published MCR news. + * The lower right pane shows beam status details. It consists of 3 sections: one for the Synchrotron, one for TS1 and one for TS2. The sections are collapsible, so you can collapse any section that is of no interest to you. + * Beam status information comes from the beam logger IOC, which replaces the old LabVIEW beam logger VI. + * The beam logger IOC is already serving beam status information to all instruments (not just those running IBEX) + * The beam logger IOC actually runs on the accelerator control system + +## IOC Log View + +Describe the IOC Log view +* The IOC Log system provides a unified way of managing status and information messages from the devices attached to the instrument. +* Copies of status and information messages from the DAE are also routed via the IOC Log system. +* The IOC Log messages are primarily intended for diagnosing problems with attached devices. +* The IOC Log view provides you with a convenient means of viewing those messages + * Demonstrate how messages can be filtered and searched (by date/time, severity, etc.) + * Messages are automatically saved to a log on the control PC + * **N.B.** Messages are held in a log which is quite separate from experimental data + +## DAE View + +Describe the DAE view +* The DAE view allows scientists to set up and control the operation of the DAE. +* The DAE view is very similar to the DAE VI in SECI. +* The DAE view comprises 6 tabs. + * Run Summary + Displays run summary information and any recent log messages. + * Experiment Setup + . + * Run Information + . + * Spectra Plots + . + * Diagnostics + . + * Vetos + . + +* As previously noted, copies of status and information messages from the DAE are also routed via the IOC Log system. + +## Blocks & Groups + +Blocks & groups in the new GUI work in the same way as they do in SECI. +* Blocks and groups are displayed at the top (and centre-right) of the GUI window. +* Blocks & groups are part of your configuration. You define blocks & groups by creating and editing configurations. +* In EPICS, an IOC makes information about the status of a device available by publishing *process variables*. In general, an IOC will use many process variables to fully describe the state of a device. Typically, you won't be interested in all of these - just a sub-set. Show how the GUI allows you to select which process variables that are of interest and to define these as *blocks*. +* Once a PV has been defined as a block, show how it can be assigned to a group. +* Make it clear that blocks are automatically logged (just as they are in SECI). Note also that the logging of blocks is independent of the message log. +* Show how blocks can be viewed in the LogPlotter - so the user can always view the history of a block. +* Show how a set of defined blocks & groups can be saved as a configuration. +* Show how a previously saved configuration can be re-loaded. +* Explain what is saved in a configuration: + * groups, blocks, links to settings files, synoptic view +* Demonstrate sub-configurations +* Demonstrate Management Mode. + +## Scripting + +In the new GUI, scripting is built-in. You don't need a separate windows to run a script (although some users prefer to have a separate window). +* Click on the Scripting button to display the Scripting view + * point out the scripting area +* The scripting language is Python +* Python brings a number of advantages over Genie + * *what are they? - we need to enumerate them* +* Python equivalents of the most common Genie commands (e.g. cset) have been created + * demonstrate that, for example, ``cset arg`` becomes ``cset(arg)`` + * it is not difficult to convert a Genie script into a Python script + * display a *before (Genie)* and *after (Python)* script. Emphasize how similar they are. +* Any block (or PV) can be used in a Python script +* Demonstrate running a Python script + * use a script that you prepared earlier: **don't wing it!** + * point out use of Genie-like commands where possible + * show how to pull in data from blocks (or PVs) + * read data from previously saved file if beam is off + * manipulate data in Python - show off what Python can do + * display data in a graph + * copy/paste a graph into MS Word (or similar) + +## Device Support + +EPICS provides support for a wide range of devices. For many devices we can simply download the appropriate driver from the EPICS web-site. Because the source code for these drivers is available, we can, if necessary, adapt an EPICS driver for our specific purposes. +* In EPICS, drivers & IOCs combine to provide control of devices attached an instrument. In addition, we use CSS to create OPIs (graphical user interfaces) for devices. In combination, drivers, IOCs and OPIs provide the functionality of a LabVIEW VI. +* We currently have EPICS drivers for Galil motor controllers, Eurotherms, Julabo water baths, CAEN, Kepco & Thurlby power supplies. We also have controllers for PLCs, jaws-sets, sample changers, rotation stages and goniometers. +* EPICS drivers and controllers for additional devices can be developed as required. +* Adopting EPICS does **not** mean throwing away existing LabVIEW drivers. + * We have created a communication interface, called lvDCOM, to enable EPICS to communicate with LabVIEW VIs. + * Where a device has a complex or specially customised LabVIEW driver, lvDCOM allows us to continue using the LabVIEW VI. Any data that the VI makes available to EPICS can be defined as a block and used in the normal manner. +* Demonstrate an existing EPICS IOC/OPI: for example the Eurotherm IOC & OPI – demonstrate simple operation, ramping, calibration, etc. +* Demonstrate interaction between EPICS and a VI: for example +* Demonstrate starting & stopping of IOCs. Show how startup messages, etc. are captured in the message log. +* Discuss settings files and their relation to IOCs and VIs. + +## Synoptic View or Instrument Overview + +The synoptic view provides an interactive overview of an instrument. +* The synoptic view shows all the devices (or a defined sub-set of devices) attached to an instrument. +* The synoptic view is configurable (via the Synoptic Editor). Show how you can create different views for different instrument setups. +* Incident neutrons are shown coming from the left. +* Each device is represented by its own icon and a small number of key parameters. +* Show how you can drill-down by clicking on each device icon to get more detail. + * Clicking on a device icon displays the device UI, which will display additional controls and information that cannot easily be included on the top-level synoptic display + * Device UIs are sometimes referred to as OPIs (OPIs are analogous to LabVIEW VI screens) + * Demonstrate how some devices (e.g. motors) allow you to drill-down to deeper levels (beware: some of the deeper level screens can get very complex) + * Show how the Up, Next and Prev buttons are used to navigate around the various layers of the synoptic view. + +## Motors View + +The motors view provides a convenient way to view the status of all the motors used on an instrument. +* The motors view is very similar to the *Table of Motors* used in SECI by some instruments. +* Each row corresponds to a single Galil motor controller. The number of rows displayed will depend on the number of Galil controllers attached to the instrument. +* The columns list the Galil controller ports (up to 8 ports per Galil) +* Each entry in the table represents a single Galil controller port (one (or zero) motors per port) + * Each entry identifies the motor by its unique ID number + * Each entry displays the current value of the motor's position and its current set-point. + * A grey entry represents an unused port (no motor is attached, or motor is switched off) + * A pink entry represents a port in use: motor is attached, but stationary + * A green entry represents an active port: motor is attached and is moving +* Show how you can drill-down by clicking on each motor entry to get more detail. + +## EPICS & Extensibility + +EPICS is a framework for creating *distributed* control systems. The ability to have more than one PC controlling different parts of an instrument is inherent to EPICS. +* The default position will continue to be that we will use a single PC to control all of the devices attached an instrument. +* However, there are some situations where it is advantageous to have specialist items of equipment controlled by a separate PC. For example + * the interchangeable cameras on IMAT + * the LARMOR spin-echo system \ No newline at end of file diff --git a/doc/processes/meetings_with_scientists/Regular-Demos-of-IBEX-to-Scientists.md b/doc/processes/meetings_with_scientists/Regular-Demos-of-IBEX-to-Scientists.md new file mode 100644 index 000000000..eb8b5f242 --- /dev/null +++ b/doc/processes/meetings_with_scientists/Regular-Demos-of-IBEX-to-Scientists.md @@ -0,0 +1,15 @@ +# Regular Demos of IBEX to Scientists + +The regular 4-week Sprint Review is not the ideal forum for demonstrating new developments in IBEX to scientists. Instead, we will demonstrate each new release of IBEX prior to its deployment to groups of instrument scientists (or, when appropriate, to individual instrument scientists). Demos will be arranged as follows: +1. Developers pair off to demonstrate IBEX to each group of instrument scientists. +1. One member of each pair takes the initiative to contact the scientists to arrange the demo. +1. Ideally, arrange the the demo to happen out of cycle and near the end of a sprint (that’s what is most convenient to both scientists & developers), but demos can be held at other times, if necessary. +1. Choose a convenient location to perform the demo (e.g. in the instrument cabin or a scientist's office (if there is room), or a meeting room) +1. Demo all the changes since the last release (or previous demo), with emphasis on those items of most relevance to the instrument +1. Gather and share feedback + 1. Create tickets for issues; if it has already been asked for add the `re-request` label and a comment of which instrument re-requested it. You will probably also want to add a proposal flag. + +Over time we can change the pairs, so that everybody gets exposed to different instruments & instrument scientists. + +Use the [sprint/release demos](https://github.com/ISISComputingGroup/IBEX/wiki/Timetable-for-Instrument-demos) page to plan each round of demos.. + diff --git a/doc/processes/meetings_with_scientists/Training-Instrument-Scientists-in-IBEX.md b/doc/processes/meetings_with_scientists/Training-Instrument-Scientists-in-IBEX.md new file mode 100644 index 000000000..2ee5397d2 --- /dev/null +++ b/doc/processes/meetings_with_scientists/Training-Instrument-Scientists-in-IBEX.md @@ -0,0 +1,173 @@ +# Training Instrument Scientists in IBEX + +From time-to-time, we run IBEX and genie_python training courses. This page collects information that you will need to organise and run the IBEX and genie_python train courses. + +## When should we run IBEX and genie_python train courses? +There is never an ideal time to run IBEX and genie_python train courses. Running training courses during cycle is difficult, because scientists are usually heavily involved with their instruments. Running training courses out of cycle is difficult, because scientists use the opportunity to take holidays, attend conferences, catch up on other work ... + +The best compromise is to run IBEX and genie_python train courses during the shutdown, shortly before the start of a new cycle. Why shortly before the start of a new cycle? Because, people tend to forget what they have learned unless they have an opportunity to put it into practice. So, as a rule of thumb, try to schedule the course 2-3 weeks before the start of cycle. You could also run the courses during the week before cycle, but you might need this time to make final adjustments to instruments, so try to avoid this time if possible. + +Because training has to be synchronised with ISIS cycles, you need to think about planning training sessions well in advance - 3-6 months in advance. + +## Recording that a Person would like to attend the next Training Session + +We record people who would like to attend the next training session by listing them on a ticket entitled "TRAINING: Invites for next session" + +## Training materials + +- [Mantid](https://www.mantidproject.org/Documentation) +- [IBEX](/overview/Links-and-Resources) + - There are various slides at the end of the material: "Reference card", "GUI Reference Card" and "Useful Locations". These are supposed to provide a brief reference of the major IBEX features, GUI elements and a memo sheet for folder locations respectively. Print off a copy of each for each delegate. +- [genie_python](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/genie_python-and-IBEX-%28Introduction%29) + +## Planning the IBEX and genie_python training courses +This section summarises the steps you will need to consider when planning the IBEX and genie_python training courses. + +### Check with the Mantid team +The Mantid team regularly run a Mantid course and an Introduction to Python courses. Check with the Mantid team to discover their plans. You can often collaborate of training activities. It is very useful if the Mantid team can deliver the Introduction to Python course, prior to the genie_python with IBEX course. + +### Select a date for the training +As noted above, select a date for the training 2-3 weeks ahead of the cycle. You may need to select 2 or 3 dates and only make a final decision when you know availability of the venue. You might also need to book a day or so either side of your chosen date, to allow for set-up and clear-up before/after the course. + +Typically we run the introduction as 9:00-12:00, and Introduction to `genie_python` as 13:00-17:00. This has worked well in the past. We have experimented with balancing the time as 9:00-12:30 and 13:00-17:00 but it made the morning feel long and the afternoon feel short. + +### Advertise the training course +Tell instrument scientists and visiting scientists of the proposed dates for the training course and invite them to sign up to attend. Scientists will typically need 2-3 months advance notice, so make sure you tell them early. Make sure you tell scientists to respond by a cut-off date - otherwise, it can be tricky trying to squeeze people in if they respond only 2-3 days before the course is due to run. +1. During this step check for general training tickets on the backlog because any people who need to be invited will be listed here. + +### Decide which training venue to use +Recent training courses have been presented in R80 (rooms CR16 & CR17) and in the Central Design Facility (CDF) suite in the Atlas Building. The CDF is a dedicated training and collaborative design facility, operated by RAL Space. If you wish to use the CDF, you need to book it with RAL Space. The CDF currently has 14 work stations. Historically, attendance has fluctuated between `<50%` and `100%` of those who sign up. It is possible to pair program on the training course. Feedback on this has been mixed so it is best not to overbook courses. + +To book the CDF Suite: +* Contact Will Grainger at RAL Space. Will administers CDF activities. Tell him which days you'd like to run the training. Will will tell you if those days are available. +* Contact Chris Gibbins. Chris is the IT guy for the CDF. Chris can help you install IBEX and genie_python images on the dedicated PCs in the CDF suite. Chris prefers to have >1 week notice to set up the machines. **If using the CDF suite, it is wise to book the CDF for at least half a day before the training course, so that we have an opportunity to install IBEX on the machines. Otherwise, the room may be in use, preventing installs.** + +To use CR16 & CR17 in R80 +* Book CR16 & CR17 via Outlook. These rooms are heavily used, so you need to book early to get the dates you want. +* Contact Facilities IT to arrange to have laptops made available to the training delegates. You will also need to coordinate with Facilities IT to get IBEX & genie_python installed on each laptop. + +To do it remotely +* Ask Chris MS to create some virtual machines and install a training system on them as below +* If you do not have enough VMs then ask users to use IDAAS to log in to DEMO (test that you can do this first). Make sure that you make people aware that this means they may be "fighting" over the same server. +* Contact the participants and let them know how to log in to> these machines +* Last time we ran remotely we did it in these 4 sessions: + 1. Introduction to IBEX: + * How to start/stop everything + * What do the different perspectives do + * A basic overview of what’s going on behind the scenes + 2. Configurations and Synoptics: + * How to create and edit configurations + * How to create and edit components + * How to create and edit synoptics + 3. genie_python and Scripting in IBEX: + * Basic genie_python commands + * User and instrument scripts + * The script server/script generator + 4. Converting between open genie and genie_python: + * The major differences between the languages + * Some example conversions + * Hands on conversion of a selection of SANS2D user scripts + +### Review the training course materials +Make sure you review the training course materials. Have any new features been added to IBEX or genie_python that might require a change to the course content or are their training tickets on the backlog which should be considered? if so, update the course. It is a good idea to run through the course materials even if there have been no significant changes - just to re-familiarise yourself with the course contents. + +### Send Delegate list to CDF +If you have decided to use the CDF suite, you need to tell RAL Space (i.e. Will Grainger) the names of each of the delegates, so that they can arrange access (e.g. swipe passes) to the Atlas building. RAL Space need 2-weeks notice to do this. A simple spreadsheet naming each delegate, their e-mail and indicating if the individual is a member of ISIS staff is sufficient. + +### E-mail course joining instructions to all delegates +1-2 weeks before the training course is due to run, send an e-mail to all delegates, telling them how to join the course. Make sure you tell them: + * where the course is being held (building, room number) + * which day(s) the course is running + * start time, finish time (e.g. if the course is due to start at 9:00AM, tell delegates to arrive at 8:45AM, so that the inevitable late-comers don't disrupt things too much). + * any other joining instructions (e.g. who to call if they get lost) + +### Set up and check the venue & facilities + +#### General + +- At least a week before the course, book any tea or coffee requirements with catering. Start navigating from the [Catering page](https://staff.stfc.ac.uk/core/catering/Pages/default.aspx) on the staff intranet site to plan catering. +This will lead you to the web-site of the [current catering franchise holder](https://www.ralcatering.com/), from where you can place an order. +Typically we book: + - Coffee/tea (80%/20%) for 9am and 1pm. Usually we include some biscuits and a bottle of water + - We typically don't book lunch since most people leave to do other things over the break. + - You will need a project code to book with catering. Ask one of the managers (e.g. Freddie Akeroyd) to provide one if you need it. +- A day or two before the course runs, visit the venue to make sure everything is ready. You do not want to be trying to sort out problems 5 minutes before the course is due to start. Things to check are: + * are the desks arranged as you want them? + * are the audio-visual system (e.g. projector) working correctly? + * are the laptops/workstations properly setup and working? + +#### CDF specific + +The workstations at the CDF are generated by cloning a template machine. The source machine will need setting up a few days before to allow time for cloning. Contact Chris Gibbins a week before the training course to arrange a suitable time. The template machine will need the following software installed: + +- [MySQL](https://www.mysql.com/downloads/) + - Visual c++ redistributable. MySQL may manage to install this itself. Otherwise you'll need to download and install it manually. Be sure to get the architecture that matches MySQL (presumably 64-bit) +- [Notepad++](https://notepad-plus-plus.org/) +- [Java](https://java.com/en/download/) +- [Git](https://git-scm.com/download/win) +- IBEX (server + client) +- [Mantid](http://download.mantidproject.org/) (if appropriate) +- [Mantid training data](https://sourceforge.net/projects/mantid/files/Sample%20Data/TrainingCourseData.zip/download) +- IBEX training configurations + - Clone using `git clone http://spudulike@control-svcs.isis.cclrc.ac.uk/gitroot/instconfigs/inst.git [MACHINE_NAME]` + - Switch to branch `NDXTRAIN` + +Before cloning the machine, it is helpful to apply the following settings to avoid having to do it individually: + +- Set the DAE to use appropriate TCB, wiring, spectra and detector tables from the configs. Verify you can start a run on the base machine +- Set notepad++ to [replace tabs with spaces](https://stackoverflow.com/questions/455037/convert-tabs-to-spaces-in-notepad) +- Create desktop shortcuts for IBEX client + server + +The following changes should be applied to each of the workstations individually: + +- Change the resolution from 4K to 1080p +- Rename the instrument config folder in `C:\Instrument\Settings\config` to match the machine name +- Rename the `init_inst_name.py` file in `C:\Instrument\Settings\config\[machine name]\Python\inst` to match the instrument name +- Delete the `.settings` directory in `C:\Instrument\Apps\Client\configuration`. This will empty the IBEX preference store and notably make sure it starts up pointing at the local instrument rather than the machine it was cloned from. +- Change the PV prefixes on the PVs associated with the Eurotherm component in the training synoptic to match each training machine +- Make sure you can successfully start a run +- Make sure that you can successfully start a scripting session +- Stop the IBEX server (else it won't work properly for the next person who logs in) + +We may be able to automate all of those steps. For the time being the first three can be accomplished via a script. The following was a non-generic script that I used for the January 2018 training. I haven't refined it yet in anticipation that we can do something more comprehensive with AutoIt+Python in the future. + +``` +import shutil +import os +import socket + + +def delete_stored_gui_settings(): + shutil.rmtree(os.path.join("C:\\", "Instrument", "Apps", "Client", "configuration", ".settings")) + + +def rename_config_folder(): + root_config_folder = os.path.join("C:\\", "Instrument", "Settings", "config") + possible_config_folders = [os.path.join(root_config_folder, "DESKTOP-NP89I4S"), + os.path.join(root_config_folder, "DESKTOP-NT4N3KQ")] + config_folder = [p for p in possible_config_folders if os.path.exists(p)][0] + shutil.move(config_folder, os.path.join(root_config_folder, socket.gethostname())) + + +def rename_script_module_initializer(): + script_folder = os.path.join("C:\\", "Instrument", "Settings", "config", socket.gethostname(), "Python") + files_to_delete = [ + os.path.join(script_folder, "init_desktoc9.py"), + os.path.join(script_folder, "init_deskto24.py"), + os.path.join(script_folder, "init_desktoc9.pyc"), + os.path.join(script_folder, "init_deskto24.pyc"), + ] + for f in files_to_delete: + if os.path.exists(f): + os.remove(f) + possible_init_paths = [os.path.join(script_folder, "init_desktop_np89i4s.py"), + os.path.join(script_folder, "init_desktop_nt4n3kq.py")] + init_path = [p for p in possible_init_paths if os.path.exists(p)][0] + shutil.move(init_path, os.path.join(script_folder, "init_{}.py".format(socket.gethostname().lower()))) + + +delete_stored_gui_settings() +rename_config_folder() +rename_script_module_initializer() +``` + diff --git a/doc/processes/pm/Estimating-Sprint-Capacity.md b/doc/processes/pm/Estimating-Sprint-Capacity.md new file mode 100644 index 000000000..3375013d1 --- /dev/null +++ b/doc/processes/pm/Estimating-Sprint-Capacity.md @@ -0,0 +1,26 @@ +# Estimating Sprint Capacity + +Sprint capacity is not used by this group to limit points in a sprint, but to track work throughput via a burndown chart. + +The values applied to tickets can be found in the announce channel on our teams instance. The points for a ticket should include time for reviews and any rework and reviews that may be needed. At the last update of this page these are: +``` +1 point < 1 hour +2 points < 0.5 day +3 points < 1 day +5 points < 2 days +8 points < 3 days +13 points < 5 days +20 points < 8 days +40 points < 16 days + +Timeboxes - pointing allows for reviews +0.5 day - 3 points +1 to 1.5 days - 5 points +2 to 2.5 days - 8 points +3 to 4.5 days - 13 points +5 to 7 days - 20 points +``` + +The value assigned to a sprint is `the_number_of_days_in_this_sprint * the_average_from_the_previous_three_sprints_of_(points_completed_in_sprint/the_number_of_days_in_the_sprint)` + +The number of days in a sprint is the sum of the days that the team members are available between the start and end date, with a multiplier to allow for the time spent not doing work on tickets which varies based on the responsibilities of different team members. \ No newline at end of file diff --git a/doc/processes/pm/PM-Workbook.md b/doc/processes/pm/PM-Workbook.md new file mode 100644 index 000000000..fb9cc5a93 --- /dev/null +++ b/doc/processes/pm/PM-Workbook.md @@ -0,0 +1,7 @@ +# PM Workbook + +This workbook is used to support the calculations for the number of points available in a sprint, and an indication for interest in the flow of tickets. + +It can be found [here](https://stfc365.sharepoint.com/:x:/r/sites/ISISProjects-181/Project%20Management%20Documents/IBEX%20Pm%20calculations.xlsx?d=w44859227499244e6bf092cc7b74411a8&csf=1&web=1&e=joqvGu). + +Details of the tabs and what to do with that information, including milestone creation, are all in the workbook. \ No newline at end of file diff --git a/doc/processes/pm/Prioritisation-of-Tickets.md b/doc/processes/pm/Prioritisation-of-Tickets.md new file mode 100644 index 000000000..c320ec7fd --- /dev/null +++ b/doc/processes/pm/Prioritisation-of-Tickets.md @@ -0,0 +1,23 @@ +# Prioritisation of tickets + +We are presently not ordering tickets to a finer detail than high/medium/low, and this is often based on overarching priorities brought in by stakeholder managers, and the group generally. + +These are notes on prioritisation this is not an ordered list yet: + + - Stops Instrument running + - Impairs running instrument + - Needed for user program Now + - Needed for user program soon + - Needed for commissioning program Now + - Needed for commissioning program soon + - Maintenance helper + - Instrument scientist requested feature enhancement + - User requested feature enhancement + - Operations requested feature enhancement + - Dev requested feature enhancement + - New functionality (e.g. script server) + +Plus categories + - Instrument is pure IBEX + - Instrument is pure SECI + - Instrument feature could work in SECI but is IBEX instrument diff --git a/doc/processes/pm/Sprint-Burn-down-Chart.md b/doc/processes/pm/Sprint-Burn-down-Chart.md new file mode 100644 index 000000000..88da99e36 --- /dev/null +++ b/doc/processes/pm/Sprint-Burn-down-Chart.md @@ -0,0 +1,35 @@ +# Sprint Burn down Chart + +The [sprint burn down chart](http://shadow.isis.cclrc.ac.uk/ibex/burndown-points.html) is intended to show the team's progress through a sprint. It is reviewed daily, during the daily stand-up meeting, so that everyone in the team can see the team's progress. + +The chart is deliberately arranged to show progress as a count down. At the start of the sprint there are points available to the team. As work progresses, individual backlog items are completed and the item's number of points is deducted from the target. Thus, the lines on the chart show the cumulative progress of the team towards completing a number of points. + +The target is the total number of points in completed tickets the team should expect to achieve in the sprint. + +#### What do the lines on the Chart mean? +The chart shows 3 lines: + 1. Completed - shows the target minus the number of points in completed tickets as a function of time. + 1. Completed (Ideal) - shows the "ideal" progress, assuming points are earned at a steady rate. The purpose of this line is to act as a comparison to the Completed line. + 1. Review + Completed - shows the target minus the number of points of tickets in the Review & Completed columns. + +In an ideal world, the Completed line will coincide with the Ideal line. However, it is not realistic to expect backlog items to be completed at a steady rate - backlog items are discrete items, of differing size, taking varying times to complete. In reality, the Completed line will always diverge from Ideal line. + +#### So what is the purpose of the Ideal line? +If the team completes backlog items in a metronomic fashion, the Completed line will closely track the Ideal line. If the Completed line diverges from the ideal line, it provides an immediate visual indication of the team's progress relative to the ideal. + * if the Completed line is above the Ideal line, the team is completing tickets in a slower fashion + * if the Completed line is below the Ideal line, the team has completed tickets quickly +Small divergences are usually just a reflection of the discrete size of backlog items - they usually correct after a few days. Larger divergences are an indication of a problem. + +#### So what do divergences mean?: +Divergences above the Ideal line can mean: + * Something is stopping the team from completing tickets, be that time, available tickets, absences, or similar. + * There is a bottleneck in the process; something is preventing backlog items making it through to completion. + * Something else is diverting the attention of the team. + +Divergences below the Ideal line can mean: + * That larger tickets were completed in a sprint, although the work may have spread over multiple sprints. + * That a focus on reviewing tickets has occurred. + +When the team sees the Completed line diverging significantly from the Ideal line, it needs to take action, to correct its course. + +The Review + Completed line helps diagnose potential problems. It shows the number of points in the Review and Completed columns. It is a measure of the what the team could potentially achieve if all the items in the review column were completed. If there are a large number of items in the review column, it will diverge from the Completed line. It is an indicator of a build-up of items in the Review column. The greater the divergence between the Completed and the Review + Completed lines, the greater the build-up. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.02.14.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.02.14.md new file mode 100644 index 000000000..c780d0d5d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.02.14.md @@ -0,0 +1,32 @@ +# 2018-02-14 + +## Sprint_2018_01_18 +#### Date of Sprint Retrospective: 14-02-2018 + +1. Testing API Changes + * we made a change to genie_python (changed enumerated types) but did not create unit tests to test the changes. This came back to bite us. + * we had a similar issue with #2942 - changes to DB records had impacts elsewhere, which caught us out. + * Lesson: make sure there are tests that test your changes. + +1. SM300 Motor + * We estimated 8 points for this (thinking it was a big ticket). In reality, it required much more effort than 8 parts. + * Creating a motor_record was more complex than we expected. + * we didn't really have a good understanding of SM300 motors + +1. Identify IOCs we can contribute back to EPICS Community + * We have quite a collection of IOCs now. Some could be of value to others. + * Identify IOCs that would be useful to others. Work out the best way to package them up for distribution. DO to organise a meeting to discuss further. + +1. Deployment + * We left deployment until the last minute, mainly because we were waiting for GEM & SANDALS tickets to be completed. + * We should recognise that we don't have to delay the release for late tickets. We can proceed with the release and hot-fix any instruments that require the late tickets. + +1. Build Speed + * Full, clean builds on Windows take rather a long time - up to 8h. + * We should investigate ways to speed up the build. Consider as a topic for stand-down day. + +1. Identify Motion Control Tickets + * create a motion control label in GitHub to allow the Motion Control group to easily identify tickets of interest to them - DO to action + +1. Shadow Instrument Scientists + * Individual members should shadow instrument scientists for time-to-time - to watch them using IBEX and to learn how it could be improved, to better support the things they need to do. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.03.14.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.03.14.md new file mode 100644 index 000000000..0bf21877a --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.03.14.md @@ -0,0 +1,30 @@ +# 2018-03-14 + +## Sprint_2018_02_15 + +### Date of Sprint Retrospective: 14-03-2018 + +1. Build Performance + * builds are now much faster. + * there is still room for improvement. + +1. Windows 10 testing + * Windows 7 will be supported by Microsoft for only two more years. + * over time, instrument machines will migrate to Windows 10 (probably the new LTS version of Windows 10) + * we need to be prepared for this + * also need to consider what happens to SECI during this time period + +1. Prune the Backlog + * we agreed the backlog should be pruned. Be ruthless about it! (we can always re-open tickets if we are too ruthless). + +1. In coming sprints we will prioritise Script Server work, then E4 migration and then support for LSS scripting & graphing (because scripting & graphing will benefit from the Script Server and E4 work). + * Kevin will follow up with Andrew Caruana to get more details of LSS scripting requirements. + +1. How do we track user requests? We will create a process that adds a re-request label and consult this during planning + - Can we do better about managing expectations? Probably not but can increase the visibility of the request using re-request label + - Is it acceptable to tell people a feature can not be achieved because of engineering constraint? Yes if it is not an experimental critical feature; but probably check with the group first. If it is experiment critical we need to flag this and be creative. + +1. How do we create scientist release notes? They should be created during the release by the release builder (see wiki). + +1. Workflow for genie python development? See ticket 3046 + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.09.26.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.09.26.md new file mode 100644 index 000000000..06fcd556d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.09.26.md @@ -0,0 +1,21 @@ +# 2018-09-26 + +1. Want to put Tools in one place. [John: maybe 2 - deployed and non-deployed]. Examples are + - config checker + - config upgrade + - caget + - archive data grabber + - ioc generator + - installation and upgrade scripts. + - Added to [tech debt idea page](/processes/meetings/Technical-Debt-Stand-down) +1. OPI Python logic should be tested, for future OPIs test logic. + - see [#3619](https://github.com/ISISComputingGroup/IBEX/issues/3619) +1. Change meeting lengths, planning should be 30 mins shorted and prep should be 30 minutes longer. +1. Create a ticket to look at removing stale branches.[#3620](https://github.com/ISISComputingGroup/IBEX/issues/3620) +1. OPI should be sent to scientist for who it is being developer (added to IOCworkflow and [OPI Creation](/client/opis/OPI-Creation)) +1. Add Readme to all submodules explaining purpose, these can link to documents in module especially for standard epics docs. This is so everyone knows what things do. It would also be good to remove old code submodules that are not needed and generate dependency lists. + - Added to [tech debt idea page](/processes/meetings/Technical-Debt-Stand-down) +1. Finally looked at cycle problems slack channel. This is not a good method of recording these let think of something else. +1. `Friday Days` and `Tech Debt Days` both still important if we have time in the sprint +1. Story points, is it ok? Yes broadly it seems like a good estimation tool. +1. Another Nicos dev? Tom said yes. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.10.29.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.10.29.md new file mode 100644 index 000000000..4731a18d0 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.10.29.md @@ -0,0 +1,28 @@ +# 2018-10-29 + +Generally slow sprint because of holidays. Please announce holidays to Kevin where possible so it can be planned in. + +* Should we automatically measure how long tickets took? + * No this is too hard but we should add comments to tickets which took too long or was overestimated. PM will pick this up at sprint end and make us a list to discuss. +* Changes over a certain size to be pair reviewed? + * Yes for the moment create a label to indicate this + * Look for a tool to measure complexity change (ticket being added) +* Access control on `INST:LIST`? + * No fine for the moment, but change process for setting it to include a review before running. +* Review column > 10 items, ok? + * Maybe because there were many tickets for release. Rory to monitor it for next time. +* Remove old Jenkins jobs? + * Yes, Freddie to check that nothing is extra is captured in the old jobs and to back up then remove +* Remove E3 client and RCPTT? + * John to check that it hasn't been run on any instrument and then create a ticket to delete jobs after backup of Jenkins and move it to client folder (instead of `client_E3`) + * John to create RCPTT test transfer tickets +* Wiki checks should be smarter? + * Yes, Liam to write a ticket +* Drop in sessions? + * Last one good with Ron + * Check the [list](https://github.com/ISISComputingGroup/IBEX/wiki/Instrument-Control-Drop-in-Session) + * Monitor success, has been asked for in and out of cycle by scientists + * Ask Mantid team what they do +* More demonstrating + * Yes, please start powerpoint and review earlier + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.11.21.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.11.21.md new file mode 100644 index 000000000..9a5a583f0 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.11.21.md @@ -0,0 +1,45 @@ +# 2018-11-21 + +Short sprint with too many tickets made it weird; make sure next sprint is reasonable. + +- Good to install before machine physics; everyone agreed. + +- 2nd console issue should at least document [I have added to user manual). There is a ticket to discuss larger issue + +- We should link from places in the GUI to the user guide on at least all views. Ticket created #3814. + +- Migrating instruments vs improving gui/ fixing bugs + - We are happy we are making deadlines for migrations but it is a push + - Make sure that we prioritize IOCs better within an instrument, what should be done now vs later + - LVDCom vs EPICS IOC usually epics is better we have not been back to any ones we said we would quickly do and come back to! + +- How much to push the Script server? + - push it, it is the way forward + - should provide training + - mention in upcoming demos (there is a list of advantages) + - [X] Dom to create a template of questions to ask at Instrument scientist demos + +- Simulation mode + - look at Adam's scan simulation library + - integrate into NICOS + - make it easier to use + +- Future Ideas + - Ask Kevin how it works + - Review it a couple of times a year + - this page needs updating + +- Script migrations + - Should be done more with scientists so they own the script. Change wording on the migration page [done] + +- Move sprint end to after Christmas [done] + +- Maintenance View + - We should worry more about long-term maintenance of the project now we are delivering it to multiple machines. We have some tickets "in flight" and we should make sure we don't deprioritise them because it is important for the future. + +- Bunch of tickets: + - https://github.com/ISISComputingGroup/IBEX/issues/3817 + - https://github.com/ISISComputingGroup/IBEX/issues/3816 + - https://github.com/ISISComputingGroup/IBEX/issues/3815 + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2018.12.20.md b/doc/processes/retrospective-notes/Retrospective-Notes-2018.12.20.md new file mode 100644 index 000000000..ebf804652 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2018.12.20.md @@ -0,0 +1,36 @@ +# 2018-12-20 + +## General + +* Should we add the impact of our tickets to help us prioritise them? + * Decided to try initial sort by urgent and important quadrants. + +## Mad + +* Review column has been very full + * Remember to put one in, take one out + * Has been hard because some people have been stuck on long tickets +* Muon separator became a black hole of time + * Can we highlight time spent on this? [Kevin given action] + * Deadline set was not realistic, in that they still haven't plugged in the kicker. This is something to learn for future projects +* Hard to track progress through projects. + * We should investigate story mapping to illustrate the stories we are working on and their progress +* Hard when you go down to an instrument to decide whether a device is connected properly. Talked about cable mapping tools and getting electronics to fill in forms + * Decided to start recording it whenever we go down to an instrument on IOC specific pages. + +## SAD + +* Tickets which scope creep are bad, it is hard to recognise when items are going to creep + * Be mindful of estimates, if you are meeting then bring it up immediately + * Stop scope creep for IOC by adding an analysis step (Keithley 2001 was example) + * [John to add to process between sprint prep and planning if that is ok this feels like when we should do this] + +## Glad + +* Friday and Tech debt days were good, it was good to see them in the sprint again +* New instrument migrations were good. Process is now a fine art +* Instrument demos were useful with good feedback +* Live view was "ridiculously useful" +* ENGINX thought their requests had been well managed + * All instrument scientists were happy seeing improvements in IBEX + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2019.04.24.md b/doc/processes/retrospective-notes/Retrospective-Notes-2019.04.24.md new file mode 100644 index 000000000..74c21b5eb --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2019.04.24.md @@ -0,0 +1,29 @@ +# 2019-04-24 + +## Stressed Feeling + +The major issue this sprint was the feeling that no progress has been made this sprint and that we are very busy. There is a general concern that the next sprint we have said we will convert a new instrument but there is a feeling we do not have time for this. The new instrument will be resolved at sprint planning. Progress this sprint has been slow because: + +- lots of support calls because of the loss of a key member of the group +- Easter holidays and APR season took time out of the sprint +- we need to make sure we can do the number of points in a sprint. +- need to be prepared to say "No" to requests sometimes; if someone doesn't want to do this or are not sure then request should be passed to team lead, group lead or PM. + +## Number of Reworks + +Have a lot of rework tickets, to help this we have agreed to do informal reviews if possible before the ticket goes into review. (This has been added to [workflow](/processes/dev_processes/Tickets-and-their-Workflow)) + +Also agreed to try and make pull requests smaller. + +## Emulators in Support Modules + +We want a discussion about what should be in support modules. Should the emulator and/or OPI be in here? + +## Use LUA + +In general, we would like to LUA instead of `st.cmd`. There is little risk to this we should start by converting an IOC and make sure it works and then add it to the device generator. [Ticket#4288 created]( https://github.com/ISISComputingGroup/IBEX/issues/4288)] + +## Pending Tasks + +We should make sure that pending tasks scheduled for the next sprint are in sprint planning. [Added to sprint planning description](../meetings/Sprint-Planning) + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2019.09.11.md b/doc/processes/retrospective-notes/Retrospective-Notes-2019.09.11.md new file mode 100644 index 000000000..36180a5ac --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2019.09.11.md @@ -0,0 +1,50 @@ +# 2019-09-11 + +## Release was on schedule (or close enough) + +Just a note to be happy about. + +## Releasing fixed bugs + +We should always merge the hotfix and then prioritize the clean up for that hotfix. + +## Retrospective post-it notes vs magic board + +There are positives and negatives to both. +We can write solutions to problems on the back of post-it notes. + +## Server moves and releases clashing + +It is difficult to coordinate due to reliance on facilities I.T. etc. + +However, we should improve our communication when it comes to this sort of issue. We should organize meetings in which we discuss what we know about (or at least the rough plans) for the issue ahead and attempt to come up with a shared plan that will work around the issue e.g. release earlier or delay the release. + +## Standups + +* Are standups still useful? + * Yes +* Could we make them more useful? + * Possibly + * We do well at stopping conversations getting too long and technical + * Sometimes there are lots of small tasks which are not worth talking too much about at standup + * What we can do is to elaborate more on how far we are through certain tickets + * We should focus specifically on the **fine-grain** of what we have achieved yesterday and what we want to achieve today e.g. how far are we through a ticket, not that we are just working on it + * We should talk about blockages more +* Can we try to share blockages even if we are no longer blocked? + * Yes it may be helpful for others so they can easily know who to come to if they get blocked by the same thing +* Can we look at ticket steps to complete? + * Yes +* Should we split into 2 groups so we can talk for longer? + * No + * It is useful so that we are all updated on what everyone is doing in the team + * It is useful so that when we are impeded we have more expertise to help + * Talking for longer is out of the scope of a standup. As such we should organize to have other discussions/meetings if required, for example for subprojects that are occurring + +## At SAG can we emphasize emergency jobs if they are large? + +This has already happened at the SAG meeting earlier in the day before this retrospective + +## After/during sprint retrospective should we note down and work out ways to implement what we have discussed? + +This is already being done in these notes. + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2019.10.09.md b/doc/processes/retrospective-notes/Retrospective-Notes-2019.10.09.md new file mode 100644 index 000000000..a4f2cd02d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2019.10.09.md @@ -0,0 +1,85 @@ +# 2019-10-09 + +## How do we know if we're using flash reviews too much? Is it a problem? + +Flash reviews are for small and/or urgent changes. + +Before asking for a flash review we should consider the size and urgency of the change. + +When we flash review we are still adding code to the codebase no matter how small, thus our reviewing should be as rigorous and considered as normal reviews. + +This rigour does not take away from the benefit of flash reviews as our process (using the slack channel) brings it to the attention of people quickly and allows us to skip the normal movement of tickets in the project board etc. meaning it is still faster. + +We seem quite happy with our flash review process, but we should still be careful to not overuse it by carefully choosing when to ask for a flash review or not. + +## Is the review column ordering obvious and understood? + +The simple answer is no. + +People had mixed ideas about how we were meant to be ordering the column. + +Further to this, the ordering was inconsistent with the ready column and the new GitHub projects tool for moving tickets between columns inside the ticket puts the ticket at the bottom of the column (contradictory to our current process). + +We have decided to flip the ordering so that we insert tickets into the review column at the bottom and take tickets from the top. + +This makes our workflow consistent with our ready column and the GitHub tool. + +## Is it useful to log "extremely" quick support issues? + +Pros: + +- Allows the person working on the support issue to solidify the issue further in their mind +- Allows the sharing of information and creation of a dialog about the support issue +- Allows us to refer back to information on past support tickets that occur again +- Creates a process where we can easily understand what has happened and review any changes that are to be made + +Cons: + +- Possibly wastes times of the person working on the support issue if it is very simple +- Some issues are ongoing so the ticket may stay around for a while and clog up the board +- Often a support issue is a repeat of other support issues so we are repeating our logging of information in new tickets when we do not need to + +Ideas and answers: + +- Support issues should have tickets for the pros listed above +- We are creating a new project board for support tickets so they do not clog up our development board +- Discussed: Having a process in a support board or other board for common support issues so we do not need to repeat the logging and can simply add a "+1" to it +- Suggestion: A review of a support ticket should include checks for whether the relevant information has been added to FAQs, troubleshooting or other parts of the user manual + +## What do people think of the new "Pending Tasks" board? + +The board is very useful for time-sensitive tasks and is good for workflow management. + +A couple of people thought the name was not representative of the use case of the board. + +### Sub-discussion: Why are tickets referenced on the board with a note and not simply the ticket itself? + +We wanted to add the time to the ticket so the answer was to reference them and add the date as a note. + +We decided that if the ticket is on the pending tasks board we should add the date to be done on/by to the name of the ticket, as in "ticket-land" we would not necessarily know that the ticket is time-sensitive. + +## Our builds/system tests have failed because the db unit checker reports a bad db that has just been pushed + +It shows that our continuous integration is doing its job at least partly. + +We should have a process that stipulates of running the db units checker on our code before we merge a pull request. + +Ideas: + +- Have this on a git hook (there is a Friday ticket for this already) +- This should be on the reviewer's checklist when reviewing tickets on EPICS-ioc + +## Our system tests are unhappy, this makes us unhappy + +Having all these tests tangled up it is very difficult to know when a bug or problem has been introduced into the codebase by accident. We would like to be able to turn up and say "The system tests are failing which isn't like yesterday so what's been added to the codebase since then". + +First step: Work out why our tests are failing! 3 tickets should be made (one for each system tests pipeline) to review why this is happening. +Second step: Once we know why we should make a plan to improve our system tests and actively work on + +Also noted: If we could finish the parallelisation of the system tests they would be easier to run on our machines so understanding why one has failed would become easier. This would, however, increase the complexity of the system tests so it could possibly cause more issues. + +## Should we prioritize reworks in the ready column? + +Yes, for definite. + +When we have done a review and set it for rework we should tell the assignee that we have done so. They can then prioritise their work accordingly. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2019.11.06.md b/doc/processes/retrospective-notes/Retrospective-Notes-2019.11.06.md new file mode 100644 index 000000000..761f60c5c --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2019.11.06.md @@ -0,0 +1,47 @@ +# 2019-11-06 + +## How to handle non-critical but serious bugs + +e.g. IOC is updated to fix bug. We know it could be used on other instruments but isn't being used now. What do we do? + +The procedure we decided on is documented [here](/deployment/patch/Modifying-Code-on-an-instrument) + +## Stop doing 0 point tickets + +0-point tickets are skewing the analysis of our capacity to get work done in the sprint. + +0-point tickets are often unplanned tickets such as support. + +There were 2 possibilities: + +- Recalibrate the pointing system to be sensitive and not include 0 points +- Allocate a block of points to multiple tickets that would be 0 points e.g. 5 points to support + +We have decided on the first (recalibration). It has been noted it will take 3 to 4 sprints until we have calibrated ourselves to work with the new points system. + +We need to understand that the points count for when a ticket is started to when it is truly finished and in the complete column. + +Because the points system is now more sensitive to smaller tickets it means that we will get tickets with very high points. This highlights our need to split tickets more carefully, which is something we need to work on. It has also been noted that some tickets (e.g. writing an IOC) do not make sense to split. We need to be careful with how we handle these. + +## What is our process for reviewing support tickets? Should we have a designated support reviewer as a helper to the person on support? + +This question was noted because at one point there seemed to be a lot of support tickets in review that seemingly weren't being reviewed quickly enough. + +Using the flash review process is recommended for support tickets that are small enough. + +It has been noted that we could have a support reviewer that is responsible for ensuring support tickets are reviewed throughout a cycle, though no plan was put in place to do this. + +It has also been noted that most people do, but that we should ensure when we finish a ticket and move onto a review, that review should be of the same complexity as the ticket we have just finished or we should do multiple reviews to match the complexity. + +## We seem to sometimes have reflectometry, pending tasks and HIFI tickets on the IBEX board. Is this deliberate? We don't point and prioritize them the same, should they be on their own boards and not the IBEX board? + +These tickets are also contributing to IBEX so should be in the IBEX board and pointed accordingly as well as their own board. Their own board is more used to track the progress of the sub-project. + +Having these tickets in the IBEX board helps people working in those sub-projects to remain aware of what needs to be reviewed etc. + +We should remove the review column from the reflectometry board. + +## In sprint retrospective should we quickly discuss our progress on the last retrospectives notes and plans? + +Yes. + \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2019.12.02.md b/doc/processes/retrospective-notes/Retrospective-Notes-2019.12.02.md new file mode 100644 index 000000000..9aadcb640 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2019.12.02.md @@ -0,0 +1,45 @@ +# 2019-12-02 + +## Should we make support tickets private? + +The concern is that we include sensitive information in support tickets and that this may either break GDPR or cause security concerns. + +We have decided that we should not make support tickets private. The sensitive information that could be in these tickets (or in regular development tickets) should be placed in the icp shares and a link provided to that information which can only be followed in house. + +## We should have more demos at review + +Yes. The standard should be to demo a ticket and only if there is no reasonable way to do this we will "mention" it. It is worth encouraging people to demo rather than mention prior to the sprint review. + +## We should have fewer single points of failure + +Yes. When handing out responsibilities and jobs we should be more careful to not place the full responsibility in the same person's hands each time. This could be done by pair programming, or splitting tasks into smaller tasks and distributing these subtasks between people. + +We should be aware that it is an issue and consider this at standup and when defining how we run planning. + +## Should the supporter check Nagios oddities spotted during standup? Should these be ticketed as support tickets? + +Our current process is to note the oddities at standup and pass any issues out to people at that point. This is good because we share out the issues to people who may not have come across them before reducing single points of failure. + +It, however, has been noted that we have a few persistent Nagios issues that should either be ignored or chased down. We should create a ticket to deal with these, maybe as part of our fix all tests day? + +We should not put too much time into this as Freddie is setting up a new monitoring system. It has also been noted that not only Freddie should do this. We should do it collaboratively to reduce single points of failure. + +## Dealing with unique challenges of talking to hardware and limited testing + +We should make more use of our test instruments/have a more structured approach to using them. No concrete plans were put into place about how to make this reality. + +## Sprint planning took 3 hours, is this ok? + +In short: We are agreeing to spend more time on sprint planning (3 hours with structure breaks). We aim to take a more conscious approach to not talk about implementation and instead focusing on acceptance criteria. + +We came to the conclusion that the extra time is ok. We possibly have not been spending enough time planning. We have a large team with relatively long sprints. + +However, we recognise that we need to take a more structured approach and be conscious of how we are running planning meetings. In regards to this, we have decided to lengthen planning to 3 hours with a structure break in the middle. + +We must also consider our productivity in the meetings. It has been recognised that we often spend more time discussing implementation than we should. Though some discussion is necessary to understand pointing we often go a step further than is necessary. We have planning poker for a reason and we do not need to talk in as much detail to decide upon points. If we must discuss detailed implementation in a group this should be done in smaller more focused discussions at another point. + +We should focus on creating acceptance criteria for tickets and either agree on them in the meeting or if we cannot we should have a process for dealing with this. What was suggested was to create spike/exploratory tickets with the information we have and then spin out new tickets with solid acceptance criteria. We do this to a certain extent but maybe a more structured approach should be taken? + +We have also noted that we may need to structure our tickets better with flow charts and acceptance criteria in a standard format. + +It was also noted that preplanning has often taken longer than expected as well and we should consider how the whole process fits together and affects each other rather than just planning. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.01.08.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.01.08.md new file mode 100644 index 000000000..b95c583f3 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.01.08.md @@ -0,0 +1,30 @@ +# 2020-01-08 + +## Losing track of requirements + +Some requirements of the system, especially those which are older might only be held in the memories of developers rather than written down. + +We concluded that all requirements must be covered with system tests. This would prevent any more requirements from only being honoured by the developers who know them. + +To make sure that we haven't dropped any requirements, we should go through the IBEX requirements document (~70 pages) and use this to generate more system tests. We should not do this all at once. + +## Statistics on sprints + +It would be useful to see the stats on how many points we have completed in the previous sprint before the review. + +These statistics are already made as part of managing the IBEX project. What would be distributed would probably be the number of completed tickets at the point the report was compiled. + +## Do we list the support systems in the release schedule? + +Some of our systems, such as DEMO, have ended up with out of date software, such as SQL. Are these support systems in the release schedule and maintained appropriately? + +Created a ticket to make a decision about the best approach: https://github.com/ISISComputingGroup/IBEX/issues/5072 + +## Dev Release Notes + +We are about to make a release but the release notes are missing a lot of the changes which have been made. + +We should be more careful when starting and reviewing tickets, this process is documented on the [wiki](/processes/dev_processes/Tickets-and-their-Workflow) (and also many PR templates). + +## Config checker and systems tests caught several bugs +There was much rejoicing 🎉 \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.02.05.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.02.05.md new file mode 100644 index 000000000..a2bfb672c --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.02.05.md @@ -0,0 +1,75 @@ +# 2020-02-05 + +We are happy with how well we have coped with a sprint that threw a lot at us. Thank you everyone for working hard. :) + +## ENGIN-X Portable Galil: Could we have identified the cables as the problem sooner? + +Cables were tested individually to spot faults but the issue was that both the cables were faulty and weren't tested together earlier on. + +It is noted that there were a lot of possible causes for the problem and it was hard to isolate the issue to this. + +Solution: A portable laptop with IBEX installed on to test. + +- We will have Kathryn's old laptop when set up +- Need to add this laptop to the releases +- Has an IBEX Builder account +- We should add a local admin account + +## Update list of component stewards + +Yes, we should, it is very out of date. + +People should be consulted as to who should be added. + +The components stewards should all be STFC people at this stage. + +## Script generator testing feels stalled + +This is due to not having or knowing the availability of licenses for squish testing. The licenses are limited to 5 for writing and one for running (used by the builds). + +We need to work out who has a license and who should have a license. We should have a floating license so people who need it but not very often, can use it. + +Kevin will inquire about options for licensing in terms of having a floating license or changing the names on the license. + +## A near trivial ticket #4938 (LOQ Eurotherm) took less than 1 day to resolve, but 45 days (elapsed time) to review. +Why? + +This ticket was not put in review so people were not aware of it. Sometimes these things slip through, but we should always be careful to follow procedure. + +Thank you Dom for checking up on these things to keep it in order. + +## Are we all using the review column correctly? Should it be prioritized? Should there be an urgent review tag? + +The review column is prioritized (the top is the most urgent to review). There is an urgent tag to use that applies to the whole tickets life cycle. + +These urgent tickets should be placed higher in the column and we should be working through the column as per [procedure documented](/processes/dev_processes/Tickets-and-their-Workflow) + +## Training deployment + +Deployments had to be done late the night before training because the training room was booked up the day before. + +We should book out the room for a certain time (probably a day) before the training so we have time to deploy to them. + +A positive point was that our deployment took 3 hours between 2 developers for 16 machines, which is fast, well done John and Tom. + +An alternative to be would be to have virtual machines with IBEX deployed on that we can remote desktop too. This would be a slower experience and we are not sure about connectivity from ATLAS and R78, but this could even be more realistic. + +We have [extensive documentation for training](/processes/meetings_with_scientists/Training-Instrument-Scientists-in-IBEX) + +## This sprint was a bit stressful with EMU and a few things such as IE unexpectedly thrown at us + +We've coped with it well but it would be nice to have something like a Friday to make up for it. + +## We should think about spreading responsibilities away from Tessella people so as not to silo information for when they leave :( + +We need to be able to do things without Tessella people to prepare for them leaving. This includes areas such as support, training and the SANS DLS. + +In all areas, STFC people should make a conscious effort to offer their help when possible and Tessella people should make a conscious effort to ask for help. + +This is as simple as including a third person in a conversation between to Tessella people. + +We could make a label for tickets that it is very important to have STFC people on? + +## Should component stewards be STFC people? + +Yes, as noted in component stewards discussion documented above diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.03.04.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.03.04.md new file mode 100644 index 000000000..69194565f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.03.04.md @@ -0,0 +1,111 @@ +# 2020-03-04 + +## We should update component stewards + +Yes, this was missed after the last retrospective. Some combination of James, Kathryn and John to do. + +## Squish licenses + +- We are happy with what we have for the budget we have +- Floating licenses/more licenses will cost more + +## There are lots of tickets completed 👍 There are lots of tickets also in ready 👎 + +- Possible reasons: + - Lots of support tickets (1/3rd of the sprint) + - Lots of tickets leftover from EMU rush +- We are not extremely unhappy about this as we got a lot done, but it would be good to manage it better + +Suggestion: To encourage throughput we prioritize reworks and reviews over current tickets as well as new tickets. + +- Pros + - These reworks and reviews are likely a higher priority than our current ticket so it fits in with our prioritization + - It should encourage tickets to not hang around in ready and review and decreases their time from ready t complete + - May encourage people to document their progress more on a ticket when they are recording what they have done prior to switching to a rework +- Cons + - More context switching + - Switching state of a machine from one ticket to another takes time + +Conclusion: This cannot be a blanket rule as there are many edge cases. For a sprint, we will try this. People should find a practical stopping point in their current ticket to do a rework if the situation arises. The metrics to decide on this will be: does everyone hate it? Has our throughput increased? + +## Should we create support tickets for all work for all instruments (SECI and IBEX) whether in cycle or not and whether we're contacted individually or not + +Yes, within reason. + +- The documentation is useful for future issues to create context +- It should be recorded as support, otherwise, it seems we are pulling in tickets without the proper process +- Helps to capture our effort +- Support is defined as: if a scientist needs help +- SECI tickets should be recorded only if they are of relevance to IBEX +- If a support call took very little time e.g. writing the ticket would take more time it may not be worth it + - It stops being time-effective or useful to others +- If a support call is one of a common theme we should record all them together as a tally to identify areas of the system we should improve + +## We should demo more in sprint reviews + +Yes, we need to take steps towards this. + +- DEMO should be deployed to earlier to give people a chance to set up their demo + - There is now a reminder for a few of us to remind the group we need to deploy at the start of the week so it is organised earlier +- We don't necessarily need to demo on DEMO it can be on our own machine + - Though demoing on DEMO can be useful to point out issues surrounding performance, state that we did not experience on our own machine etc. +- It is a good chance for a dress rehearsal prior to scientist demos (we are a more forgiving audience) + +## Should we try again with inviting scientists to sprint reviews? Relying on "proxy" product owners has failed multiple times recently + +- We are happy with the current process of having separate scientist demos. +- Getting them to come is difficult and would extend reviews a lot. +- However, we should be more strict in demoing to scientists. + - We should organise these as part of the release (added to the ticket template) + +## Clean up after upgrade + +Upgrade process was leaving a trail of mounted network disks. Upgrade scripts should clean up after themselves. + +There is a ticket for this. https://github.com/ISISComputingGroup/IBEX/issues/5208 + +## Dashboard bug: This was an unnecessary own goal. Why did our testing (system, automated, manual) not catch these? + +Reasons why it wasn't caught: +- Wouldn't have been seen unless we had neutrons +- We haven't got 100% coverage +- The push for EMU made us rush +- Mistakes can be made +- Our simulation isn't very strong for pretending we have neutrons + +Actions: +- We should be more aware to use TDD + - It is noted that it is difficult to know what to test on legacy code and that it is easy to spot a bug when you know what bug you're looking for +- If we demoed it may possibly have been caught, we should be more strict with demos + +Positives: +- We were fast and reactive with our fix +- We are more aware of this now and know we should be more careful in testing and reviewing + +## EMU: Switch back to SECI failed because of the presence of 32-bit Open Genie components. Is it likely to occur on other muon instruments? How should we avoid? + +- This is now fixed +- There may be more banana skins +- As part of our migrations, we should test that we can go back and plan for a possible switchback + - This has been added into the steps here: https://github.com/ISISComputingGroup/ibex_developers_manual/wiki/Migrate-an-Instrument-Control-PC +- This was tricky to catch +- We should have a set of standard things to test across similar instruments e.g. motors for refl and start/stop for every one + +## Linter is perceived as slow by some instruments. The linter may not really be slow, but is perceived as such by some instruments. How can we make ourselves more conscious of performance? + +- We have a ticket and ideas for speeding things up. https://github.com/ISISComputingGroup/IBEX/issues/5210 +- HDDs rather than SSDs have slowed us down here, there is a plan to upgrade. +- Manual tests should have time limits. +- Demoing and testing on demo should help us with this. +- We should try to be aware of how our code changes performance. +- Memory leak has added to this concern +- We should consider writing more squish performance tests +- We should check with the SAG how important they think performance enhancements are +- ScriptServer may help with the speed + - We should add linting to the script server + +## Make a standard email to send release notes out when we make a release. Release process should include sending this out. + +Yes, added to the process in [creating a release](/deployment/Creating-a-release) + +[Standard email format](/deployment/deploy/Updating-Instrument-Machines) (not necessarily a good email, please consider it's contents). diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.04.01.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.04.01.md new file mode 100644 index 000000000..8a7707aee --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.04.01.md @@ -0,0 +1,25 @@ +# 2020-04-01 + +## We still didn't update component stewards + +This keeps being missed. + +## We should repoint tickets before planning + +- Now added as step to [sprint planning](/processes/meetings/Sprint-Planning) + +## At least whilst working from home retrospective notes should be put in a dedicated teams channel + +## In teams we can't use ticks/crosses easily for flash reviews + +- Instead use shocked face emoji as looking, happy face as approved and sad as rejected. +- Using a spreadsheet is too heavy so remove this + +## Should we come up with a template for instrument demos? + +Yes, the demo ticket should include this. We should make sure that more junior members of the team have sufficient input in to doing demos. + +## Reviews have slowed since moving to working from home + +- This could be due to people not being confident in how to review - we should make sure that PRs include instructions on how to review +- We could monitor who is owing reviews but this seems a little draconian, John will look into doing it. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.05.28.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.05.28.md new file mode 100644 index 000000000..134de8341 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.05.28.md @@ -0,0 +1,38 @@ +# 2020-05-28 + +## Specific Device IOC Pages on wiki + +As discussed in previous retrospective the [Specific Device IOC page](/Specific-IOCs) has grown to be about both the software and the hardware. This means that having the IOC in the title is misleading. It may also be good to review the kit we have listed there and make sure we have a good amount of info for devices and that they're well organised. I have added a [note](/processes/meetings/Technical-Debt-Stand-down) to do this in the next tech debt stand-down. + +## Organisation during meetings + +We can sometimes overrun or get off topic in meetings. This is especially bad on ZOOM as it's harder to pick up on social cues and there isn't someone physically kicking you out of meeting rooms. To help with this we should have a checklist of things we do at the start of each meeting, if appropriate: +* Explain the meeting's purpose +* Appoint a time keeper, responsible for saying if the meeting is overrunning or suggesting breaks when appropriate +* Appoint someone to take minutes if decisions are being made in the meeting + +We should also ensure we pause if we drastically change topics so that people can have a chance to leave the meeting and notify people on teams if we feel that others might be interested in the topic. + +## Onboarding + +We're onboarding a new member in August, this may be tricky whilst working remotely. We have to be particularly careful that we: +* Make it obvious that lots of people are asking question all the time and that it's encouraged to do this +* Make sure we make efforts to catch up with new starters and see if they are struggling with their specific ticket + +Adam mentioned that one thing he would have found very helpful as a new starter is a step by step guide on how to write an IOC, see [here](https://github.com/ISISComputingGroup/IBEX/issues/5435) + +## Working Remotely + +We haven't had so many unplanned tickets, which is good. This is likely due to no support and less casual interactions with other groups. Losing these casual interactions, especially with scientists, may be bad: +* There may be information we're missing from the 'bumping into' scientists, they also might be less aware of what we're up to. On the other side it might be good that people are only getting in touch when they have important information to share +* Teams isn't great for casual discussion as it's limited to chat within teams rather the whole organisation +* We have the IBEX users group in teams but it's not been widely used. We should start using it ourselves to announce things in a casual way, which will hopefully lead to more scientist use. John will use it to announce the `b.` namespace introduced in https://github.com/ISISComputingGroup/IBEX/issues/5401 +* We should make sure that we responds on this teams chat quickly to encourage people to use it + +## General thoughts on the sprint + +* We missed the target number of points for this sprint +* Having a visible burn-down chart that is automatically updated would be good. We currently have an automatically updated spreadsheet but no chart (http://shadow.nd.rl.ac.uk/ibex/) +* We should avoid taking things out of the bucket before reviews +* Can we set up a tool to tell us how long a ticket has been sitting in review. Freddie will add functionality into the project board task to tell us when a ticket has been review for over a week +* There was some discussion on how to remember when you have tickets that you asked to be reworked that you need to re-review. It was suggested that people put a note on the ticket or look into configuring gut hub alerts for this diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.06.24.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.06.24.md new file mode 100644 index 000000000..615eb1da0 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.06.24.md @@ -0,0 +1,66 @@ +# 2020-06-24 + +## Previous Retrospective + +We should make a ticket to add to the project board checks that notifies us when a ticket has been in review for a week or more. Currently, the project board checks code is not in git. + +## The new method of running planning and pre-planning, where we point as we assign to high, seems to cut out a lot of duplicated discussions and lets us have shorter meetings. + +Yes, we like this! 👍 + +## Could we use the same password for all zoom meetings and have it included in the email? + +This issue has been resolved by clicking the link. There are two passwords a numeric for the phone and a regular password. + + +## Recording Architectural decisions + +There was a discussion at the LabVIEW user group meeting today about documentation. One thing that was recommended as particularly useful by one contributor (and seemed interesting to me) was to keep architectural decision records (ADRs). We often meet to discuss things and make decisions but don't necessarily formalise the results so they can be tracked and easily located later. They might also have an audience in our user base and in training for future developers. i.e. why is it done like this? + +https://github.com/joelparkerhenderson/architecture_decision_record + +We have a [decision log](/processes/Decision-Log), which is being used more and more, and it is becoming useful. We should use a more solid template. The link above has some good ideas of how to layout this template. + +## Added tickets + +John created an added tickets channel. Kevin has already been recording these added tickets. It is useful to be able to track which tickets have been added and why. Therefore we will create a label and people are expected to write on the ticket why it was bought in. John will remove the Teams channel. + +We need to be aware of why we are adding tickets and we should only be doing this in discussion with others (especially those likely to disagree with us). We should also look out for patterns emerging with tickets being added in. If there is a pattern we need to plan to reduce the need to bring in tickets at the root, maybe something is not currently robust enough or we need to consider our planning further. To find this pattern we will need to look at the reasons for tickets being bought in and possibly flash reviews (though we do not want to add extra process to flash reviews as that is not their purpose). + +## How do we think the background phone call for tech debt day went? + +Some felt it useful and enjoyed being connected with the team, others weren't so interested. If people wish to create it then on Friday afternoons they are encouraged to do so, making the idea become semi-regular. People don't have to join in if they do not want to. + +## We didn't get full participation in tech debt stand down. Why? Do we care? + +We rarely ever get full participation. Some feel there was less this time around, others feel there was the same amount but different people were involved. We still believe it is above the threshold before we start to wonder if the days are worth it. People aren't obliged to join in, but it is a nice opportunity to work on something different. + +## Panning poker site vs zoom poll? + +We prefer the site. The only issue being that the values don't match our scale. There are other sites we could look at though. + +## Microphone quality + +Some people are difficult to hear in meetings due to low quality microphones. It is not always obvious to the person speaking. Thomas, Bish and Chris now have solutions or solutions are on their way. We can test out our own audio in Teams and Zoom. Dom is a `dalek` today. + +## EPIC tickets processes + +We like Kathryn's suggestion:"I'd like to propose the following process, that should allow enough sanity for whoever is tracking the points, as well as being able to be worked with in a sensible fashion: the EPIC is added to proposals, we point and timebox against the EPIC. When the end of the timebox, or end of the sprint is reached - whichever comes first - points are assigned to the sub-tickets that were undertaken, the EPIC is sent back to the proposals column (if there is still work to do on it), the points value and timebox reference removed, as well as the face of whoever was working on it so that we aren't prejudiced for prioritisation. There is probably some refinement to this needed, but it might make it less painful for us to use them" + +## Only 3 tickets in review and only 1 of 4 tickets in ready were not reworks + +Yes, we like this! 👍 + +## Regular snapshots of the burn down graph are useful + +Yes, this is really helpful to see! We don't want to end up chasing metrics though. + +## New style of release notes was very helpful during demos and drop-in sessions + +Highlights and breaking changes sections are good! Make sure that the person doing the ticket writes it in a nice and verbose way. We should add writing the highlights and breaking changes into our processes. + +## Timings of the review meeting + +Can we do a time per ticket in the review? Another suggestion is to put estimated times on the slide. + +What we chose was to order the review slides by the longest time first. When someone puts their slide in they should insert in between one that they believe to be longer and one that they believe to be shorter. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.07.22.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.07.22.md new file mode 100644 index 000000000..158ebd013 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.07.22.md @@ -0,0 +1,63 @@ +# 2020-07-22 + +## Previous Retrospective notes +* Still need to make the ticket for stale reviews (DO will make and propose for next sprint) +* Architectural decisions - getting better at using the [decision log](/processes/Decision-Log). Copy the style of the last decision is the current 'template' +* `added during sprint` label exists - label being used, adding a comment or to the description seems to be used appropriately and we should keep doing so +* Friday open channel - didn't really happen, but if we want to do it we should. Should make more of an effort with the new starter +* Burn down charts are good - the pinned info on teams was highlighted +* Release notes style is better - but must remember during demos that there are previous releases that need to be included when talking to the scientists +* Timings of the review meeting - this one was a short one, but we think that shortest first would be a better order for ease of timings, it would be helpful to add a rough idea of how long the slide should take (sprint review template created and uploaded) + +## Addition of size labels - functionality added with no documentation +Documentation has been added since +In use on all public repositories, based on the number of lines of code, adds the size automatically on creating the PR +Some people find it useful, others see it as nonsense - use the information if you want to + +## When marking as impeded need to make sure we add a reason why +Sometimes this is forgotten, brought up as a reminder + +## Starting to get a lot more late arrivals to meetings +We will start on time anyway +You can always log into the meeting before it starts +Consider using snooze not dismiss (so 15-minute warning, hit snooze, 5-minute warning - log on) (Editors note - you can change the delay on your pop up from outlook if you have it open in your own calendar to something else possibly if the standard 15 minutes doesn't suit you) + +## Coffee often has low attendance +Some people not necessarily willing to start the meeting if no one is there already - it is suggested that you start it and carry on working - put it in the background +Camera off is fine for coffee +Whilst working remotely we are meant to have two meetings each day, and coffee is our second one, better attendance would mean we don't have to consider something more formal +Leaving early is fine, arriving late is fine + +## The burn down chart we are using shows an increasing gap in the number of points between completed tickets and the total number in the completed and review columns - we are implementing more tickets than we review +The end of the sprint will have been thrown by the release etc. but the gap increased slowly from the beginning +The number of tickets in and out should be staying static, but the number of tickets has increased as well (end of the previous sprint there were 3 tickets in review, end of this sprint there are 11) +There are some instances where there is nothing available to review either because they need specialists to review them, they are already under review, or they were by the developer looking for a review +A code chat on how to review was suggested, it has been added to the ideas list and will be done sooner - this would be a discussion rather than a presentation +Will create a channel to go through reviews, what was hard, what was easy, things that have been left off, things that should not need to be done by the reviewer +Consider doing reviews as a group again for some tickets +Flash reviews - maybe need a different process, also need to get better at rejecting flash reviews - if you can't review it in less than half an hour using just the git code view it probably isn't a flash review +We will expand the use of the flash reviews channel to include urgent reviews as well by adding a link (e.g. for test fixes) to the ticket (rather than the PR), as anything that will take more than an hour combined should be accounted for with a ticket and points + +## It great to see a good number of live demos, and they were good demos too + +## Were any of the tickets added regressions of existing functionality? Are there any patterns? +Yes - we caught them and fixed them in the manual testing +Previous regressions - yes, some had been seen before in other releases, some had appeared historically and not been fixed. +Macros not being saved - needs a Squish test +If it can be automated and failed then we need to need to get it automated +Need to check if we can have a squish day, but need to confirm how many instances can be used at a time +The spreadsheet is long, and not straightforward, need a system test day to automate those that we can and verify those that we can't and what can be fixed +Need to also consider other ways of recording the tests and their statuses + +## Do we want rotas for the meeting roles? +Possibly useful: one for chairing, one for timekeeping, one for note-taking - create and try them + +## How did the release go? +It was close to the line, and it was stressful - there was quite a bit of out of hours working to get it done +There is a mix of it feeling that it is a long time before it might be used for various reasons, however, we are deploying and going to be able to demo before the scientists need to use it which is a good thing to have achieved and probably better for our reputation +Releasing close to cycle has risks, and it was asked about releasing again before the cycle, any bugs found we can patch and fix, but not release new features as we are currently unable to test thoroughly enough for that "last minute" style of release +That kind of "last-minute" release can be done confidently, but it takes a lot of time to reach that stage of test coverage +There was a discussion about bugs being released and the impact, and how far we are from being able to test everything, OPIs are an example of untested items which are widely used + +## There is a reflectometry goal being aimed for, and there is a plan for that which has increased confidence +There was a brief discussion of plans and their usefulness in tracking progression - whilst a plan might not last long, it is often reassuring to have a starting point at least \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.08.19.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.08.19.md new file mode 100644 index 000000000..e7c615f20 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.08.19.md @@ -0,0 +1,50 @@ +# 2020-08-19 + +## Previous retrospective notes + +* We are fine with the size labels on PRs +* We are fine with the way impeded issues are handled +* arrival at meetings- got better immediately after but diminished again, we will start on time anyway +* Might try to virtually "get up" in teams to prompt others to join standup +* coffee attendance is slightly better but still less than ideal. +* we were implementing more tickets than reviewing, this seems to have changed now + - this being brought up has driven people to review more + - we've got more points done than predicted, maybe because of not doing as well last sprint. + - try not to read too much into the burn down graph +* meeting roles seem to be working +* did we make this release more stressful by imposing arbitrary deadlines? Possibly, but it's more likely that they will always be stressful + - maybe don't leave enough time between manual system tests and first deployment + - need to have a meeting about manual which system tests need to be removed + +## Recording zoom code chats +* can only store them for 3 months anyway, and need to have permission of all involved, so might be not worth the effort +* especially if something changes in the future making the code chat outdated, a new watcher might become misinformed +* only record if somebody is unable to attend but wants to see it +* only record if everyone is comfortable being recorded, if you are uncomfortable being recorded but not willing to speak up about it, contact K.B. or D.O. and they will champion the issue. + +## Mantid has a week after release to tidy up +- this is equivalent of our tech debt stand down, but for a week. +- Mantid has more of a need for this than we do due to the size of their team and the manner in which they manage their development +- We might put aside a "no new dev work" sprint in long shutdown, and maybe 1 week per year of this, but no changes right now. + +## Should we run the system tests locally more? +* No, this is the purpose of the build servers, only run them on your machine if you think it will break a system test +* Could try to put something on Jenkins to highlight what was added since the last success, but this might not be possible/helpful given the large number of repositories they are reliant on +* Perhaps add a step in the PR template to check if anything might impact a system tests, but might be hard to follow due to breadth of system tests + +## Instrument demo pages +* Change it to one page with headers for the each release instead of one per demo meeting. +* Move all previous pages into same format. + +## Low scientist attendance to some demos +* If only one person is attending molecular spectroscopy demos each month maybe add them to another group to save dev time +* Potentially present to their group meeting, check with them. + +## Good progress with points completed +* we've got more points done than predicted, maybe because of not doing as well last sprint. +* try not to read too much into the burn down graph +* will see if it continues to next sprint + +## Stand up order +* If people want to change it then they can do so at their leisure, list is useful so people don't get surprised by being at the start, also highlights people who aren't there. + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.09.17.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.09.17.md new file mode 100644 index 000000000..39e8fddb3 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.09.17.md @@ -0,0 +1,38 @@ +# 2020-09-17 + +## Previous retrospective notes + +* Try recording code chat in upcoming code chat +* Mantid has a week after release to tidy up - Not necessary for us as because of our development management style +* Only run system tests locally if you think it will break the system +* Instrument demo page - One page with headers for each release instead of one per demo meeting. +* Sprint 2020.08.19 went well +* Can change stand up order + + + +## Making Review easier + +* Suggest classes/files as a starting point +* Leave some tips on reviewing + +## Undocumented features + +* Some tools such as slack were still being used by the IS and the knowledge about the tool was not distributed evenly nor maintained in the wiki +* Document any information/features that you know which could be useful to the team for development/support. + +## Hard to keep track of support Email (answered/unanswered) + +* Possible to remove fluff? +* 5-6 different email for the same topic which is hard to remember (maybe get in touch with Freddie?) + +## Test environment for our product + +* Installing and running script generator was fine on dev machine whereas it failed on user's machine. We should always test the installation process in an environment which better represents a user's machine. +* Careful with acceptance criteria +* Think about pre-requisite + + ## Deciding when to move sprint meetings and prioritise other tasks + +* Let others know that you will not be able to attend sprint planning during stand up +* Poll to reschedule sprint meetings \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.10.16.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.10.16.md new file mode 100644 index 000000000..163fd29d9 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.10.16.md @@ -0,0 +1,97 @@ +# 2020-10-16 + +## Last retro + +- We should remember to leave notes for the reviewer +- John will make a page for undocumented features +- We should put emojis next to the email in teams to say we are handling the email +- We should always reply from experimental controls where appropriate rather than our own email +- Freddie will bypass footprints and send directly to experimental controls mailbox for SANS or the Muons +- Dom to check if ticket for script generator testing environment (Ticket is [here](https://github.com/ISISComputingGroup/IBEX/issues/5701)) + +## As per an article in in.brief from HIFI. To quote: `Thanks to the work of the ISIS computing team, the group were then able to access the computers that control the beamline, turning their office into a replacement hutch for the week` so the solutions we put in place are being acknowledged, and are proving useful + +Chris MS: Yes, nice to get that warm fuzzy feeling - notes of diamond and a hint of rabbit too + +## Just a note to recall that technical chats should happen in the technical channel, as it gives people the option of dropping in if they have something to contribute or want to learn about something. It also provides an indication as to whether the conversation has been had or not so that we can see that the help has been given if needed, or not as the case may be + +We should make an effort to move private chats to technical where appropriate. Though we often do this, the more effort to do so should increase knowledge sharing and allow people's input who may otherwise not be involved. + +## [10/6 12:33 PM] Oram, Dominic (STFC,RAL,ISIS) + +Tom, John and I had a discussion about the best way to treat patches that instrument scientists send us for general IBEX code. With a view of wanting to expedite these to encourage scientists to work with us we thought the process should be: + +There must be a ticket created with the code attached in some way: + +- This ticket is a support ticket as we're supporting the scientist in getting their code into IBEX but is therefore 0 points +- The ticket goes straight into review and is reviewed by one of us in the usual process +- If we're happy then it's merged, if not we put it as rework and ask the scientist to do the rework. If the scientist is not happy doing the rework we will propose it as usual for one of us to look at + +I'm trialing this process with [#5776](https://github.com/ISISComputingGroup/IBEX/issues/5776) + +[10/6 4:20 PM] Woods, Kevin (Tessella,RAL,ISIS) +I think this is a good idea, but I have a couple of queries: + +- Is it really a 0 point ticket? If it requires anything other than a trivial amount of effort, then it is not a 0 point ticket. +- Scientists don't always consider the wider picture. Their solution might work for their particular needs, but often it won't easily generalise or may not be maintainable. We should take care not to accept code that generates a high maintenance overhead. + +Yes we agree, though it should not be a 0 point ticket. +Dom will write this process onto the wiki. Done [here](/processes/dev_processes/External-Contributions). + +## As well as when preparing new instruments we should probably consider IBEX and genie_python training more regularly as a reminder and most especially for new starters + +When we do IBEX training we should invite others than the targeted instrument. We should also run a yearly training course to catch any new starters and others that feel they need training. + +## We now have support calls coming in via a number of channels - e-mails (whether direct or to ISIS Experiment Controls), Teams, phone calls (and Zoom voice-mail messages). Should we have someone who is responsible for making sure each incoming issue is assigned to an individual for investigation & resolution? The task could be done on a weekly rota basis (like the role to run the daily stand-up meetings - in fact, it could be done by the person running the daily stand-up meetings) + +Traditionally this has been done by the "ISIS supporter of the week" who is also the on call person in the evening, but including more people in the role is probably a good idea + +[Monday 10:15 AM] Woods, Kevin (Tessella,RAL,ISIS) + +OK. In that case, maybe all it needs is for the "ISIS supporter of the week" to make it clear that incoming issues have been assigned and/or are being dealt with. Perhaps just a quick message in Teams? + +There is a supporter of the week who should make sure everything is being handled. This is not necessarily to handle it themself, but to ensure someone has taken an action. Supporter of the week should put emojis on the dealt with emails in teams if the person handling it has forgotten. + +We should all log into zoom when we start work so we can pick up calls. Freddie will move the zoom call hours to be 10-5. + +## nagios being extremely slow often slows down standup while we wait for it to load. Could we get nagios moved to a faster server? + +It's very * 3 painful. + +It is on the list of things to move onto better servers. + +We will leave it as it is and just handle it at standup. + +## I really like the oversight that we get with people using the ISISExperimentControls email address. Can we expand it more to include things like the discussion being had with PEARL about migration? + +This would enable sharing of information between us. + +John thinks that we should trust everyone to do work and then ask if help is needed and write docs to summarise. This would avoid clutter and noise on the email. We should add any information and conversation from emails onto the ticket. + +Conclusion: If a ticket has been written from an email chain then we should put it on the sharepoint and link it. + +## We have a number of tickets that are still on the board, have had little movement and we just move from sprint to sprint. Even worse there are tickets in the backlog that have had work done on them a while ago but no subsequent movement. This makes me sad because it feels like we have a lot of half finished jobs that we are close to being able to call done but not quite. How can we get these tickets over the finish line? If the people that were doing them are now busy can we assign others to just finish them off? Could we do a tech debt day on just going around and finishing these tickets? + +Kevin: Seconded, Attached is a list of the tickets marked as "in progress" today (13th October). I have highlighted in red tickets that have been "in progress" since before the current sprint (I strongly suspect these tickets are not actually making any progress). I have highlighted in yellow tickets from the current sprint that have been "in progress" for over two weeks. + +We still have some tickets like this. There have been some tickets in the backlog that were not on the board but were actually complete. There are also tickets out there we need to document and test but were "completed" a long time ago, we should not be afraid to admit we don't have the time and ask for someone else to finish it off for us. + +Let's create a label saying "I don't have time to finish this can someone else pick it up" which means we don't have to take people's face off them. Done, see "To pick up" ticket. + +Lets check the in progress for stale tickets. + +## We've done most of a cycle with users on python 3 :) + +Yes! :) + +There were a few wobbles. + +## We have been hitting our target for points done in a sprint more since we can see the burndown chart + +Yes! + +It can feel artificial and a bit micromanagementy. It is also not truly accurate. + +Some feel more motivated because you see progress. + +Reminds us if there's loads of support tickets we haven't put through. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.11.11.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.11.11.md new file mode 100644 index 000000000..0af20c10e --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.11.11.md @@ -0,0 +1,53 @@ +# 2020-11-11 + +## Last Retrospective + +- Documentation for our approach for external contributions [here](/processes/dev_processes/External-Contributions) +- Invite for training course went out to all scientists. Had 3-4 extra scientists join that were new to ISIS. We should perhaps run one every couple of releases or so. +- Running the training course remotely went fine. Lessons learned: + - Could have used some more training machines to be set up + - More time would have been beneficial in places + - It was harder to bring additional trainers in a remote course. Only one can speak at a time, as opposed to in person, where the second trainer can walk around and help with individual issues. +- Supporter of the week: Can't tell at the moment how this is going, we have not really been in cycle yet +- Had another case where replies from scientists got missed because the initial email was sent from a private inbox. Please remember to use the ISIS email account for mass emails. + +## Items from this Sprint + +### It would be helpful to have an e-mail template to use when informing scientists that we wish to update instrument control PCs. Obviously, the reasons for each update differ, but there are some common points we should always include: +1. why we are proposing an update +1. when we are proposing to perform the update (making sure we give plenty of notice) +1. asking scientists to respond with days/time that are most convenient to perform the upgrade +1. explain the consequences if a scientist declines the update or fails to respond. +There may be some other points we wish to include. +A template will help us to always remember these key points. + +- Yes, this is a good idea. Should also include a list of issues that are being fixed with the update. DO has volunteered to add a template to the wiki. +- If you are unsure about whether an email to scientists is appropriate, do not be afraid to run it past someone else. This is a common and valuable thing to do + +### Last year we did not run a short sprint over Christmas (sprints starts were 7/11/2019 and 9/1/2020) - this year we currently have a sprint scheduled for 10/12/2020. Do we wish to have a short sprint this year or do the same thing as last year? + +Christmas sprint has about 12 working days. What should we do with this time? We don't want to make this a full sprint to save us spending extra time on sprint planning etc. Options: +1. Extend sprint starting tomorrow to 6/1/2021. +1. Could do a long sprint starting from 10 Dec.? - Means we may come back in the new year and not remember what we planned +1. Extend sprint starting tomorrow by a week to 17/12/2020. New sprint starts 6/1/2021. Week before Christmas is for Fridays/Tech Debt/finishing off tickets + +Decided on option 3. This will hopefully give us a nice clean slate for the new year. + +### It's been a long time since we've had a Friday, can we have a week of them? Or maybe a whole sprint full? See Freddie's message above + +See above + +### POLARIS is happy that our sample changer update caught the fact that a sample was dropped and stopped the experiment so that it could be corrected + +Good! They were happy that while they did lose time, they did not lose much time. + +### Our current sprint is 2020_10_15 but our "slides" are sprint review 2020_11_11, should the ppt reflect the sprint name and not the date the sprint review was carried out? That would make correlating at any later stage easier if required. + +Yes, this is a good idea. Call the file `Sprintreview_start_`. Add start and end dates on first slide + +## Additional Comments + +- We are happy that POLREF is now running on IBEX and SURF is planning to use it this cycle. +- We are making good progress on SANS2D. Different groups coordinating well with each other (us, scientists, network team, ...) +- We are excited about Windows 10 updates. Build is "mostly" automated, process is now documented [(Link)](/systems/inst_control/mdt/Building-a-windows-10-instrument-machine-from-MDT), everyone should in principle be able to set this up. +- What to do with trivial support issues? Creating a ticket every time seems like a lot of overhead --> Put them in a teams channel, then record them permanently in a ticket at the end of the sprint. `support-issues` channel has now been created \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2020.12.16.md b/doc/processes/retrospective-notes/Retrospective-Notes-2020.12.16.md new file mode 100644 index 000000000..915667dbf --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2020.12.16.md @@ -0,0 +1,36 @@ +# 2020-12-16 + +## Items from the retrospective before last +### Supporter of the week +- If on-call keep an eye on the emails etc +- It's good that people are keeping an eye on it, just need to remember to add the note +- The being on-call/supporter of the week wiki was highlighted +- The wiki page needs updating, including which teams channels to watch over and checking up on pending tasks that are due that day +- DO volunteered to update the notes with some aspects +- CMS will have a look sometime about any other tasks + +## Items from the last retrospective +### Email template +- Still needs to be done, DO will look at this +### POLARIS Sample Changer +- POLARIS was happy, but transferring to SANDALS meant that the system was not easy to use there, partly due to balancing the needs of different instruments, and partly due to the present working situation +### Support channel +- Helpful, but probably need to keep an eye on patterns/trends. However, the developers are usually pretty good at keeping an eye on those +- Useful for the quick issues but which still allows knowledge to be shared + +## Items from this retrospective +### Drop-in sessions and the shared/private email discussion +- Ways to fix the response options are listed, and the wiki has been updated with appropriate instructions +- Training and similar should follow the same pattern +### Release notes +- Lots in the dev notes that were not moved into the release notes, and there may be some that are missing from the dev notes as well +- There are tickets in the backlog that cover this, these will be proposed for the “Friday” sprint +- Review should have an entry in the Dev release notes, review complete needs to be added to the scientist visible release notes +### Presence on site not enough if there are multiple tasks at once +- Considering more people on-site to allow for meetings and support, and deal with multiple items +- Remind the other teams that advance warning of things moving that may need help to set up advance warning helps us to make sure that we can provide an appropriate level of support + +## Additional Comments +- Did very well with the tickets, we should be happy with the graph +- Board is relatively clear +- Tickets in progress or in review get completed, tickets in ready and the bucket will be considered for inclusion in the Friday sprint, any not added will be reprioritised as standard in January diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.01.06.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.01.06.md new file mode 100644 index 000000000..bd56ada17 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.01.06.md @@ -0,0 +1,24 @@ +# 2021-01-06 + +## Items from the retrospective before last +### Email template +- Still needs to be done, DO added to his list. +### POLARIS Sample Changer +- DO suggested a reasonable amount of time (~3 days) is required with the device to finish off remaining issues. + +## Items from last retrospective +### Drop-in sessions and the shared/private email discussion +- Initial invitation can be sent from shared ISIS ExptCtrls account. Select setting to not require response to avoid clutter. +### Release notes +- New process in place and automation following shortly +### Presence on site not enough if there are multiple tasks at once +- Plan at moment is to have 2 people on site per day if restrictions allow and demand dictates. Will wait to see how current situation develops before deciding on strategy and rota. + +## Items from this retrospective +- Friday tickets not being reviewed quickly enough, true for general tickets too. Reviews should take precedence over rework, especially towards end of sprint. +- Suggestion to have Monday morning for reviews to save context switching between consecutive work days. Developers should at least self-assign tickets to be reviewed at this time. +- Friday "sprint" went well and was enjoyed on the whole. Other ongoing tasks did get in the way of people taking on a specific Friday ticket. +- Discussion about shorter sprints to concentrate on specific matters (c.f. Mantid tidy-up week) "Technical Debt Sprint"? Decided not realistic in cycle, but more likely in longer shutdowns. + +## Additional Comments +- Discussion on GitHub price plans followed main meeting \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.02.03.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.02.03.md new file mode 100644 index 000000000..cdb7feb89 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.02.03.md @@ -0,0 +1,81 @@ +# 2021-02-03 + +## Items from the retrospective before last +### Email template +- now done + +## Items from last retrospective + +### Friday tickets not being reviewed quickly enough, true for general tickets too. Reviews should take precedence over rework, especially towards end of sprint. +#### Suggestion to have Monday morning for reviews to save context switching between consecutive work days. Developers should at least self-assign tickets to be reviewed at this time. +- Seemed to work pretty well but not everyone is always available on Mondays to do reviews +- Ended up with a lot in review because we had quite a lot of things under review +- throughout the sprint there were a lot of plateaus in the burndown chart - could be to do with interviews etc. By the end of the sprint we were quite above the line +- the dips on the burndown chart are Mondays so it must be working +- Sprint was in January - quite a lot of things that are out of our control, dark and dingy and not really able to get out much. Given the circumstances we think we did the best we could and the Monday review slot proved to be effective + +### Friday "sprint" went well and was enjoyed on the whole. Other ongoing tasks did get in the way of people taking on a specific Friday ticket. +- it was + +### Discussion about shorter sprints to concentrate on specific matters (c.f. Mantid tidy-up week) "Technical Debt Sprint"? Decided not realistic in cycle, but more likely in longer shutdowns. +- beginning of long shutdown maybe we'll do a technical debt sprint + +## Items from this retrospective + +### We have a number of tickets that are effectively "tidy up hotfixes". Can we have a sprint focusing on this at the start of the long shutdown? +- would be nice to get them more properly fixed +- there are some things we did very quickly last year, it'd be good to look at those again. +- long shutdown probably isn't the right time as they're not that important, perhaps this shutdown would be better? +- list will be made for these tickets + +### Suggestions for streamlining planning a bit more: + +- when we decide a ticket is high priority, leave it in the "proposal" column initially (just move medium/low out). we can then sort it directly into the right place in high later on +- maybe we should do away with "bottom of high column" tickets. we never get to the bottom of high. just put it in medium and save ourselves some time when we order them + - or just make an effort to change our thinking so low is low, medium is medium and so on. Bottom of high should really be in medium. +- when the contractors leave we could think about how effective we are doing planning - it goes on for a long time +- we should see if priority can be done using planning poker on the new poker site as this might be a bit fairer + + +### We should update all the Zoom invitations for all our sprint meetings - Daily Stand-Up, Planning, Pre-Planning, Review & Retrospective - to have Alternative Hosts. That way, we don't have to create a new meeting just because the person who created the original meeting invitation isn't around. +- need to confirm this works on zoom - the issue is somebody being in another meeting they have scheduled and zoom will not let you have two meetings you have created running at the same time. The docs suggest an alternative host can start a meeting if "join before host" is not enabled +- we'll test it out + +### record of systems time +- Have been trying to feed systems tickets from tasks through the project board to keep some form of review possible. Generally it is a case of creating a controls ticket which contains non-public system details and then encapsulating it into an IBEX ticket with a reference. This means that it can be reviewed and passed along though the IBEX board. Currently, it is not possible for me to add an estimate of the time taken to do the ticket as a label other than zero (I think with training) without it breaking the more strict conventions of the IBEX project point checks. + - We could add a few more generic systems labels. in blue, say, which are "1/2 day", "1 day", "2 days" etc. or possibly put the detail in the ticket title e.g. "Clone a new instrument machine (1 day)". This is more of a historic reference - or these points could be added to the control systems ticket in the conventional way - but they may be a bit less visible on the IBEX project board. The systems tickets are very often incremental and can be re-done regularly as a process so this is less attractive - although it could be a guide to the time to run the whole ticket once. + - we could have a separate private repository and a label for them, the tickets in IBEX could point to the private repo. The point of this is so that it can be put on the IBEX project board + + +### if there is a support ticket which doesn't involve any code changes there is not likely to be much in the way of a sensible release note for the users +- we could simply check the label for "support" and skip the release note mechanism +- alternatively, only generate a release note request if someone makes a pull request and not from the ticket itself + +### I like the new release note PR system, should we always change the PR title to be "Ticket YYY: " rather than have "Update ReleaseNotes_Upcoming.md"? It doesn't affect the operation of the mechanism, but I think it looks nicer when listing all the PRs +- If we agree this in the retrospective we could put it as a warning on the checker +- github doesn't make this easy like they do with PR body templates +- we should start doing it anyway +ACTION: Dom should update the project board checks to check these + +### Whether rightly or wrongly, we've stopped using the Reviews channel, are we happy with that situation? +- never really took off, was made for discussing reviewing best practices etc. +- we don't use lunch either as no one is on site + +### Do we need to think about paying for the planning poker site? +- probably, but there are alternatives, we will try them out first. + - trying this out at next planning meeting + +### Creating a new ticket to fix errors introduced by a PR feels wholly and egregiously wrong +- resolved, confusion was caused so next time don't be afraid to change titles to make things clearer + +### How good were our estimates for individual tickets? +- ok, but musr migration took twice as long because jack was involved as well, this isn't a bad thing because now someone who isn't as experienced has been shown what to do for future migrations +- this is probably reflected in the burndown chart + + +### This may not be the appropriate place to discuss this, but if so, how is "Coffee Time" working for everyone? I notice it tends to be a few usual suspects joining, so I just wondered why that might be... Bad timing? People busy? Not keen on chatting? Too many VCs already? This may not be a huge problem, but we do need to at least try to keep in touch. Are people doing that separately perhaps and therefore don't feel the need for a group chat? +- difficult to stay in the zone sometimes when working as its a bit like context switching, not because we don't like talking to people its just difficult +- sometimes have meetings at that time +- do we need to have an evening round-up? +- should we do coffee at the start of the day? or move stand up +- maybe we should have an activity or something for a bit of fun at coffee - show and tell, something else? werewolf? suggestions on teams social channel diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.03.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.03.md new file mode 100644 index 000000000..0ec879a47 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.03.md @@ -0,0 +1,38 @@ +# 2021-03-03 + +### Items from last retrospective + +* Monday reviews are still working well +* We're better at being stricter for high/medium/low +* We have now set general meetings to have alternative hosts +* When systems tickets get added to review (mostly by CMS then they will be pointed based on expected review time) +* Release notes PR system is still well liked +* Games on Friday coffee has attracted more people, which is good + +### Items from this retrospective + +#### New planning site +* No need to add ticket numbers for each story when pointing +* We should start the session afresh before each planning to stop phantom accounts hanging round +* It's ok not to have don't-knows on the priority points + +#### Why are some people quiet in pointing tickets etc +* People have different communication styles +* We should discuss estimates but it might be worth taking less time on wildly different estimates +* Quieter people tend to be unsure and voting on gut reaction (which is fine) and feel like they learn more after the discussions +* We will try the chair being the first person to input in the discussion of points +* By missing out on the quieter people way may be losing the concept of how complex is it for the average developer + +### Adding flash reviews to release notes +* If it's a flash review that needs something in the release notes it's probably not a flash review +* Flash reviews were originally created to be process-light +* We just did a large change in how we did release notes, we should see what users think about this first. Next release we should discuss +* If reviewing a flash review and you think it's very user facing then ask them to put in release notes +* When we do the instrument demos we should ask what people think of the release notes + +### Spread of ticket sizes +* Pair programming more would be good, we may be doing less of this remotely but it's still happening +* We could split tickets more but we're pretty good at this +* We should do more splitting up in an ad hoc way rather than up front as developers will know a bit more at that point +* We should encourage newer developers to pick up big tickets too +* We should ask at the end of planning if we have too many big tickets \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.31.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.31.md new file mode 100644 index 000000000..1a0870943 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.03.31.md @@ -0,0 +1,64 @@ +# 2021-03-31 + +## From last last retro + +- Monday reviews were working well until this week (mostly with cycle stresses) +- Doing well for using medium and low +- We haven't been putting system tickets in because the review column is long and we're trying to get everything in for cycle +- Release notes PR system we are happy with +- We haven't done games on Friday coffee for a couple of weeks + - This may be work pressures but it would be good to make a bit of an effort to organise/join it + +## Items from the last retro + +- New planning site is getting easier +- Re-voting and the chair starting the discussion has helped to improve the discussion +- If a flash review needs release notes then it is not a flash review +- Big tickets aren't scary we should encourage people to pick them up when it is not possible to split them up + +## How to manage security issues + +- Could use the private git-repo or Microsoft teams board and link from Github +- There are lots of different types of security vulnerabilities and there is a current concern from the organisation about security +- The important thing is that we have enabled a conversation about it in planning + +Action: Chris MS Will ask Anthony from FIT how they do it + +## Ticket, PR and branch cleanup in long shutdown + +- We should do this as we notice them +- We should also do a proper cleanup over the shutdown +- Most of the backlog is surprisingly valid +- Has been added to the long shutdown list + +## What can we do to ease situations like the one we are currently in where we are short staffed (for very valid reasons), but when the urgent workload is high? + +- Shall we do a 5-week sprint? + - We have APRs, new starters, IBEX training, lots of stressful things coming up + - Let's be careful what we add to the sprint + - Still keep a bucket + - We think 5-week sprint is better than 3 week +- Can we only point for 4 weeks of work and do Fridays if we're not doing support? + - Concern is that we underestimate Friday tickets and overestimate their importance + - Fridays are nice to have so may relieve stress + - Or could do technical bookkeeping tickets e.g. the above-mentioned Ticket, PR and branch cleanup (we'll leave that choice up to the dev) + - Fridays do help user experience improvement +- Yes we're going to do this +- Let's talk about this after the fact so we can organise the start of cycle sprints in a more solid way +- We like these ideas +- We can change the length of sprints to ensure releases are created in good time for the cycle + - We can do a full release earlier on and a patch release later + - In the long shutdown, we can do a few releases so we are ready + - Can we get more of a manual release process into squish to reduce this + +## Themed sprints + +- Decide on one or two themes for a sprint +- Prioritise the theme in their own columns +- Have a couple of people dedicated to each theme with others helping them and others focusing more on support +- Needs management because we aren't fully sharing that information with the rest of the team +- People can take on different themes and lead each time so it is shared around + - Ensure people have rotated around so we don't lose the ability to work on lots of different parts of the system +- Support can be set as a theme +- We should bear this in mind in planning +- Kathryn will upload document with idea of how to manage the project with themes and variable sprint lengths diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.05.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.05.md new file mode 100644 index 000000000..fe02dc1b9 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.05.md @@ -0,0 +1,63 @@ +# 2021-05-05 + +## Items from last retro +* Managing security issues - current org. concern because of many different types of security vulnerabilities +* Ticket, PR & Branch clean-up - do a proper cleanup over the shutdown, and as we notice them +* Short-staffed when urgent workload is high - point for 4 weeks of work and do Fridays if not doing support +* Themed sprints + +## Future Ideas wiki [page](https://github.com/ISISComputingGroup/IBEX/wiki/Future-Ideas) +- Used for long-term planning, "idea repository". Should we maintain this? + - people haven't added anything in there recently + - unsure of how to keep track of things in the long term + - good place for things that are nice to do and would like to when we have the time + - great place for tasks for placement/graduates +- Keep it + +## Planning Poker Rooms - anonymous voting +- traditionally the selected cards are not anonymous + - would we prefer anonymous, or displaying the vote cast? +- No strong opinions, keep it as is + +## Experiment Controls email - using too much? +- hasn't bothered anyone +- good to see what goes in and out + +## All releases seem to be stressful +- this round has been relatively calm, but it has still had its moments + - there have been discussions in the past, but is it worth thinking about it and revisiting it towards the end of this calendar year? +- do deployments much earlier, 2 weeks is too late; can deploy early is release is ready +- release was getting pushed back because of MUSR, stop doing that - release for all instruments and do a MUSR patch release late +- make sure to tell the scientists that we are deploying to establish hard deadline; scientists also want a window, so they know when to be available +- don't delay the release unless it breaks instruments - if not ready before the start of release sprint, don't add it + +## This sprint has been stressful +- need time to decompress after this sprint +- pushing ourselves hard to get things ready for cycle - this is affecting morale +- we should think about ways to avoid this happening again +- it has been a bit stressful but overall people feel ok +- if it helps, can "dial-down" next sprint + +## Technical articles channel +- we should send more technical articles round the group, should we have a channel? + - don't want people to feel obligated to read it + - reading could be allocated as training time + - see if people are okay with 2 chapters in a month, if not maybe 1 chapter + - neutron training course was the same (though it was more on the physics side of things), while this could be more about software +- this channel could be good training. Try to pick a book or online articles and discuss it at a code chat, see how it goes + +## Support tasks - 0 pointers or not +- How are we counting support tasks? 0 pointers or not? + - allowed to be 0, but if it takes time it shouldn't be + - burndown chart is -20 because of the support chunk + - decide capacity according to trend and keep to it +- carry on with support being 0 as long as it doesn't take too much time + +## Second week in cycle too early for sprint review +- is second week in cycle still too early for a sprint review? still busy on support. + - always going to be hard and there's always the possibility of support calls in the middle of it + - maybe split the meetings to be "less interrupted" if something comes up, instead of having a long meeting +- yes, split reviews and retrospectives. + +## Half an hour is too short for retrospective +- Change meeting time to an hour \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.26.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.26.md new file mode 100644 index 000000000..6f8553329 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.05.26.md @@ -0,0 +1,39 @@ +# 2021-05-26 + +### Previous retrospective notes discussion + +* Happy to keep burndown chart calculation as is. i.e. begin with estimate of capacity covering all projects + +* Technical Articles meeting arranged and suggestions for books etc. requested. + +* Zero points possible for support tasks, if only few minutes spent. Suggestion to count and categorize posts in "support-issues" channel to spot patterns. Agreed to discuss before next cycle at end of long shutdown. + +* Split of review and retrospective meetings: most people accept. Agreed to alter order of meetings and having on separate days i.e. review Tuesday, retrospective Wednesday, planning Thursday. Will have think about how it could work during next sprint, but not change anything, and try in practice next cycle. + +* Variable sprint lengths also accepted by most, as long as not too long (>6 weeks?). + +* "Friday" ticket time: most agreed with KVLB plan to have a week of them every now and again. + +### Retrospective Notes 26/05/2021 + +* Agreed to keep same chair for both sprint review and retrospective meetings. + +* Suggestion to add alarm history to Alarm view - add to Muon snag list and discuss at later date. Can add Friday ticket label for interesting tasks. + +* Remove "Complete" Column. Easier for reviewer to close ticket at end of review (if work satisfactory). + +* User feedback - HLM website proved useful for tracking a problem. + +* Discuss sprint themes at pre-planning meetings - agreed to continue. + +* "Support" label tickets requiring release notes - discussion on release notes in general. Another label will be created for "Release notes not needed" to apply to any ticket, support and otherwise. Reviewer to check. + +* Snags lists popular, at least with Muon group, also for reflectometry. Will eventually be one for each science group if many requests/problems come in from them. Not everyone in group keen on the idea until after migrations. Some confusion between definition of snag list and prioritised list of instrument tasks. + +* Release & deploy schedule discussed. Balance between features and stability for instrument scientists - some prefer each one. Don't want to deploy too late in a shutdown so IS testing time is reduced. + +* Cable and connector amnesty during shutdown (Short of particular types of MOXA cables). Generally agreed good idea. DPK voluntold to organise. + +* Suggestion to visit each cabin and blockhouse to tidy & label wiring, especially in rear of cabins. Possibility for a Friday or TechDebt ticket. + +* Cycle went well. Support calls at manageable levels, although busy at start with hardware issues (DAE etc) as well as week before. POLREF unlucky with many issues. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.06.23.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.06.23.md new file mode 100644 index 000000000..b64311f2a --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.06.23.md @@ -0,0 +1,92 @@ +# 2021-06-23 + +### Items from the last retro: +* add Alarm history to alarm view - hadn't been added to Muon snags list, has now, discuss at later date. +* "Support" label tickets requiring release notes, this has been changed we use the "no release notes" label now if things don't need release notes. +* Return to discussion about snag lists and balance between features and stability for instrument scientists. Meeting with group leaders in future might help with this. +* Visit cabin and blockhouse to tidy & label wiring only had a ticket for one instrument. Tickets for all or large ticket with checkboxes? Do this with someone else, will be difficult with 1 person, possibly 3 on pearl. +* Cable amnesty, a start has been made, but possibly talk to halls committee? problem of not knowing where cables have been so don't know if they need checking etc. Ask health physics for advice? + +### Email to teams: +* Create a ticket in support issues. Discussion should happen here. +* Have emails forwarded to more people to avoid having it marked as read if it's seen by anyone. + * note for who would be happy to have emails forwarded to you from the exp-controls inbox. + * Possibly forward ticket only the first time, this way you don't get all the replies? + * Ticketing system? get Footprint status? + * almost no one used Footprints web interface, this is why its in teams + +### Proposed tickets were added to the planning board before the pre-planning meeting. Good! + +### How did Tech debt go? +* Tried something different with allowing a bit more time rather than tech-debt day. +* Never had the planning meeting because we were partway through the week and the pre-tech debt ticket wasn't finished yet. +* A lot of people didn't get much tech-debt done because of having things to do already probably because of not having a meeting? +* Will be stricter next time with tech debt. +* didn't feel like people were willing to engage, and cycle was a pain so not pushed, next tech debt isn't in cycle. + +### Work-life balance +* messages and emails out of working hours can lead to a lack of work-life balance and burnout, please take time when your not always thinking about work. + +### Reflectometry Board +* Purpose of the board is to give a prioritised list to pick items from. + +### Change to automation of review complete column +* Change the name to done? +* if something is closed we can know what state it was in. +* completed label has been removed + +### Issue Templates +* Discuss at next retrospective when Jack is present? +* Suggested template has a lot that would need to be removed +* At least having headings might be useful +* Similar to Pull request templates. +* a lot to change on the first line so might as well delete, could just be less detailed about what needs to be in it because new people could just look at other issues and experienced people would just delete it. +* can we select different templates? + +### In Progress Impeded and Ready +* Labels are useful, could be helpful on different project boards. +* Possibly automate them? + +### Un-touched project boards +* several project boards haven't been updated since 2019 +* support board leftover from before teams channel +* remove them + +### Burndown behind or not? +* discuss another time when relevant people are here. + +### Investigating new technologies +* When do we try new tools that might be useful to the project? +* Migrations have to be done, so give the first 6 months to see how well migrations are going perhaps? +* Raise technology and discuss whether possible benefits are worth further investigation. +* Future ideas board? +* Fridays come in weeks, good time to look into it? +* Review migrations to see if we're ahead or behind and then consider them, possibly 3 months in. + +### Meeting times +* A lot of the time only 1 or 2 people in meeting room at the start +* are times/days problem or people not being ready? +* This review/retrospective has had a very short review, somewhat a-typical. + * more often runs on to 2 hours. +* We split them while we were in cycle to avoid missing both due to support. +* once new starters are in address that's a good time to update it +* Having them together means it can work as this time needs to be free. +* Split them during cycle and not outside of cycle +* block out more time, start earlier, finish same time, more time for a break between review and retrospective if the review takes a full hour. +* 3 o'clock start and 1.5-hour run leads to outside of core hours. could move it to 2 until 4 but most people not bothered by 4:30 finish. +* Start at the same time but have a longer break, very likely to overrun. + +### Important dates +* Server shutdowns and electrical shutdown dates made be useful to have on the experiment controls calendar. +* Other important dates might be useful to have in there. +* check if shared calendars work with alerts. +* Overlaying of calendars. + +### Students +* how often coming in +* supervision of them +* get higher occupancy of the room? have screens added? + * may have to wear a mask +* needs quick resolution +* have them on site the first week, but possibly not have them in every day that week? + \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.07.21.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.07.21.md new file mode 100644 index 000000000..b84bfc57a --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.07.21.md @@ -0,0 +1,54 @@ +# 2021-07-21 + +## Items from last retro + +- Tech-debt and Fridays have been collaborative which is good! Similar comments about planning tickets for Friday. Plan is to make a ticket to organise Fridays/tech-debt so meetings are had to plan and review tickets. We should have 2 standups to encourage collaboration or a channel that is always open to drop in and out of. +- Improved automation of the board is nice. +- Kathryn has deleted a couple of unused project boards which is good. +- Code chats should be organised to talk about investigating new technologies. +- Important dates: we need to check if alerts work, adding important dates have been useful +- Having new starters on site means that we organise ourselves more to get on site which is good. We should look at continuing this. + +## Coffee + +- Let's make coffee a very short second stand-up and then people can stay on for coffee +- It's ok to send apologies if you're busy just like in the mornings +- Need agreement from Freddie +- Compulsory to either attend or message why you are not attending + +## Passivity + +- We don't tell people we need to do something until we are doing it, we should give more warning for things like getting requirements, doing deployments etc. +- How can we become more proactive? +- Who owns the NDX, how can we share it better with the instrument scientists? This needs to be discussed more in depth +- An example is the SANS2D migration where scientists didn't know about the component steward and talked to different people for each ticket + +## Keeper + +- Someone needs to try out keeper to discuss how it works and present if we can use it and how to migrate to it (maybe at a code chat) +- Jack will do it, James will organise the code chat + +## Issue templates + +- We can have multiple templates to choose between (this is useful for releases and a generic release template) +- Our current generic template contains a standard title "Component: Short description", a user story with some more detail, acceptance criteria and some extra notes +- We can change the generic template but we need to make sure our process is light and tickets are readable +- Though including things like where is the code +- We use issues as user stories not issues in reality +- Jack has previously used + 1. How: What is the issue + 2. Where: Where the issue is likely to be (being as specific as possible inducing relative file paths etc) + 3. How: How did the issue come about + 4. Reproducible: Yes/No and any additional information + 5. How to test the issue is resolved: A set of instructions / a script / include test file if required to act as a reference for the developer when tackling the issue and the reviewer when work is complete. +- Both Jacks will write their own and compete to win the ultimate issue template + +## Burndown suggesting we haven't done enough + +- Yes it was difficult to calculate this at the start with so many unknowns + +## Tasks board isn't being interacted with much + +- We will put some into proposals that we can do as actual tickets +- It's not so useful out of sprint +- We should tidy it up soon diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.08.25.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.08.25.md new file mode 100644 index 000000000..38277a8a4 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.08.25.md @@ -0,0 +1,68 @@ +# 2021-08-25 + +## Items from last retrospective + +- Suggestion about changing of coffee - we did it during the tech debt week then stopped. If you have a late coffee you might just be saying what you're about to say in the following morning. Maybe it should be a catch up on problems - though we already do this in technical. Once we start coming onto site more we won't need to do coffee or a second standup. +- Keeper - still need to do this - ACTION on JH +- Issue templates - JA has a pull request sat waiting for this so we could use this. Adam also has a template. - ACTION: get this merged and see if we use it +- Burndown suggesting we haven't done enough - It was a difficult one to calculate. This sprint was a bit easier to calculate points for and we seem to be more on track. Hooray! +- Tasks - Not tidied up yet, KB will do so before the end of the week(ACTION). + + +## Items from current retrospective + +### IOC tests turning into a useless metric +We have made a ticket for this and planned/proposed it, we dealt with it at the time. + +### Checkstyle being prioritised over IOC tests +Ditto, we are dealing with it. +They are slightly prioritised because they are much easier fixes than fixing IOC tests and it takes a lot more effort to fix IOC tests. Missing documentation in the GUI is also very important for the future. Checkstyle is an easier graph to look at because it's stable and reported on the jenkins job. We should fix both rather than concentrating on one or the other. + +DO mentioned checkstyle can be a bit heavy handed, it complains if you don't write docs for every function. Should we be doing it on all of our other languages? Yes we should, though this adds another metric that needs to pass + +We should run this on pull requests instead and make the build fail if the warnings go up. JA will put a ticket in for this for the next tech debt. We have a build for this already but it's rather broken and not plumbed in as it stands. + +### It was strange but really good that members were in the office and having a discussion on teams +Meant that people off-site could jump in at points which helped with teamwork and hybrid working. There are some funny side effects with this if you are sat in the office like hearing things twice over but overall it worked OK. When we're back at work we should maybe worry less about distancing and use things like the big screen for standup and so on to get around that problem. + +### Avoiding re-adding release notes back into ReleaseNotes_upcoming in after a release +We should add a ticket to write a script that will rebase all tickets to the current master/main and avoids this problem. This script should be run after doing a release. ACTION: JH create ticket + +### Happy that the Python version being incompatible was spotted while doing the release +It was good that we did the release in the shutdown because mid cycle this could cause panic. +We should be careful about dependency updates because we aren't necessarily building on the same machines as we're deploying to. If BR hadn't have noticed this we could have ended up with problems after creating the release or even worse deploying it. + +### planning board has columns for themes - do they work? has the automation been working +Automation has been VERY helpful and project board checks are now pretty much solely moaning about release notes rather than invalid labels. +themes in the planning board are possibly not that useful. + +### Was the release at the start of the sprint useful? +We used to do it at the end and this time tried to do it at the beginning. Think it was useful, meant it didn't overrun from the last sprint. + +### is the "added during sprint" label helpful and if not shall we remove it +Yes and therefore No. + + +### Discussion around sprint themes +DO does not like themes because as a manager it's tricky to give people the answer to "what should I work on next" without context of the tickets in the theme. Beforehand it was just "pick something from the ready column" but now it takes longer because it's difficult to advise on the tickets we aren't sure about. We also don't talk about it during planning which we used to but we don't really have enough time to do it. We have a large team and it's hard to talk about every ticket because it takes up so much time. Themes give us an idea of things we HAVE to do in the sprint, for a migration for example beforehand there may have been things left out of the migration because we didn't get round to them. + +We have had less planning discussions but this may or may not have had implications on managers and so on as it puts more pressure on them to know about all of the tickets in the theme. + +The problem at the moment is the sprint is completely full with themed items which leaves us with no room to pull things into the sprint if needed. + +What about if we had themes but got people to prioritise the top 3-4 tickets for each theme depending on what is high priority for the themes. + +In pre planning we could ONLY discuss theme stuff to get around having stupidly long planning meetings where we discuss themes AND other proposed tickets. + +We have lots of new starters so the order of the ready column doesn't matter because most people don't pick a ticket off the top of the ready column. We could get component stewards to prioritise the ready column. + +To summarise what we're going to do: +Preplanning: we will only discuss themed tickets that have been proposed by component stewards +planning: we discuss anything that has been proposed since pre planning and generic tickets that don't fit in themes. + +Once the planning meeting is over the general tickets from high/medium will go into the planning board, KB will do this, and the themed tickets will be put into the project board by the component stewards who can then order them based on priority. + +Where is this process documented? +It's in the files in general, we should update the wiki to point at the spreadsheet showing component stewards, who is working on themes etc. + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.09.22.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.09.22.md new file mode 100644 index 000000000..4a02bdc1b --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.09.22.md @@ -0,0 +1,57 @@ +# 2021-09-22 + +## Items from last retrospective + +* IOC tests being a useless metric had - lots of progress but still needs some work +* Checkstyle being prioritized too high - not much else to say +* Good to see people in the office having a teams meetings - again good, now going to start booking meeting rooms though for larger meetings +* Avoid re-adding release notes - ACTION: JH to create ticket +* Incompatible python version spotted when doing release - good +* Themes columns on planning board - current system is nice, automation is helpful +* Release at start of sprint - will be trying it again this time +* "added during sprint" label - opinions unchanged +* Sprint themes discussion - Feelings that themes have caused issues with prioritization + + +## Items from current retrospective + +### Tests not passing +* close to getting them done +* consistent failures taken out, need to keep an eye on fixing them + +### Team morale +* Potential of an away-day, but need to take into account different conference levels +* Possibility of booking a conference room big enough for everyone on site +* FA going to talk to people individually to assess comfort levels +* Office occupancy will be continually assessed, looking to increase occupancy levels in various ways (keep an eye on CO2 monitor) + +### Feeling the distance that being remote has caused +* Feeling that we're doing a good job + +### Burndown chart jumping +* This was mostly due to a lot of big-point reviews going through at the same time, including release and tech-debt week + +### Planning Fridays +* JH not a fan of the concept of planned Fridays as opposed to spontaneously picking up tickets for them, but understands that this could be too chaotic +* Whilst remote especially, best to have some planned conversation before starting to work on tickets to replace the small office conversation that would otherwise occur +* A week of Fridays might be too long, especially getting reviews through +* Next Friday week, consider only starting new Friday tickets on Monday and Tuesday, then only working on existing tickets and reviews for the rest of the week + +### Would people prefer being told which themes they're meant to be working on +* Everyone seems on board with this, KB happy to continue letting people know and willing to move things around if a developer has preferences. +* Assignments posted in "announce" on teams after planning + +### Noting how much IBEX work done at end of sprint +* Suggestion that people let KB know at the end of a sprint how much time was actually spent on IBEX +* Happy to be half-day granularity + +### Working with EMU Scientists +* Good that we were able to talk with scientists, set up kit with them etc. + +### Squish +* Now that we have a floating squish license, should run training for developers so that everyone is able to write tests + +### Information about accessing blockhouses +* Hard to remember safety walkthroughs for all instruments, best to find the scientist to do it for you +* Should be a copy of local rules on/near the door if needed + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.10.28.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.10.28.md new file mode 100644 index 000000000..f6024f04d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.10.28.md @@ -0,0 +1,27 @@ +# 2021-10-28 + +## Items from last retrospective + +* IOC tests are better but still a bit flaky. Some fail on some servers but not others but down to only 4 failing +* We would rather not remove the failing tests but will propose a ticket to fix them +* JH has created a [ticket](https://github.com/ISISComputingGroup/IBEX/issues/6862) to avoid merge conflicts on release notes + +## Items from current retrospective + +### Sprint Planning was Fast +* Likely due to good pre-planning + +### Fridays/Tech-debt +* We are probably having too many of these so need to reduce the frequency +* People generally like having them for a full week but also find it hard to slot the Friday work into other things happening that week +* Could have a nominated number of points for Fridays over the whole sprint but this would be hard to manage +* It's hard to make it a "fun day" when remote and spread over a week +* ACTION: KB will think about ways to make it more fun + +### Moving configs from control-svcs +* ACTION: DO to create [ticket](https://github.com/ISISComputingGroup/IBEX/issues/6863) + +### Managing Squish License +* ACTION: LC to make spreadsheet to manage who has the license (is set up on teams) +* Discussed restarting the server every night but this will kick the CI off so will not do it +* There are instructions on the wiki for how to restart the license server if someone has taken the license \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.11.17.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.11.17.md new file mode 100644 index 000000000..c8715b487 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.11.17.md @@ -0,0 +1,70 @@ +# 2021-11-17 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [@ThomasLohnert](https://github.com/ThomasLohnert) | [@Adam-Szw](https://github.com/Adam-Szw) | [@JackEAllen](https://github.com/JackEAllen) | + +--- + +## Items from last retrospective +- A Squish Spreadsheet has been made by [@LilithCole](https://github.com/LilithCole) to manage who has the license located in General on MS Teams. +- IOC tests are better but still a bit flaky. Some fail on some servers but not others but down to only 2 from 4 failing. +- There are now instructions on the wiki for how to restart the license server if someone has taken the license. +- Friday Tech Dept: ACTION: [@KathrynBaker](https://github.com/KathrynBaker) will think about ways to make it more fun + +--- + +## Items from current retrospective + +## Workflow +### Next Sprint will include a FRIDAY +Try and get everyone in if possible - more fun when people are in the office + food! + +### Timekeeper also moderates voting room +It was agreed that the timekeeper should act as the moderator for voting rooms to enforce voting does not overrun time allocated for an issue. + +### Move some documentation into repos as to wiki? +- URL's should be added to repositories and WIKI to include links to relevant repositories and resources. +- Repositories should include URLS to all relevant information such as WIKI pages, manual locations (where applicable), Server locations and other associated repositories (for example an IOC repo should include URL's to emulator and OPI). +- [@ThomasLohnert](https://github.com/ThomasLohnert) will make an issue for this which includes script generator automatic documentation. +- [@JackEAllen](https://github.com/JackEAllen) will also make issues to tackle adding of URLS to some repositories as Friday tickets in the next Sprint. +- A suggestion mentioned was to try and ensure that URLS are present for each repository as part of PR acceptance criteria. + + +### Large backlog of reviews this sprint a sign of insufficient training or other barriers? +* It was highlighted that reviews are not easy if information reviewers need to review a PR is not provided. To improve on this, It was suggested that PR's should include verbose instructions for testing including: + - Machine locations + - Relevant documentation needed (manuals, wiki pages) + - Tasks involved to test PR are listed in the order that they should be tested. +- A suggestion agreed upon was to modify issue templates to include the header "How to Test" to be completed when the developer is ready to submit work for review to list testing instructions. +- [@JackEAllen](https://github.com/JackEAllen) will update issue template to action this. + +### Implement repository linters to remove code format and styling from review process. +- During discussions about barriers for reviewing PR's, it was suggested that we could add linters as git actions to remove the need to review code format and styling from the review process to allow developers to focus on the logic and speed up the review process. +- It was agreed that this would be nice for some repositories - mainly Python +- It is known that Mantid host [technical training](https://github.com/ISISNeutronMuon/ISISTrainingCentre) within ISIS Computing Division, which can extend to Experimental Controls and others performing computing roles within ISIS. [@JackEAllen](https://github.com/JackEAllen) will contact Mantid to follow up on this to see if we can attend a joint training sessions on Jenkins to provide all members within IBEX the knowledge to create git actions with Jenkins to action the implementation of Linting and other tools to handle code styling and format. + +### Use Squish only for reviews during last week of sprint +Mutual agreement that during the final week of a sprint, the squish license should be used primarily for reviews only. +The priority of squish license usage during the final week of a sprint can be prioritised as: +- High: Reviews +- Medium: Reworks +- Low: New work + +--- +## Equipment + +### The office mic is barely legible for stand-up, is there a better solution we could use? +- [@rerpha](https://github.com/rerpha) will contact [@KathrynBaker](https://github.com/KathrynBaker) / [@FreddieAkeroyd](https://github.com/FreddieAkeroyd) to discuss solutions to resolve this. Options discussed included: + - Mic mounted to the ceiling + - Mic that everyone can hand around the office (Not the most Covid secure option) + - Order a VC system if cost effective so we can hear and see everyone in the office. + +--- + +## Other + +### Would everyone be interested in organising a secret Santa this year to get us in the festive mood? +Everyone is interested in a secret Santa and festive meal in the new year. +- [@JackEAllen](https://github.com/JackEAllen) will organise the [secret Santa](https://www.elfster.com/). +- [@LowriJenkins](https://github.com/LowriJenkins) will organise the meal for the new year. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2021.12.15.md b/doc/processes/retrospective-notes/Retrospective-Notes-2021.12.15.md new file mode 100644 index 000000000..3a2cad0a2 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2021.12.15.md @@ -0,0 +1,66 @@ +# 2021-12-15 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [@JackEAllen](https://github.com/JackEAllen) | [@daryakoskeroglu](https://github.com/daryakoskeroglu) | [@Adam-Szw](https://github.com/Adam-Szw) | + +--- + +## Items from last retrospective + +### Next Sprint will include a FRIDAY + +### Timekeeper also moderates voting room +We didn't get to test it yet. We should try this at the next opportunity. + +### Move some documentation into repos as to wiki? + +### Large backlog of reviews this sprint a sign of insufficient training or other barriers? +We made some effort to increase amount of reviews coming through, especially among junior team members. +As we are making effort towards adding more of 'how to review' instructions to our tickets, we need +to keep in mind not to post any security risks that come with having our project be a public repository. These +could for example include IP addresses, device names etc. + +### Implement repository linters to remove code format and styling from review process. + +### Use Squish only for reviews during last week of sprint +Mutual agreement that during the final week of a sprint, the squish license should be used primarily for reviews only. The priority of squish license usage during the final week of a sprint can be prioritised as: +- High: Reviews +- Medium: Reworks +- Low: New work + +We tried to follow this ruleset during this sprint and it seems to have worked. We agreed to keep this in place. + +### The office mic is barely legible for stand-up, is there a better solution we could use? +This issue hasn't been dealt with yet. + +### Would everyone be interested in organising a secret Santa this year to get us in the festive mood? +We did the secret Santa and the team enjoyed it. We are yet to organize a new year's meal. + +--- + +## Items from current retrospective + +### Centralise the snag lists to the IBEX project sharepoint +These are generally not as readily visible as the projects etc. are in GitHub + +### Apart from trying to be organised next time and realise in better time when it isn't worth having a pre-planning meeting, are we happy with not having one when it feels unhelpful? +Considering how this is a rare situation, we agreed that the best way to deal with it will be asking this question at the end of the standup, so that people can have an opportunity to voice their opinions on this. + +### This keeps getting brought up, but I think we should have a 'cull' of icp-write members for the sake of security in case anyone's GH account is compromised and they are difficult to reach +For the security reasons we agreed to: +- Everyone should make sure that they are using 2-factor authentication on Github +- We might want to look into removing members who have not been on the project for a long time + +### Start over the rotas in the new year +We decided to go for it as nobody had an issue with that + +### Some new members of the team have trouble merging branches on various repositories due to lack of permissions + +### What is the best way not to forget to update submodules - should `EPICS_repo_checks` post a message to Teams on failure? +- We decided not to add message on teams. +- We might consider adding checks or 'todos' in the templates to remind people. +- Optionally we could add a pre-commit message check. + +### When adding new sections or editing existing sections of the wiki, I would like descriptions to be as verbose as possible to be more accommodating to new members within the team +We can make things more verbose where we can, but we have to look out for possibility of duplicating information. Where it exists, we should favour linking other pages rather than keeping extensive amounts of information on one page. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.01.05.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.01.05.md new file mode 100644 index 000000000..216feebfc --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.01.05.md @@ -0,0 +1,54 @@ +# 2022-01-05 + +### Items from Retrospective before last: + - The office mic is still often illegible. + - as of this meeting still an issue. + +### Items from Previous Retrospective: + - Centralise the snag lists to the ibex project sharepoint. + - not discussed as Kathryn did not have access to reflectometry. + + - Cull icp-write members for security: + - Require users to have 2-factor authentication on github. + - This will remove everyone who doesn't have it already enabled. + - This may handle most of those who need to be removed. + - This will only effect write access. + - Look into removing members who have not been members of the project in some time. + - Change them to read-only? + + - New team members still running into permissions problems on some repositories: + - Often issue is to do with different permission settings for writing and merging. + - Might be possible to automate adding new members to permissions using github actions. + - Jack Allen to look into this. + + - Best way to handle reminders about updating submodules. + - Ideas: + - Message on teams? + - todo in issue template? + - pre-commit message check? + - Repo-checks do actually catch this, but this usually delays being noticed until the next standup. + + - Adding to/Editing wiki sections. + - Try and make things more verbose, but if information would be duplicated then using links instead. + - Finding things can be extremely difficult on the wiki, so: + - Try and make names of pages clear and relevant. + - The wiki likely needs a cleanup, probably still the best option for documentation though. + - Searching better at top left within repository, rather than using the wiki provided search-bar. + + +### Items from this sprint: + - Discussion of how we want to handle versioning, differences between Major/Minor/Patch: + - What we would classify as a major change and what the scientist would may differ greatly. + - If we want to change this, we might have to change how we deploy, or how we number versions, or both. + - If these things change we may also have to alter our build system to avoid changing more than is necessary. + - Discussion of deployment at other facilities. + - There doesn't seem to be a common approach among them. + - More use of static builds with unrolled DBs? + - multiples versions of asyn at a lot of other facilities. + - We've avoided this for ease of testing. + - Discussion to be continued at next retrospective. + + - This sprint was very short: + - Seemed to work fairly well, and was well tracked by shadow. + - Was useful as a tech-debt sprint, similar to Mantid's post-release period. + - Up to date calenders would make it easier for sprints similar to this to be planned in future. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.02.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.02.md new file mode 100644 index 000000000..ff7c67491 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.02.md @@ -0,0 +1,124 @@ +# 2022-02-02 + +## Items from last sprint: + +Can we make our deployment process clear with respect to version numbers of current and previous? Major/Minor/Patch version. +How do we account for e.g. hotfixes, roll-backs, patches, etc. Or a deployment of a single IOC, or a support module? (e.g. StreamDevice) + +- Perhaps link to wiki page from GUI `About Box` showing versions and hotfixes (exists already). Or an ID suffix on version number? +- Each IOC and module has a version number created during build. Can then compare. +- How do other facilities handle this? +- Decided need to discuss in separate meeting. + +------------------------------- + +## Items from Current Sprint (2022_01_06) + +How to account to support work? + +- comment on `Support Channel` in Teams +- KVLB will review at end of sprint and account for points spent + + +What would everyone's opinion be on trying to bring in one or two issues each sprint to automate some of the manual tests we do as part of preparing for releases to slowly begin chipping away at them to speed up release testing? + +- Use Squish as we have it +- Always good to speed up release testing +- Difficult to prioritise testing higher than instrument work +- Occasionally convert one test after creating ticket and then review +- Have a list of tests/tasks/tickets on wiki for "spare moments" e.g. on Friday afternoon +- Decided to get ISIS cycle(s) out of the way first and then have another think, probably a meeting. + + +Would revisiting the concept of code freezes be a good idea? + +- Already do something approximating code freezes +- Discuss at meeting above if required + + +Can we consider using GitHub Discussions to contain technical conversations? I feel it may help us as a team to better organise and archive technical discussions taken place under custom categories and pin important conversations (sort of like an internal stack overflow). I believe it may also help reduce duplication of questions asked and answered within the team. + +- Group worried about number of search locations +- Any technical question should already be answered on wiki, add it if not. +- Suggested that JA give a Code Chat on the subject +- UKRI Universal Documentation Project ("UDOC") - option for our manuals(s)? + + +Might be interesting to just discuss the idea of the ICD lightning talks we attended this week, seemed an interesting idea. Maybe we could put an odd slot in for one or two (e.g. one after stand-up once a week?). Not necessarily to to do a block meeting of them but let a few technical tricks and ideas out of the bag in a 1-2 minute slot. Could also be presented to ICD if appropriate. + +- For info: 4 to 6 months is frequency for ICD talks +- Book weekly slot and people can volunteer if have something to talk about +- After Stand-Up meeting is ideal, if time allows and not running in to next meeting +- Give people option not to attend if not interested +- Suggestion is to have after Stand-Up of final day of sprint + + +What would everyone's opinion be regarding merging the Social and random teams channels together to reduce the number of channels we have? +Is there history in one or both chats that we particularly care about and would prefer not to lose? + +- Group not been using certain channels very often, so no harm in removing. Especially "lunch" as was only used in lock-downs. +- Always option for individuals to hide any channels on own client +- OK to delete "social", "lunch" and "reviews". DONE. +- Suggestion to have "system" channel for CMM to record activities. e.g. reclone system, fix RAM, alter GFX car, move physical machine or VM. May be better on Sharepoint on a per instrument basis. + + +Can we discuss [this ticket](https://github.com/ISISComputingGroup/IBEX/issues/6973#issuecomment-1015399722) as a group to look at automating the release build? +- all documented on wiki, so assumed straightforward to create Jenkins job. +- deployment more involved and risky. start task with just automated release and then run system tests. +- create ticket for automating release, not deployment. Win10 deployment is very different to current system, so will revisit when in place. + + +If we can be more descriptive / verbose with PR titles and descriptions, I like the idea of using the auto generated release notes feature on GitHub for release tags going forward. Currently PR's vary quite a lot in how well they are written so the auto generated release notes don't look very nice. +If we can be stricter with how well we write PR's and commit messages (title and body), I would like to place the Release notes link in the release tag like we do already and then the auto generated release notes beneath that to be more descriptive with the changes included in each release tag per repository. + +- with tags, can automate release notes by selecting appropriately +- summarise commits in PR +- possible to create "draft release" as nightly build? +- can't see benefit. difference in customer and developer release notes. may not work for customer, as specifically created for instrument. +- suggestion to create demonstration release using most recent release for comparison. + + + +We probably don't need to buy a mic for the office webcam now, but it's probably worth shutting the door at standup as people walking past the office can be picked up on it now with the "low" noise suppression option ticked on zoom. Otherwise though I think it's much clearer! + +- camera angle too narrow to show all present. +- suggestion to try separate mic and camera + + +`DDaT` launch PC recycling and sustainability scheme (sharepoint.com) Volunteers required to re-image obsolete (but still useful) IT equipment for resale to staff or donation to charity. Scheme currently being run from Polaris House, but maybe if/when expanded to RAL, could be an idea for a team-building exercise (or perhaps more of a bus man's holiday!) + +- probably more useful team-building exercises we could do, like joint programming, or hackathon. + + +------------------------------- + + +### Extra comments: + +Run out of tickets for people to pick up during sprint, so bear in mind for planning meeting. + +- probably due to large tickets (e.g. deployment, release) +- ask on General channel so can help with work on current themes + + +Are we planning a late Christmas meal, if not then something else e.g. pizza or something on site? + +- can consider now restrictions relaxed +- respect opinions if people don't want to attend or can't +- can book large conference room for social distance for activity e.g. lunch +- need ideas for numbers attending so that office capacity accounted for +- can meet at Dish on Harwell campus for those on site? + + +OK to pick up "Friday" ticket at any point? + +- Yes if all other sprint work done +- useful for training + + +Sprint Review: + +- Release created and deployed on time +- See how deploying early worked for the upcoming sprint/cycle - did we create more patches as a result? Did scientists test early as that was the whole point? + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.23.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.23.md new file mode 100644 index 000000000..967d50d2f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.02.23.md @@ -0,0 +1,83 @@ +# 2022-02-23 + +## Items from last sprint: + +How to account to support work? +- Some tickets were created with appropriate amount of points in them + +What would everyone's opinion be on trying to bring in one or two issues each sprint to automate some of the manual tests we do as part of preparing for releases to slowly begin chipping away at them to speed up release testing? +- Use Squish as we have it +- Always good to speed up release testing +- Difficult to prioritise testing higher than instrument work +- Occasionally convert one test after creating ticket and then review +- Have a list of tests/tasks/tickets on wiki for "spare moments" e.g. on Friday afternoon +- Decided to get ISIS cycle(s) out of the way first and then have another think, probably a meeting. + + +Would revisiting the concept of code freezes be a good idea? +- Not discussed in this retrospective + +Can we consider using GitHub Discussions to contain technical conversations? I feel it may help us as a team to better organise and archive technical discussions taken place under custom categories and pin important conversations (sort of like an internal stack overflow). I believe it may also help reduce duplication of questions asked and answered within the team. +- Not discussed in this retrospective + +Might be interesting to just discuss the idea of the ICD lightning talks we attended this week, seemed an interesting idea. Maybe we could put an odd slot in for one or two (e.g. one after stand-up once a week?). Not necessarily to to do a block meeting of them but let a few technical tricks and ideas out of the bag in a 1-2 minute slot. Could also be presented to ICD if appropriate. +- For info: 4 to 6 months is frequency for ICD talks +- Book weekly slot and people can volunteer if have something to talk about +- After Stand-Up meeting is ideal, if time allows and not running in to next meeting +- Give people option not to attend if not interested +- Suggestion is to have after Stand-Up of final day of sprint + +What would everyone's opinion be regarding merging the Social and random teams channels together to reduce the number of channels we have? +Is there history in one or both chats that we particularly care about and would prefer not to lose? +- Channels merged, topic closed + +Can we discuss [this ticket](https://github.com/ISISComputingGroup/IBEX/issues/6973#issuecomment-1015399722) as a group to look at automating the release build? +- Not discussed in this retrospective + +If we can be more descriptive / verbose with PR titles and descriptions, I like the idea of using the auto generated release notes feature on GitHub for release tags going forward. Currently PR's vary quite a lot in how well they are written so the auto generated release notes don't look very nice. +If we can be stricter with how well we write PR's and commit messages (title and body), I would like to place the Release notes link in the release tag like we do already and then the auto generated release notes beneath that to be more descriptive with the changes included in each release tag per repository. +- Discussed as part of current retrospective, notes below + +We probably don't need to buy a mic for the office webcam now, but it's probably worth shutting the door at standup as people walking past the office can be picked up on it now with the "low" noise suppression option ticked on zoom. Otherwise though I think it's much clearer! + +`DDaT` launch PC recycling and sustainability scheme (sharepoint.com) Volunteers required to re-image obsolete (but still useful) IT equipment for resale to staff or donation to charity. Scheme currently being run from Polaris House, but maybe if/when expanded to RAL, could be an idea for a team-building exercise (or perhaps more of a bus man's holiday!) + +## Items from current sprint + +How patched is the release? Was the timings good for us, did we get the change requirements before the cycle started? +- We still need some tidying to be done on sans2d +- Patching was made easier thanks to visual studio version being the same and +because we had a developer build +- We had releases done by the time of testing on the instrument giving us extra time +- Overall we are happy with this. This workflow should be documented on wiki + +If we can be more descriptive / verbose with PR titles and descriptions, I like the idea of using the auto generated release notes feature on GitHub for release tags going forward. Currently PR's vary quite a lot in how well they are written so the auto generated release notes don't look very nice. +If we can be stricter with how well we write PR's and commit messages (title and body), I would like to place the Release notes link in the release tag like we do already and then the auto generated release notes beneath that to be more descriptive with the changes included in each release tag per repository. +- We looked at the draft of auto-generated release notes +- We decided that we should think about making pull request titles more descriptive but +we will not be spending time on auto release notes + +Would it help to know the points expected at the start of planning meetings to help with the awareness of what we can do? +- We will implement this and see how it goes for a couple of sprints + +Do we want to rethink the way we approach discussion tickets? We often spend time discussing a discussion, then ask about MUST/SHOULD/COULD/WANT/NOT, then finally (hopefully) have the discussion. There are quite a few of these "decide on architecture" discussions in the backlog, and they can impede the code being done as we don't get around to the discussions. How about I (or someone else if they want to) books an architecture meeting for an hour or so every sprint or two (as with planning and review/retro they won't count for points). The agenda for the meeting will be based on recent proposals to begin with, and shared in advance. A backlog of discussion tickets can then be created and maintained offline (rather than in existing meetings). +- We should treat those tickets as a separate backlog for this purpose +- Meetings will be setup and we will try this for some time to see how it goes + +Should we think about splitting planning during cycle too? +- We will be splitting during the cycle too so that team members that could be on +support during one meeting can then join another meeting + +We frequently run into problems with Eclipse RCP environments (IBEX client and Squish), would it be worth sending some people on some kind of training courses? I feel a bit guilty as I haven't found a lot of time recently to help people resolve these and I'm always just kind of muddling through myself, I think having some more expertise in the team would be useful. Anyone interested be learning more about this? +- We could send someone to a training on this and then that person would be able to help rest of the team with Eclipse related problems. We need to decide who should attend the training + +I don't really understand the distinction between https://github.com/ISISComputingGroup/IBEX/wiki and the developers manual - should we merge the two? +- There is a reason to the separation. One is for developers only and the other for everyone +- We might have some content that's in wrong places that needs to be looked at +- We could use some version history but we are not urgent about this + +Should we have a "team day" every month/other week where everyone comes in? +- Yes. There is a proposed date of 25th of March + +## Additional notes +- We discussed the issue of camera in the office diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.03.23.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.03.23.md new file mode 100644 index 000000000..a33f9c74e --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.03.23.md @@ -0,0 +1,109 @@ +# 2022-03-23 + +## Items from last sprint + +How patched is the release? were the timings good for us? +- Not very patched compared to last time we looked at it. SANS2D have a few patches but are happy. +- Overall happy with release timings. Workflow is still not documented on the wiki. + +PR auto generated release notes +- Extra overhead but we should be better at titling PRs anyway for tracking + +Points expected at start of meetings: +- still trying this + +Discussion tickets: +- Now trying architecture meetings, we need another one but it might not be this sprint given we've got cycle, release and deploy +- we should do one about networks but we should hold fire as there will be changes + +Should we think about splitting planning during cycle as well as review/retro? +- We are leaving as one for now. + +Eclipse RCP training: +- has been decided that a few people are going on a training course for it. Freddie may also do the course as well as he has a license. + +Should we merge IBEX/wiki and the developers manual? +- some things will be changed and moved around, this is still in progress. +- instrument info moving to the dev manual, this is in progress +- main ibex manual is being rearranged a bit, "" + +Team day: +- we have this coming up on the 25th of March + +Camera in the office: +- now even wider angle +- not pointing in the right direction at the moment +- stack of books or mounting it on the wall might be an idea to make it even wider + +## Items from this sprint + +Can we set a challenge to the team to get stale reviews done and get the project board checks to stop moaning and giving warnings? +We can cheat about this slightly to stop them getting stale +- There's always a reason things are stale? No, sometimes it's just the way it is. +- Generally if it's at the top of review it's either been there the longest or it's very urgent. +- This is more of a reminder rather than a discussion. Why do we have things in review that have been there for months? + - Hard to review/not urgent/no one understands it anymore perhaps - we should get these done anyway. + - We should say about them more in standup and get them through review + - If it's hard to review, perhaps we should give more info in the ticket on HOW to review it. + - This is getting better, we are moving in the right direction. Some are squish test tickets which can be tedious to review. Perhaps we should have a discussion at standup if there is a ticket that is difficult to review. + - pair review may be an idea to get through the tickets that take a lot of time. + +Should we hit the button to get rid of everyone from the GH org who does not have 2FA enabled + - we need a way of doing this for `ISISBuilder` + - we might be able to do this via an app or email rather than text message for it. + +We should have a known issues page for accelerator beam stuff being down like the MCR news that the scientists can use for update +- Should be something we can all edit +- why don't we use the shared inbox? because people who want to know might not be scientists so we cant just email all of the scientists. +- we should put this alongside where our shared phone number is listed and put it on that page as it might stop people from calling +- we need a discussion with the internal web admins in the computing group? to discuss this + +Zoom now works if you're logged in from multiple places. hooray! + +INTER could not find the new zoom support phone number so ended up coming to the office instead. the previous number is now redundant as all landlines have been removed. +- We should get some cards laminated again with the details on as the ones round site are out of date. ACTION: get these printed. Perhaps we could put a QR code of the our web page on it. And a picture of a goat! DKg would like to do the designing of these cards. KB will send DKg the previous one +- 1763 should still work from the existing landlines in cabins, but will ring to zoom. We couldn't ring them back without looking up their number on the intranet +- this is inconsistent as WISH seemed to come through with their own numbers... +- If you add them to contacts they show up as their instrument name. + +We decided to move parts of the IBEX wiki into the code repo, would it be an idea to do the same to the dev wiki? +- this makes it easier to view version history +- you can still edit it online and it makes it slightly clearer to view the structure of the wiki +- this is easy to do +- this will break the wiki check so we should keep this in mind - though this is easy as well +- one click less to have the same info +- we should double check this won't break relative links etc. and retain history + +Graylog move to SCD cloud +- created a Friday ticket for this, didn't quite get it working +- the ticket for making `graypy` more hardy should be treated with a higher priority than the instance location as it caused some scripts to fail + +Hackathon to move all IOCs into support directory +- we should do this instead of the next Friday as a hackathon, they are fun +- ACTION KB will propose a date + + +Sigh of relief for TS1 commissioning being pushed back +- we are doing well at supporting TS2 I think - so might be a good thing that we can carry on just doing so as it gives a chance for junior members of the team to help with support more. +- there are positives and negatives for this +- more work may need to be done in a shorter time, so there is an element of panic here too. It also mixes up our timelines slightly for things like win10 etc. + +What is making people sad/mad/glad +- There were 49! conversations in the support issues channel. We have been very busy but we are doing well. +- It has calmed down a bit now, we have managed to jump the hurdles well. + +- It would be good for junior members of staff to come along with support calls especially ones that require being in person +- hard to do this remotely as we don't immediately log everything in support-issues as it is happening +- there is no shame in saying "I'm not sure let me check with the team" +- Would the Sandwich students like to come along to support calls? yes +- There is a sense of ownership when you pick up a support call but this does not mean you're solely responsible for that problem + +Call queues - has the ring timer for call queues for level 1 been reduced? It seems to ring for a very short time and then go to the next level - this can make it difficult to have time to get everything ready to answer the phone +It's too short. Perhaps we should get everyone a lab phone as it's easier to accept a call on a phone than with a headset. + we will increase the ring time for 15 seconds as 10 seconds was too short to be able to answer it before moving onto the next person + +In terms of the amount of support calls we got done we should be proud of ourselves as we've been off sick/on leave etc as a team. We shouldn't be disheartened by the state of the burndown - we have been doing brilliantly. + +As someone who cares about the user interface side of things I am glad that we've had lots of changes recently - they are tangible things that show up and show change in IBEX so although they might be seen as "nice to haves" they are very useful. + +Lightning talks - we forgot what was happening so that was the reason we didn't have much to talk about. SJ will put a note on the code/gui chat page diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.04.20.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.04.20.md new file mode 100644 index 000000000..d0b687497 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.04.20.md @@ -0,0 +1,74 @@ +# 2022-04-20 + +## Items from last sprint + +Stale Tickets: + - We have warnings for long periods, perhaps there should be errors for even longer ones? to force us to look into ones that have been there for a long time. + +2FA on on github: + - Make sure every account we care about has it enabled because others will be kicked out. + - Possibly send an email, though github may actually do this itself. + - ISIS build does have 2FA. + - Done. + +Zoom contact details: + - Check for old support number and remove it (just TS2 for now) + - Print out new laminated ones + - Its possible that the call queues are mangling caller ID, might need to test it. + - Possible that if its forwarded from queue 1 to 2 it might say its from call 2. + - This can cause issues with calling people back. + - Some have caller ID and some don't. + +Moving Wiki into the code repo: + - Would be just moving the ibex wiki repository to the main repository. + - Possibly hold off to evaluate a different solution that Jack is looking into. + - Docusaurus, Jack may be hosting a talk on this. + - Good built in search engine, same one as stackoverflow. + +Graylog move to SCD cloud: + - No updates to getting it working on SCD just yet. + - Check with SCD cloud support. + - Running in docker has been somewhat unreliable. + - We don't currently have Graylog running. + +Moving IOC tests and emulators to support directory: + - Happening on Friday. + + +## Items from this sprint +Cable Testers: + - It would be good to know if we had dead cables before giving support calls. + - Patch testing for Pearl + - Possibly worth borrowing from SCD in short term but might be better to have our own. + - Check with Anthony Shuttle for suggestions. + - We're not network engineers, but useful to know that the cables we're bringing around work. + - Possibly this: + - `https://www.amazon.co.uk/dp/B07WRV3N5F/ref=sspa_dk_detail_0?psc=1&pf_rd_p=828203ef-618e-4303-a028-460d6b615038&pd_rd_wg=V07Nd&pf_rd_r=9DJ8RGN71F7P4VQGVWHY&pd_rd_w=2aYQS&pd_rd_r=4daea718-1c03-482c-89c5-3ae3a10df11c&s=diy&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEyOEhJUUNEMkxYVDhMJmVuY3J5cHRlZElkPUEwNTg1NzIyWlNNRlNINUlLT05XJmVuY3J5cHRlZEFkSWQ9QTA4NzIwNTYzTDFJN0Q0WlhJRjZBJndpZGdldE5hbWU9c3BfZGV0YWlsJmFjdGlvbj1jbGlja1JlZGlyZWN0JmRvTm90TG9nQ2xpY2s9dHJ1ZQ==` + - More expensive but less risk of damaging things. + +Github Discussions: + - How to use this compared with the wiki. + - When the discussion is solved does it move to the wiki? + - Perhaps ticket based discussions? + - Discussions tickets are often organise a meeting about this topic, rather than just discussing it on github, using discussions wouldn't allow us to point it. + - Private discussions probably only possibly at the organisation level or in a private repository. + +Should we continue drop in sessions in cycle: + - We never really get many people, and when it is they often just ask about a support call. + - Why do they not turn up at them, do they not find them useful or are they too busy etc. ? + - Maybe they shouldn't happen in cycle + - FIT don't seem to do them anymore or if they do we're not on the list. + - They're less of a burden thanks to zoom. + - We can ask the scientists if they want us to keep doing them at a meeting. + - They are the customer so we should let them decide. + - Might be a way to not spam them with emails once a month. + +System Tests: + - Still some transient errors, but they are actually passing occasionally. + - Possibly autosave saving state or something similar? + - At least we know they're mostly working now, so we can pay more attention to errors. + +This Sprint: + - Test galil was very useful for the wish collimator debugging + - Hopefully next cycle will be a bit calmer because no migrations. + - Lets us focus on Win10 and other migrations for summer like TS1. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.05.18.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.05.18.md new file mode 100644 index 000000000..473631931 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.05.18.md @@ -0,0 +1,91 @@ +# 2022-05-18 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [@ThomasLohnert](https://github.com/ThomasLohnert) | [@davidkeymer](https://github.com/davidkeymer) | [@JackEAllen](https://github.com/JackEAllen) | + +--- + +## Items from last retrospective + +Cable Testers: + - It would be good to know if we had dead cables before giving support calls. + - Patch testing for Pearl + - Possibly worth borrowing from SCD in short term but might be better to have our own. + - Check with Anthony Shuttle for suggestions. + - We're not network engineers, but useful to know that the cables we're bringing around work. + +*Cable Testers have arrived and work quite well!* + +Github Discussions: + - How to use this compared with the wiki. + - When the discussion is solved does it move to the wiki? + - Perhaps ticket based discussions? + - Discussions tickets are often organise a meeting about this topic, rather than just discussing it on github, using discussions wouldn't allow us to point it. + - Private discussions probably only possibly at the organisation level or in a private repository. + + *Leave as is for the moment and remove if we can't find a use for it after a significant period of time has passed with no usage* + +Should we continue drop in sessions in cycle: + - We never really get many people, and when it is they often just ask about a support call. + - Why do they not turn up at them, do they not find them useful or are they too busy etc. ? + - Maybe they shouldn't happen in cycle + - FIT don't seem to do them anymore or if they do we're not on the list. + - They're less of a burden thanks to zoom. + - We can ask the scientists if they want us to keep doing them at a meeting. + - They are the customer so we should let them decide. + - Might be a way to not spam them with emails once a month. + +*A survey has been sent out to ask for feedback on this. We should have an answer by the next Retrospective meeting regarding whether we should continue with drop in sessions as they are or not.* + +System Tests: + - Still some transient errors, but they are actually passing occasionally. + - Possibly autosave saving state or something similar? + - At least we know they're mostly working now, so we can pay more attention to errors. + +*Still seem to get transient errors, but far less frequent. Builds VHD broke on an update, but otherwise fine.* + +This Sprint: + - Test galil was very useful for the wish collimator debugging + - Hopefully next cycle will be a bit calmer because no migrations. + - Lets us focus on Win10 and other migrations for summer like TS1. + +*The cycle has been quieter that the last with no serious issues such as the collimator. Support calls in the morning have become far less frequent.* + +--- + +## Items from current retrospective + +## Workflow + +### Windows 10 and Other Migrations. +Win10 and other migrations should be one of the main focuses over the summer. + +### Disk cleaning Automation +Time should be spent looking into using [@ChrisM-S](https://github.com/ChrisM-S) powershell script to automate cleaning of drives, moving over IOC logs periodically as part of a cron job. + +A discussion ticket should also be written up to discuss how we could automate truncating the database without affecting active runs. + +### Splitting up GitHub Issues More Finely + +A discussion took place about splitting up issues that involve writing IOC's, Emulators and systems tests as the work is usually always contained within 1 large issue. It was agreed to keep this as it is and change how we approach developing the IOC, system tests and Emulators and document the process in the wiki. + +The process discussed was to write the minimum functionality required to have a working IOC with only one function, and system test and gauge how long it would take to write the rest based on the time required to write one function. +It was also agreed that development should be done in a way that allows for the work to easily be split into additional issues if needed. + +### Sprint Lengths +How we currently run sprints was discussed in relation to how we adapt the length based on cycles and support. It was agreed that [@KathrynBaker](https://github.com/orgs/ISISComputingGroup/people/KathrynBaker) will spend a couple of weeks looking into trialing a nested sprint where sub-sprints occur for each theme within the main sprint and present this in the next Retrospective. + +### Support During Cycle +Support during the cycle has been eerily quiet... + +### Beckhoff +Small Beckhoff in a box is missing? maybe on IMAT? +Look into buying test license for Beckhoff. +Keeps hearing about weird errors from Beckhoffs - Pester motion control to get a fix. + +## Equipment + +New docks have arrived! + +Network Cable Tester arrived works well! diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.06.15.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.06.15.md new file mode 100644 index 000000000..f9210ab66 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.06.15.md @@ -0,0 +1,103 @@ +# 2022-06-15 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +|[@LilithCole](https://github.com/LilithCole) |[@LilithCole](https://github.com/LilithCole) | [@daryakoskeroglu](https://github.com/daryakoskeroglu) | + +--- + +## Items from last retrospective +## Workflow + +### Windows 10 and Other Migrations. +Win10 and other migrations should be one of the main focuses over the summer. +_No change, still as it is._ + +### Disk cleaning Automation +Time should be spent looking into using [@ChrisM-S](https://github.com/ChrisM-S) powershell script to automate cleaning of drives, moving over IOC logs periodically as part of a cron job. + +A discussion ticket should also be written up to discuss how we could automate truncating the database without affecting active runs. + +_Did not have a chance to discuss this sprint, need a discussion this sprint._ + +### Splitting up GitHub Issues More Finely + +A discussion took place about splitting up issues that involve writing IOC's, Emulators and systems tests as the work is usually always contained within 1 large issue. It was agreed to keep this as it is and change how we approach developing the IOC, system tests and Emulators and document the process in the wiki. + +The process discussed was to write the minimum functionality required to have a working IOC with only one function, and system test and gauge how long it would take to write the rest based on the time required to write one function. +It was also agreed that development should be done in a way that allows for the work to easily be split into additional issues if needed. + +_Agreed that the process will be used_ + +### Sprint Lengths +How we currently run sprints was discussed in relation to how we adapt the length based on cycles and support. It was agreed that [@KathrynBaker](https://github.com/orgs/ISISComputingGroup/people/KathrynBaker) will spend a couple of weeks looking into trailing a nested sprint where sub-sprints occur for each theme within the main sprint and present this in the next Retrospective. + +_Kathryn did a presentation about nested and sub-sprints in this retrospective_ + +### Support During Cycle +Support during the cycle has been eerily quiet... + +_No cycle this Sprint, hence another quiet Sprint_ + +### Beckhoff +Small Beckhoff in a box is missing? maybe on IMAT? +Look into buying test license for Beckhoff. +Keeps hearing about weird errors from Beckhoffs - Pester motion control to get a fix. + +_@rerpha is discussing about buying test licence for Beckhoff with the motion team._ +## Equipment + +New docks have arrived! + +Network Cable Tester arrived works well! + +_People who started using the new equipment are happy with them_ + +--- + +## Items from current retrospective + +### Project board channel on MS Teams stopped working +No one is using this channel so it was agreed that it should be removed. +@FreddieAkeroyd knows the best way to remove it from channel list. + + +### Rota table in "Meeting Roles and Rotas" wiki page is confusing +"Week Commencing" column causes confusion as it only relates to the Stand-Ups. + + +It was agreed that @KathrynBaker will split table in 2, one for Sprint Planning meetings and other for Stand-Up meetings. + +### Alternative to Plan It Poker +Plan It Poker webpage was down in the last Pre-planning meeting but it came back to life. + + +@KathrynBaker found a new website alternative to Plan It Poker: Scrumpy + + +Benefits: add issues direct from GitHub, add comments to the chain and more user friendly. + + +The members decided to try for the next Sprint Planning meeting. + + +Kathryn demonstrated how to link to GitHub. + + +No registration needed, can log in with a GitHub account but not necessary. + + +### Suggestion for different ways of managing the work in Experiment Controls +@KathrynBaker did a presentation about a different approach of managing work we are doing in Experiment Controls. +Key notes from the presentation: +* Consider all the work we have to do as "program". +* Program Increment (PI) would be things to be done between now and next release. +* Sprint is still a sprint, instead of following Scrum, we can aim to reduce Work In Progress and reduce sprint length to find a better rhythm. +* Each program will have a Kanban board: Program Kanban, Program Increment Kanban, Sprint Kanban +* More information about this management approach can be found here: https://www.scaledagileframework.com/program-and-solution-kanbans/ +* Rough timeline suggested: +* * Sprint 2022_05_19 We decide if we would like to try -- The voting happened on zoom, we will give it a try. +* * Sprint 2022_06_16 Make sure we are carrying nothing forward, and getting everything we need in for the next release done. +* * Sprint 2022_07_14 Build release, testing, prepare for training. +* * Sprint 2022_08_11 IBEX Training, deploy to TS1, hackathon/tech debt removal/something, PI planning, sprint planning +* * Sprint 2022_?_? Sprint retrospective for previous sprint, sprint planning diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.08.10.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.08.10.md new file mode 100644 index 000000000..64abf68f5 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.08.10.md @@ -0,0 +1,69 @@ +# 2022-08-10 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [Lowri Jenkins](https://github.com/LowriJenkins) | [Jack Harper](https://github.com/rerpha) | [Thomas Lohnert](https://github.com/ThomasLohnert) | + +--- + +## Items from last retrospective + +_(discussed these items very briefly to get to Kathryn's new items before she had to leave)_ + +#### Disk Cleaning Automation: +- Job could be automated but it's not trivial. No new updates for now +- Discussion on a possible solution will be added to our list on Teams (Thank you for volunteering [@DavidKeymer](https://github.com/davidkeymer )!) + +#### Splitting up issues more finely: +- We have not had an opportunity to implement this yet in the last sprint. + +#### Quiet cycle: +- Cycle has continued to be relatively quiet in terms of support. + + +--- + +## Items from current retrospective + +### Sprint + +#### Should we delay start of next sprint? +- KB suggested sprint planning/start for the coming sprint can be be delayed by a week to finish off large volume of items in ready/in progress +- Agreed upon by all + +#### PlanitPoker vs. Scrumpy - which is better? +Scrumpy Pros: +- Github tickets are more easily tied into the service +- Asynchronous voting - People can vote in advance if they will be away for planning + +Cons: +- You can only be in one room at a time, difficult to do priority / points in parallel. We could use both in a hybrid solution but might create more overhead than utility + +We will stick with PlanitPoker for now but keep Scrumpy in mind as an adequate backup if needed. + +### Customer relations + +#### Are people interested in uniforms (e.g shirts) for representing IBEX on support calls? +- Jack H feels that users can be confused about our role sometimes when we turn up on support calls +- Other support teams have specific outfits to clearly identify themselves as qualified support staff +- This does not need to be mandatory, but it might be nice to have the option +- Not everyone likes uniforms, can we have other items to identify ourselves? e.g. hats +- Jack H will investigate available ISIS "branding" options and report back at next retrospective + +#### Support phone number: +- New label with updated contact information has been printed and displayed in TS1 & TS2 blockhouses. Done! + +### Team Interaction + +#### Coffee attendance is extremely low, should we find an alternative for team building? +- Attendance has fizzled to near zero - perhaps a symptom of more people being back in the office +- Suggestion to steal Weekly Coffee & Cake idea from Mantid instead. People in the room agreed this sounds like an appropriate rhythm. +- We will do this in a conference room to enable people to join remotely. Should be convenient for conference room availability. +- We have picked Friday afternoon as the regular time for this for now. We can consider varying the days of the week this takes place if this otherwise excludes anyone from attending. + +#### Should we have standup in G06 instead of the office?: +- Team generally in favour, (more space, better sound). +- Availability may present an issue +- We will trial it when the room is next free in the morning. + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.09.07.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.09.07.md new file mode 100644 index 000000000..950ea6b1f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.09.07.md @@ -0,0 +1,126 @@ +# 2022-09-07 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [Lilith Cole](https://github.com/LilithCole) | [Lowri Jenkins](https://github.com/LowriJenkins) | [Jack Allen](https://github.com/JackEAllen) | + +--- + +## Items from last retrospective + +_Items discussed in the previous retrospective_ + +### Sprint + +#### Should we delay start of next sprint? - yes and did +- KB suggested sprint planning/start for the coming sprint can be be delayed by a week to finish off large volume of items in ready/in progress +- Agreed upon by all + +#### `PlanitPoker` vs. Scrumpy - which is better? - `planitpoker` stick with for now +Scrumpy Pros: +- Github tickets are more easily tied into the service +- Asynchronous voting - People can vote in advance if they will be away for planning + +Cons: +- You can only be in one room at a time, difficult to do priority / points in parallel. We could use both in a hybrid solution but might create more overhead than utility + +We will stick with `PlanitPoker` for now but keep Scrumpy in mind as an adequate backup if needed. + +## Customer relations + +### Are people interested in uniforms (e.g shirts) for representing IBEX on support calls? - retuning to later +- [Jack Harper](https://github.com/rerpha) feels that users can be confused about our role sometimes when we turn up on support calls +- Other support teams have specific outfits to clearly identify themselves as qualified support staff +- This does not need to be mandatory, but it might be nice to have the option +- Not everyone likes uniforms, can we have other items to identify ourselves? e.g. hats +- [Jack Harper](https://github.com/rerpha) will investigate available ISIS "branding" options and report back at next retrospective + +### Support phone number: - done +- New label with updated contact information has been printed and displayed in TS1 & TS2 blockhouses. Done! + +## Team Interaction + +### Coffee attendance is extremely low, should we find an alternative for team building? - tried coffee - attendance better if someone brings cake +- Attendance has fizzled to near zero - perhaps a symptom of more people being back in the office +- Suggestion to steal Weekly Coffee & Cake idea from Mantid instead. People in the room agreed this sounds like an appropriate rhythm. +- We will do this in a conference room to enable people to join remotely. Should be convenient for conference room availability. +- We have picked Friday afternoon as the regular time for this for now. We can consider varying the days of the week this takes place if this otherwise excludes anyone from attending. + +### Should we have standup in G06 instead of the office?: - seems better than the office G34 and G06 +- Team generally in favour, (more space, better sound). +- Availability may present an issue +- We will trial it when the room is next free in the morning. + + +--- + + +## Items from current retrospective + +### How Would People Feel About Trying Moving Stand-up Next Door to the Conference Room G06? +* G06 camera better placed. +* G34 is just nicer! +* Sound quality much better when the room is designed for it! +* Seeing and hearing people in the office much better. +* As some of those working remotely can be hard to hear, one option could be to get people headsets to be more audible. + +### Uniforms for the Team +To make it clearer to users that we are from the experiment controls team and not just random members of the general public who have wandered in and started unplugging things, uniforms were discussed. + +* [Jack Harper](https://github.com/rerpha) found an online resource for ISIS branded clothing which is used by other groups: +Found a place to order clothes from: http://www.artisanshirts.co.uk/ +The uniform would be optional as not everyone would necessarily like to wear it. + +Tasks left to do on this topic: +* Find out if there is a specific procedure for ordering ISIS branded clothing- maybe ask Jamie. +* Check what they can produce +* If there is a specific piece of clothing that someone would prefer to have, let Kathryn know directly ("I might be convinced to wear a this"). + +### Could we book rooms for an hour for code chats? +The answer: "Yes, just let the organiser know how long you need" + +### Could we make the repo_checks jenkins job statuses go to the #Jenkins teams channel rather than #General? +Possible solution discussed: +* We could try and automate builds and stop checks temporarily. +* [Jack Allen](https://www.github.com/JackEAllen) will place an existing GitHub workflow developed for personal use into a repository to be adapted to work through Jenkins. The main piece of work that will need to be undertaken to do this is setting up an authentication token for the Jenkins user to be able to push to the repository as the GitHub workflow developed relies on a personal access token which is not suitable for use in an organisational repository. + +Actions Taken: +* Wiki has been updated to include workflow: [shared utility scripts](/tools/Shared-utility-scripts) +* A template repository has been created which can be used as it is, or as a starting point for integration into Jenkins CI/CD: https://github.com/ISISComputingGroup/Update_Submodule_Workflow_Action + +### Would it be helpful whilst we are trying something different with planning if I run the next sprint planning meetings as well? +A resounding yes! `:thumbsup:` + +### Coffee Attendance +Try running only on Wednesdays and Fridays interchangeably to be inclusive of those working remotely on one of the two suggested days. + +### Potentially controversial: should we have a timekeeper for stand-up :question: :question: :question: +stand ups have recently been getting longer and longer, we need to keep them short and sweet. Daily stand-up is constructed for people to speak for a short period of time due to getting tired standing up. Those of us working remotely sit down so happy to talk longer. +When anything slightly in more depth needs to be discussed it should be broken into a separate meeting. + +To resolve this: +* Everyone should be more proactive and aware that you can interrupt someone to move on. ("words, then muting people, then violence as a last resort" :punch: ) +* Turn up on time! - We should make a start no later than 10:03am. +* To keep things moving, we should move on from looking through status screens to individual status's by 10:15am no matter what. + +### Checkstyle - How to Try and Reduce How Often We Check It As Part of Stand-up +Suggestions: +Anyone can merge is checkstyle remains the same or decreases - only admins can merge if needed and checkstyle would goes up. +Automate checkstyle in GitHub to become part of the review (can't be missed this way). Up to developer discretion to allow merging in if checkstyle would increase, but generally only allow to be merged in if checkstyle remains the same or decreases. + +The only complexity mentioned with this is we would need to tell Jenkins which branch to compare against master and that master would need pulling to ensure it is up to date. + +Action taken: +[Jack Allen](https://www.github.com/JackEAllen) will create an issue to explore the viability of automating through GitHub and label as tech depth. We will leave checkstyle as it is for now, but be stricter with merging in pull requests and be more concise with updates in stand-up - no harm in saying "can I catch you afterwards" to resolve checkstyle increases. + +Actions Taken: +- Issue to explore viability of automating through GitHub: https://github.com/ISISComputingGroup/IBEX/issues/7346 + +### Currently, It is Hard to Find New Developer Issues + +Currently, it is hard to find appropriate worthwhile issues for new starters and members in the team early in their careers - Can we be more liberal with adding things? - + +From the discussion held, there doesn't seem to be enough appropriate tickets at the moment which is why it may be hard to find new developer issues. + +Older issues are potentially suitable, but not necessarily documented well enough for new starters to pick up. +As people see appropriate tickets, they should update the description if still relevant and add the "new developer" labels to them once checked by Kathryn who is currently going through all tickets. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.05.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.05.md new file mode 100644 index 000000000..e6279b982 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.05.md @@ -0,0 +1,77 @@ +# 2022-10-05 + +| Chair | Timekeeper | Note Taker | +| -------- | --------- | ---------- | +| David | Chris | Lilith | + +--- + +## Items from last retrospective + +_Items discussed in the previous retrospective_ + +### Standup in meeting rooms +Already doing it, happy with it. + +### Code chat bookings +Will do so. + +### Planning chairs +Kathryn continue for this PI. + +### Coffee Attendance +Better weekly, need to avoid maintenance days (whoops). + +### Standup time +Better recently! + +### Checkstyle +Now 0! Make them error instead of warn? Tom will delete from standup links and make build error. + +### New developer issues +Maybe training 1 and training 2 to determine how in-depth tickets will be + +### repo_checks in general +Frequency has changed, but staying in general +--- + + +## Items from current retrospective + +### Wiki/comms + +### Hard to read diagrams +If you use a dark background on github, a lot of images with transparent backgrounds are hard to read. In future, try to upload with solid backgrounds + +### Discussion meetings +Scheduled as-and-when we can, another coming up this sprint + +### Who is working on a support issue? +Is there an easy way to make it clearer who is currently looking at a support issue so that we don't step on each other's toes or neglect an issue. +Shocked face on support thread to show if you're looking into this, smiley when done. + +### Standing up at stand up +Do the people on-site still need to stand up? Currently talking to a series of torsos, danger that if we sit down we'll start taking longer again. Will try this week, see what happens. + +## Process + +### DO we want to be officially limit the number of tickets in `in progress`? Theoretically should be swapping between `in progress` and `impeded` as needed. +Theoretically project board checks should be policing us on this, however our "standard list of warnings" isn't something that we should have. +Impeded could be used as a "parking space" for when you need to take a pause in a ticket for rework hardware testing etc. + +### automate update submodules as a github actions? +Might be just enough times when it wouldn't work that it won't be worth it. Possibly make a manually Jenkins build for it, but seems about as much effort as manual process, so probably leave it + +### Training issues +See above, possibly pointing training too. + +### On site rota +Possibly re-work it, discuss during group meeting + +## misc +###Uniforms +Shelve this discussion, not enough support from team about this + +## Any Other Business +### Support +Lots of support issues this sprint! Assigned 13 points this sprint, not done that since February. Well done! Maybe due to TS1 coming online imminently, network changes, win10 changeover, etc. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.26.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.26.md new file mode 100644 index 000000000..254ab392e --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.10.26.md @@ -0,0 +1,98 @@ +# 2022-10-26 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| [Freddie Akeroyd](https://github.com/LilithCole) | [Jack Harper](https://github.com/rerpha) | [Lowri Jenkins](https://github.com/LowriJenkins) | + +## Items from last retrospective. + - G34 vs G06 - Opinion doesn't seem particularly strong either way. + - Whichever is more convenient? + - Current Booking is until Christmas. + +## Items from this retrospective +- Uniforms + - Conclusion is no + +- Change to onsite rota One less person, do we want to change rota to rebalance it. + - Yes we should before cycle + - Perhaps a discussion for group meeting. + - do we follow the one on teams or the one on the wall. + + +- Good first issues and Training + - Implemented + - Good first issues for basics + - Training for still good training, but more complex than good first issue. + +- Planning and Pre planning + - Remove pre-planning until we start over-running on planning meetings again. + +- Windows 11 builds + - Whats FIT policy? + - Doesn't exist yet? + - Other IT departments starting to move + - Instruments are Win10 we can just say this is what we support + - It would be nice to know if something is fundamentally broken + - we won't be fixing it for a 6months - a year + - Perhaps just test squish and system tests on win11 with a win10 built client + - have to be occasional due to squish license + - squish only takes an hour though, so should be schedulable + - full build node note really needed at the moment, as we're unlikely to have developers or instruments on it. + +- Laptop and Desktop + - weak laptop at home to remote into work desktop? + - easier to not forgot. + - do we need a machine we can take down to hall if we only have desktops on site? + - we do, but need to do an inventory on them/IT equipment in general. + - do we have enough build nodes? + - we also have a little tripod table so you can use it to put laptop on where you are. + - win11 laptop talks to win10 fine over remote desktop + + +- cake with Mantid + - generally happy with the idea? + - only concern is some people on Mantid are entirely remote and might be left out? + - how do we avoid them and us scenario for people working remotely? + - do benefits outweigh this or not? + - Discussion with people from Mantid would be necessary to handle this would be wise. + - potential approach for hybridization? + - breakdown of how many people remote vs local? + - if were meeting with accelerators, do we cycle between them etc? + - periodic "joint coffee"s perhaps + - Do we want something similar with accelerator controls people as well? + - That might be useful since they are epics newcomers + - do they have coffee meetings or the like? + - they might all be on site on Wednesdays? + +- Previously running column + - relatively new, (this or last deploy?) + - could the version and previous version be automatically be on the wiki page? + - irrelevant to whether or not we actually want to display that information on the front page of ibex + - Table is already noisy and cluttered, its potentially useful, but is it useful enough to clutter this table more + - the page itself is noisy and cluttered, not necessarily specific to that table + - The quick summary of how many release notes to check is useful + - should it be on a separate page? + - Instrument details pages? + - Scientists will rarely want this, but if they do are likely to cause trouble if they can't find it. + - instrument pages horrendously out of date and unmaintained + - keeping it to homepage might be useful. + - was it added for us or because scientists wanted it? + - uncertain. + - Add links to instrument names to instrument page and put it there? + - Should we be updating instrument pages? + - Info is still useful just go through it with the mentality of "when was this migrated, how old is this page" + - Return to this discussion, currently undecided. + - perhaps add a link to the deploy wiki page on where to find the previous version in logs after an install if you forgot to check? + +- Merging Ibex.wiki and developers manual? + - project manager vs technical + - how often do scientists search the wiki? + - user manual doesn't know what version is on an instrument, + - needs time spent on it + - helps to be able look in one place for information + - however this is just for us, so if there's reason for it to be split then probably better to leave it as is. + - possible that some information is in the wrong wiki. + - for those who didn't know, wiki on the same repository as release notes. + - this means that github search -> organisation -> wikis will find information on any of these. + - possibility of website pointed at checked out search to git using git grep + - or Docusaurus perhaps? \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2022.11.23.md b/doc/processes/retrospective-notes/Retrospective-Notes-2022.11.23.md new file mode 100644 index 000000000..64e95266c --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2022.11.23.md @@ -0,0 +1,65 @@ +# 2022-11-23 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| Lilith Cole | Freddie Akeroyd | Thomas Lohnert | + +## Items from [last retrospective](Retrospective-Notes-2022.10.26): + +### Windows 11 builds: +> Perhaps just test squish and system tests on win11 with a win10 built client +- FA: correction - we should rather have a win11 built client talking to win 10 build server + +### Laptops and desktops: +- If current laptops die, they are likely to be replaced with a desktop machine and more lightweight laptop for remote desktop access & more mobility (easier to bring to halls) + +### Cake with Mantid: +- Still keen to do this, but delay until the new year. Starting a new meeting series in December is unlikely to capture a lot of attendance + +### previously running column: +- Jack was the person who brought this up, lets have this discussion when he is involved in a separate meeting + + +## Items from this retrospective: + +### How should we deal with support calls asking for help with work that could have been planned work if scientists told us about it? +- Issue: this sometimes means we would have to drop other urgent work to assist with these tasks which affects other customers. Do we need to put our foot down more? +- Things to consider: + - Will not doing this cost beam time? If not, we can justifiably not do it and just write it up as a ticket. + - Will doing this task be a rabbit hole and take up an unreasonable amount of time? We should at least have a go and try to fix it in 1-2 hours rather than ignore the request completely. If the work extends beyond that, we should be justified to say "sorry we don't have enough resource to spend on this right now, you should have told us earlier" + - If you are uncomfortable doing this, consider looping in Kathryn Baker as Project Manager +- Further action: + - Should we bring this up at the next SAG? + - SAG has switched focus so not all of them might be present. We could bring it up with group leaders separately + - Peter: Similar situations from Culham, solution was to go to team leader and let them prioritize the work + - It may be difficult to find a person with the power to make that decision, e.g. if groups are in different divisions + - Can we monitor how often this actually happens? + - Happens infrequently enough but annoying when it does. + + --> **We will keep an eye out for more of these situations but take no further action for now.** + +### Items from Post Cycle Washup Meeting: +> Reflectometry - HTS is working well now - thanks Tom +> +> Reflectometry - Plotting is improved (thanks Tom), but they do want the functionality to do things like zooming back [Editor's note: already implemented at time of meeting] +> +> Reflectometry - The slits logic changes are looking good where they are in use - thanks Freddie + +- Just mentioned for retrospective as it is nice to highlight positive feedback esp. from reflectometry group, who to experience a lot of snags + +### Is "Previously Running" Column on instrument overview useful? +- Posted something on technical to discuss this but people who could answer were not around at the time +- Discuss this offline, just give the post on `technical` a bump + +### Should we push back on overly complicated requirements that spiral in scope? +- Example: RIKEN requesting very specific GUI behaviours that add a lot of complexity +- There is only so much pushing back we can do, otherwise it will discourage migration to IBEX. +- Pushing back should be done at the point of gathering requirements. This can look like: + - Offering a simplified version of the functionality that is being asked for. Try to get to the heart of what problem we are trying to solve + - Ask for precise answers of what is needed now & what will be needed later so we can prioritize more effectively + - Kathryn: If you feel uncomfortable pushing back, put me in the loop - this is my job + +### General encouragement to pick up reflectometry tickets: +- Thomas L: I have ended up giving reflectometry tickets to Nikola since they were suitable and no one had picked them up late into the sprint. This is ok but not ideal from a perspective of passing on / retaining specific knowledge +- People may be busy with other things that demand attention which is absolutely fine +- If you are a bit intimidated by reflectometry and that is a factor, always feel free to talk to Thomas to pair up. These tickets have been selected for being well scoped and not too nasty as an introduction! diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.01.04.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.01.04.md new file mode 100644 index 000000000..d89dd3a88 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.01.04.md @@ -0,0 +1,50 @@ +# 2023-01-04 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| Jack Harper | Hannah Cawley | Nikola Roev | + +## Items from [last retrospective](Retrospective-Notes-2022.11.23): + +### Windows 11 builds: +>> Perhaps just test squish and system tests on win11 with a win10 built client +>- FA: correction - we should rather have a win11 built client talking to win 10 build server +- Contact Facilities IT and ask for their Windows 11 migration plans. After which discussions about this topic can proceed with more relevant information. + +### Should we push back on overly complicated requirements that spiral in scope? +> - Example: RIKEN requesting very specific GUI behaviours that add a lot of complexity +> - There is only so much pushing back we can do, otherwise it will discourage migration to IBEX. +> - Pushing back should be done at the point of gathering requirements. This can look like: +> - Offering a simplified version of the functionality that is being asked for. Try to get to the heart of what problem we are trying to solve +> - Ask for precise answers of what is needed now & what will be needed later so we can prioritize more effectively +> - Kathryn: If you feel uncomfortable pushing back, put me in the loop - this is my job +- Previous talking points were reinforced. General consensus was maybe trying to simplify requirements. + +### Is "Previously Running" Column on instrument overview useful? +> - Posted something on technical to discuss this but people who could answer were not around at the time +> - Discuss this offline, just give the post on `technical` a bump +- An easily accessible web page on a public wiki is convenient for people doing support off-site as well as scientists. +- The information on the page might not always be correct as it needs to be updated manually when an instrument changes versions. +- Having the previous version adds clutter to the `Home` page. +- A proposed solution was adding a link to an auto-generated web page that will hold the version history of each instrument. + +### General encouragement to pick up reflectometry tickets: +> - Thomas L: I have ended up giving reflectometry tickets to Nikola since they were suitable and no one had picked them up late into the sprint. This is ok but not ideal from a perspective of passing on / retaining specific knowledge +> - People may be busy with other things that demand attention which is absolutely fine +> - If you are a bit intimidated by reflectometry and that is a factor, always feel free to talk to Thomas to pair up. These tickets have been selected for being well scoped and not too nasty as an introduction! +- Discuss again when Thomas L. is present. + + +## Items from this retrospective: + +### Is point allocation for support appropriate: +- A little bit on the low side but generally accurate. + +### Migrating to new GitHub projects system and separating issues: +- Separating issues to their own repositories will make them less searchable and easier to be overlooked. +- New system has convenient features and will probably be used in the future. + +### Coffee: +- Regular coffee meetings should continue. +- Organising a meeting with Mantid might be challenging as they have a lot of remote members. +- Accelerator meeting is more doable, should discuss again after follow up for more details. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.02.01.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.02.01.md new file mode 100644 index 000000000..229dcb204 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.02.01.md @@ -0,0 +1,53 @@ +# 2023-02-01 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| JH | NR | DK | + +## Items from [last retrospective](Retrospective-Notes-2023.01.04): + +### +> Is point allocation for support appropriate? +- Best to apply points earlier in the sprint + +### +> Migrating to new GitHub projects system and separating issues +- Decided _not_ to use GitHub Pages for the moment, but will review at later date + +### +> Coffee +- Too tricky to invite MANTID group to coffee (as they work remotely more often), but should be possible with accelerator controls group. **FAA** talking to Ivan about this anyway + + +## Items from this retrospective: + +### Try to automate more of release workflow to avoid errors +- Discuss when ticket is next proposed (at following planning meeting) + +### Shorter sprints to avoid unplanned work? +- Try to be better about adding tickets that anyone can work on +- OK to add urgent tickets any time +- Group would rather _not_ have more context switching with more frequent and shorter meetings +- Suggested splitting tickets down further, but some can't be and so still wouldn't be finished by sprint end +- Decided to keep as is, but bear in mind the option in future + +### CO2 monitor, is it still effective? Perhaps worrying too much about it? +- Agreed that indication useful for air quality if nothing else +- Dilemma between high CO2 levels and room temperature with windows open +- SHE group and/or Estates to be contacted for current policy -> **TL to do this**. + +### Recent AwayDay very worthwhile and productive +- Good to have Hannah & Sonya there for alternative perspectives and input +- Suggestion to discuss long-term plans more often -> **TL to elaborate** +- Hopefully ideas from the day will be implemented, or at least prioritised with rest of work + +### Could be more ambitious with migration schedules +- Feeling that the group could get on with the migrations and finish them. **KVLB** agreed, but reported that the team is too stretched to cope at the moment +- Remaining migrations have most equipment catered for +- Tickets will be created for remaining kit and proposed as training for new starters. Starting soon with PEARL and HIFI. + +### General thought that the group is not currently working on 'large'(er) projects +- TS1 & TS2 coming up soon, so best not to take too much on at the moment + +### Prediction that TS1 coming back online will generate many problems for the group, having not been run for ~18 months +- Lots of equipment has been switched off and/or disconnected during that time, so the group is expecting many calls even if the problem is not actually with the control software itself (but it shows the symptoms) diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.03.08.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.03.08.md new file mode 100644 index 000000000..005ef67db --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.03.08.md @@ -0,0 +1,70 @@ +# 2023-03-08 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| LJ | JH | HC | + +## Items from [last retrospective](Retrospective-Notes-2023.02.01): + +> Try to automate more of release workflow to avoid errors + - Will be discussed in planning meeting (2023/03/09) succeeding this one. + +### +> Shorter sprints to avoid unplanned work? + - No extra discussion concerning this. + +### +> CO2 monitor, is it still effective? Perhaps worrying too much about it? + - Discussed below. + +### +> Recent AwayDay very worthwhile and productive + - Have had some follow up from this: quick wins meeting, and also various discussions. + - Mentions of assigning a day to tackle the various areas in groups: Thomas L volunteered to organise. + +### +> Could be more ambitious with migration schedules + - David and Kathryn on the case creating the new starter tickets. + +### +> General thought that the group is not currently working on 'large'(er) projects + - All agreed that not taking on too much is reasonable, given cycle and TS1 ("lots" of support!) + +### +> Prediction that TS1 coming back online will generate many problems for the group, having not been run for ~18 months + - A lot of support, but surprisingly was mostly TS2 related. + +## Items from this retrospective: + +### Standup: standing in the conference room + - Generally agreed upon that it's only fair enough as remote workers are sitting: we have reached the conclusion that people are welcome to sit/stand as they please + - Mentioned that as there is a culture of standing, unless enforced we will most likely continue standing out of habit. Thus, we will have sit-down standups for the following sprint. + - Evaluate our time efficiency next sprint, and consider the need for a timekeeper (or decide to have stand up standups again). + +### CO2 monitor + - See Thomas L's picture in teams: this is the most up-to-date guidance. We often surpass the 800 mark (e.g., _open windows, reduce occupancy_) in the office, even when it's not particularly full - as we are expecting new starters next month, this is not acceptable. + - The two solutions were seen as either living with the windows open (estate's advice, but it's winter so not particularly viable) or pushing estates for a better ventilation system. + - Freddie to ask estates. + - In the mean time, we will try to leave a fan running: evaluate our satisfaction with this next sprint. + +### Concerns regarding recording hotfixes + - General agreement that it's currently a pain to record hotfixes, and too easy to accidentally skip/miss the hotfix step on the upgrade step. + - Agreed to remove the version numbers from the hotfix page, but migrate these elsewhere as they are useful for support calls. + - Jack H to remove the version numbers from [Instrument Information / Hotfixes](https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information--hotfixes), and add a link to each instrument's wiki page with the version numbers there instead: this will make the information there more concise. + - Tom W to write ticket for investigating technical solutions for this. + +### Database truncation +- All in agreement this is necessary +- There is a [ticket](https://github.com/ISISComputingGroup/IBEX/issues/5818) proposed for this, but it may want updating with some of the technical information mentioned in the meeting. + +### Barriers which prevented [Instrument Demos](https://github.com/ISISComputingGroup/IBEX/issues/7584) from being done + - General feeling that recording yourself is uncomfortable: proposed idea of one person recording, and another editing to overcome this. + - Still very much considered valuable and a good idea. + - Discussed the lack of demo meetings, and how these can be valuable for IBEX input. Decided that it's more sensible to encourage the instrument scientists to come to us, rather than imposing meetings on them. + - Agreed actions: + - Skip demos for this sprint: it's been long enough that we should focus on preparing for the next sprint's instead. + - Each sprint's demos will cover the changes occurred since the last, as then we get a nice archive going forwards. + - Create videos both for each feature in the sprint, and the sprint features as a whole (creative license for the person doing the ticket to decide which way to do this). + - Brought up larger issue of not picking up high priority tickets. Decided this can be rectified by also keeping an eye on the 'Ready' column and the tickets at the top when checking the project board during standup. + - Hannah to note this in standup links file on teams so we remember going forwards. + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.04.12.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.04.12.md new file mode 100644 index 000000000..98ece6191 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.04.12.md @@ -0,0 +1,51 @@ +# 2023-04-12 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| NR | HC | JH | + + +## Items from [last retrospective](Retrospective-Notes-2023.03.08): + +### Standing in the conference room at standup +- people do whatever they want + +### CO2 monitor +- nothing yet, but getting warmer so less of an issue. FA to ask estates - not sure if done yet? + +### Concerns regarding recording hotfixes +Some tickets have been created for this now. + +### DB truncation +there is a ticket proposed for this to create/use mysql instance + +### Barriers for instrument demos +decided to leave them in the end +brought up larger issue of picking up higher priority tickets +- this sprint we have done this more, so just need to keep aware of it + + +## Items from this retrospective: + +### We should set up manual squish test rota +- ticket created to see which ones can _actually_ be automated. ticket has been proposed. +- we should create tickets each time we do this to track progress and be transparent. + + +### Flash reviews hanging round +- perhaps we should bring them up at standup on Mondays to get them through +- should add this on to the Monday reviews (project board etc.) +- add to list of things to check at standup on a Monday and assign people +- should we use a different way of telling about flash reviews? have to be careful about making it too heavyweight. +- checking at standup requires little extra effort, we should do this. +ACTION add this to standup list + +### Are we getting a benefit from discussing points? +- we should properly revisit this time when KVLB is here. + +### What information needs to be shared so that others can lead planning? +- not 100% confident in leading the PI boards - it has been explained but still not sure. +- do we need to not look at the PI boards - they are project planning but perhaps we need to just know how many points we have to play with +- discuss this again when KVLB is here. + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.05.10.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.05.10.md new file mode 100644 index 000000000..402efe4a5 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.05.10.md @@ -0,0 +1,39 @@ +# 2023-05-10 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| JH | DK | TL | + +## Items from last retrospective: + +### Standup: +- Everyone can stand or sit as they please. + +### CO2 monitor +- We will observe policy, nothing else to add for now. Likely to become less of an issue now that it is warmer and we can open windows more easily + +### Recording hotfixes +- Ticket to improve process is in ready. +- Version numbers have been removed from IBEX overview table to make it more readable. Added link to different page displaying live versions instead. + +### Database truncation +- Nothing to add + +### High priority tickets +- During standup, we have been pointing out high priority that have remained at the top of ready for a long time. This is good and we should keep doing it. + +## Items from this retrospective: + +### People other than KB leading sprint planning +- People are comfortable enough leading the sprint planning meeting, except for the program increment bit. +- People are also unsure about all the other pre- and post-admin surrounding the actual meeting itself. +- DK volunteered to check we have a comprehensive list of necessary steps written down on the wiki (or elsewhere that's accessible) + - We have [sprint planning](/processes/meetings/Sprint-Planning) - check this is up to date + +### Other comments (Mad/Glad/Sad) +- Glad: Support has been going well, feels like we have been good at addressing issues. However, we are also aware that TS1 has not come online yet, will likely make support more challenging +- Glad: We managed to condense a 4hr retrospective meeting into 30 minutes +- Question to new starters whether they have any feedback on onboarding process? + - Evan: It's going well, I have a good general idea of what's going one + - Any issues we had came from wider STFC/UKRI, not this team (e.g. account login issues) + - We should try to make more use of any opportunities to take the new starters to instrument halls \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.06.07.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.06.07.md new file mode 100644 index 000000000..357ba7109 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.06.07.md @@ -0,0 +1,17 @@ +# 2023-06-07 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| HC | LC | NR | + +## Items from last retrospective: + +### People other than KB leading sprint planning +- Meeting instructions will be updated. +- People are interested in trying to run the meeting. + +## Items from this retrospective: + +### Other Comments +- Sprint progressed better than expected. +- Check if on-site rota needs to be balanced better. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.07.12.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.07.12.md new file mode 100644 index 000000000..60181a91d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.07.12.md @@ -0,0 +1,36 @@ +# 2023-07-12 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| TL | NR | HC | + +## Items from last retrospective: + +### +> Check if on-site rota needs to be balanced better. +- Some shifts made so Wednesday is has better coverage, but this has been done and all seems better distributed. + +## Items from this retrospective: + +### Leading Planning: what information needs to be shared/what do we think about sharing the responsibility +- LJ ran the last planning meeting, and LC is scheduled to run the next. All went well last meeting, so agreed it would make sense to start a rota so we can assign responsibility as we do with all other meetings. + +### Outdoor Coffee™ +- HC and LJ had a trial run and thought it was lovely! +- General consensus of hybrid coffee doesn't work well. Thoughts: + - We currently have staggered on-site Coffees so that all with different onsite days get to attend. However, this was multiple rota shifts ago, so maybe this needs a rethink. So, maybe we only host in person Coffee, as online can be a bit intimidating to join. + - Maybe we consider hosting a separate optional online coffee - could be a chat, could be a zoom call. + +### Mechanism to propose tickets for future sprints, not necessarily the next one + - KB mentioned PI board is exactly for this, and people are welcome to utilise it for this purpose. However, we should consult KB before adding tickets on so it doesn't get horrifyingly cluttered. + - This would also be appreciated as it means not all the responsibility falls on to one person! + - Agreed solution (along with any notes in the ticket to detail the expectations): + - For [PI_2023_02](https://github.com/orgs/ISISComputingGroup/projects/4), tickets can be added to the PI board, in the ['Next PI' column](https://github.com/orgs/ISISComputingGroup/projects/4/views/5) + - For later PIs, tickets can be added to the correct sprint column in the 'Breakdown by Sprint' view + +### Other comments (Mad/Glad/Sad) +- Sad: Have had a very busy sprint with onboarding and support, so highest priority tickets haven't been completed. + - TS1 having lots of problems with odd changes we haven't been informed about, and bits of equipment that haven't been moved in ages. We'll have a few more instruments showing up in September so can expect similar support issues then. +- Sad: H & N leaving :( +- Mad: FMR - "if FMR comes back it will be drop kicked off the top of R80" and JH will cry + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.08.09.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.08.09.md new file mode 100644 index 000000000..393a9e823 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.08.09.md @@ -0,0 +1,54 @@ +# 2023-08-09 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| FA | LC | ES | + +## Items from last retrospective: + +### Wednesday was missing some coverage +- We have shifted the rota around a bit to balance experienced people with new starters. Cycle starts end of September, Lowri will be back in September, this should help with coverage. + +### Mechanism to propose tickets for future sprints, not necessarily the next one + - KB mentioned PI board is exactly for this, and people are welcome to utilise it for this purpose. However, we should consult KB before adding tickets on so it doesn't get horrifyingly cluttered. + - This would also be appreciated as it means not all the responsibility falls on to one person! + - Agreed solution (along with any notes in the ticket to detail the expectations): + - For [PI_2023_02](https://github.com/orgs/ISISComputingGroup/projects/4), tickets can be added to the PI board, in the ['Next PI' column](https://github.com/orgs/ISISComputingGroup/projects/4/views/5) + - For later PIs, tickets can be added to the correct sprint column in the 'Breakdown by Sprint' view + +## Items from this retrospective: + + +### Burndown Chart skewed due to BAU tasks, IBEX vs. non-IBEX tickets +- It all works if amount of Project and BAU added stays consistent; if not, the point average will be going up and down. +Based on IBEX+; It makes the most sense to put that all on the board. If you have an idea of how much work there is and what you’re doing you can better estimate. +- The main idea of the burndown chart is to help estimating and current sprint work. +- What is exactly included in “+” part of “IBEX+”? We might not be pointing stuff which is not IBEX and also is not “+”, but should that count on the burndown chart. General scope definition of "IBEX+" explicitly might be necessary +- IBEX work could be more than the “50%” discussed in previous planning, if it’s work done that is not part of the migration project/process + +### Start moving New grads into support stuff? +- Usually this is done after a year and a bit after, but we are somewhat short staffed, so getting an idea for the process generally; i.e. going down to the halls, etc. with those who are on support for ES and DM might be good to start soon. + +### IBEX training course +- For New Starters vs. for New Scientists/Users +- Learning things such as, "How do I add a block in IBEX?" and other stuff would help out when doing support. +- Might be a good idea to run a training course for us, would not need to set up external machines. Are we running a “proper” IBEX training course soon? - - Since we are migrating new instruments, e.g. Muon instruments are migrating and such. There might be new scientists who have started on new instruments that would benefit from this. +- Need to send out an email to check interest on external training. Need NDXDEMO/NDXTEST if doing for Scientists/Users +- We can make NDXTEST relatively easily for this, and we have the space and capacity to do this. +- Training course sounds like a good idea; if we just run it for developers we can just do it in G.06 on our own machines, once we have all the new starters, the week of the 20th of August. After that we can sort out if we’re doing it for scientists as well + + +### Other comments (Mad/Glad/Sad) +#### SAD: +- TL: Beckhoff issue generally, Motion Group loss of staff, temporarily losing Jack to them +- JH: INTER might have delays due to safety issues, probably starting off just their own experiments without any users. Not sure how they’ll deal with the safety issue. They pressed a “stop” yesterday which caused a large heavy thing to drop, which had hazard potential if someone had been standing underneath it. Various safety concerns with loss of power, etc. This will probably make INTER stuff run a bit slower. +SURF is most likely going back to Galils as well, as they don’t have anyone to do Beckhoff + +#### MAD: + +#### GLAD: +- FA: We got through a lot of useful tickets, and a lot of tickets generally, and we have good new people in the team. +- LC: Happy the last big block of MUSR is done and on schedule. +- ES: Seeing the cryostat/ needle valve work was cool +- FA: Hasn't been too warm which is nice in the office, we aren't overheating. + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.09.08.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.09.08.md new file mode 100644 index 000000000..dcfaa251f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.09.08.md @@ -0,0 +1,54 @@ +# 2023-09-08 + +| Chair | Timekeeper | Note Taker | +| :-------- | :---------: | ----------: | +| ES | KB | DK | + +## Items from last retrospective: + +### Burndown chart +- Discussed elsewhere (as item in current sprint below) + +### Supporting instruments (new starters) +- Keep taking them to the experiment halls to gain experience + +### IBEX Training course +- Good idea for all new starters in group, as well as new scientists/support technicians +- Run two courses? e.g. one for ExptCtrls team members and one for scientists + +*** + +## Items from this retrospective: + +### Burndown chart reflecting BAU (now using actual work ratios: IBEX to BAU). +- agree to keep with this as seems to reflect previous sprint productivity reasonably accurately + +### Delegating other IBEX project management tasks +- e.g. SE devices project representative and manage group's resulting work +- Action for group to think about which tasks they'd like to take responsibility for. Can discuss at next retrospective +- Need place to put list of tasks so that people can take them on (e.g. Teams form, tick boxes on ticket/wiki, Outlook task list on ExptCtrls calendar) + +### ES asked about reinstating the office screen to display build status etc. +- Need to ask CMS about previous control machine or a replacement + +### Winter meal out +- Discussed and that someone (usually a new graduate) should volunteer soon. + +### A group activity of some kind would be beneficial and enjoyable. +- ES suggested a visit to CCFE for a tour and then invite them to ISIS in return. + + +*** + +### Other comments (Mad/Glad/Sad) +#### SAD: +- KVLB sad JH not in team at moment, but glad he'll be back soon. + +#### MAD: +- ZK mad that bug left in TPG IOC after release. + +#### GLAD: +- KVLB v glad release, deploys, testing, etc went well and completed on time. +- Everyone glad team is back to full strength, in terms of numbers anyway. +- WS & IH very glad of help with getting started in team and project. +- ES glad he now has ~75% knowledge to be able to answer questions from other new(er) starters diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.10.04.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.10.04.md new file mode 100644 index 000000000..55d1b2c1f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.10.04.md @@ -0,0 +1,35 @@ +# 2023-10-04 + +## Items from this retrospective: + +### Attempting return to scrum poker online due to continued issues with planitpoker + +### Squish test rota + - Attempt to slowly increase squish test coverage of manual system tests + - The developer who is on standup that week + - Not everyone can do this. + - That's fine, only if developer is capable + - Week on standup, add one squish test to replace a manual system test. + + +### Another Away Day + - Last away day was organized from higher up + - What topic would be covered + - Long-term plans for team? + - Review of topics covered at last away day? + +### Central Archiving + - SQL server hosted by SCD or anyone else + - Do we have time to focus on this? + - How much time do we lose truncating databases? + - not necessarily related to this, still likely need to truncated could implement auto truncation when its also in central location + - this would be a good thing to do, but possibly a long-term issue + - possibly use Archiver appliance? + - Out of hours support from SCD issues + +### Private documentation +- If we make documentation private we can put in non public information in, possibly sharepoint because it's already GDPR compliant +- need someone to take time to look into it. + - not the highest priority +- More document fragmentation will make information even easier to miss. + - link to the sharepoint pages from the wiki, that way still single point of entry. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.01.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.01.md new file mode 100644 index 000000000..d8d231a7a --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.01.md @@ -0,0 +1,48 @@ +# 2023-11-01 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| LC | DM | TL | + +## Items from last retrospective: +- **IBEX training course:** FA did an EPICS course for new starters, but not one for IBEX from a user perspective. We may have effort available to do this in the next PI. +- **Project management delegation:** Tasks will start being allocated as appropriate to distribute the project management workload (unless people volunteer) +- **Office screen reinstatement:** + - We have not found a new machine for this yet. + - We could use one of the old unused datastreaming machines + - New machine does not even necessarily need windows, just a very lightweight OS capable of running a web browser + - JH has offered to use a Friday afternoon to set a machine up. +- **Winter Meal Out:** DM and WS to start looking into options the day after retrospective. +- **CCFE Tour:** Has not yet progressed beyond the idea stage. + +*** + +## Items from this retrospective: +- **(GLAD) New features:** Just relaying some positive feedback from instrument scientists, particularly regarding new Alarms functionality and Moxa Mappings perspective. +- **Wiki restructuring:** ES has progressed this work as part of the recent "Friday", but still ongoing +- **Central MySQL service:** JH has talked to SCD about implementing a central Mysql service. We need a proper discussion on how exactly this should be implemented but we have a contact in SCD now and we know that it is possible. +- **Should we have Standup every day?:** + - Motivation for retrospective comment was that it was perceived to be getting in the way of things like support during cycle while not adding that much (e.g. "I'm going to continue working on my ticket") + - There was a sentiment that by not having it every day we would lose more than we gain + - Checking on services regularly is important + - We have decided to keep it but should make more effort to keep it brief: + - Chair to prepare more by looking at checks and services ahead of the meeting / fixing minor issues as appropriate. There may be a bit of a transitional period as we collectively build the habit, others may remind the chair of this process for the time being + - Be more strict in mentioning what you achieved the day before in individual updates +- **PI board for planning**: We have decided to try running the next planning meeting off of the PI board instead of the IBEX Project Board +- **Merchandise:** There is a strong desire among new graduates and industrial placement students to get ISIS merchandise. + - We do not want to make wearing this mandatory + - Raise with FA for follow up. JH has the contact of the supplier and will start to collect an order from the team + +*** + +### Other comments (Mad/Glad/Sad) +- **Flat Github structure:** We don't have a good way of grouping tickets hierarchically. This would make it easier to do IOCs in parallel if its split up into sub-tickets + - We have tried this before but found that IOCs are not usually "big" enough to warrant lots of sub-tickets, and it can make development more awkward in places where developing the whole thing in parallel is more convenient + - A more structured approach to categorizing tickets hierarchically might be useful e.g. for larger features like Client, Script Generator, etc. +- **Documentation:** Is lacking in places + - It's up to everyone to keep this up to date. If you notice something missing, feel free to add it + - Sometimes the person looking for the information does not have the required knowledge to add the information, but they can ask for help with writing it, or try to write it after they have acquired the knowledge e.g by asking someone, and then getting them to validate it afterwards + - Generally, making more frequent use of diagrams in the documentation may make information much more accessible than a wall of text. e.g. A diagram showing the flow of information / macros / db loads etc. in an IOC would be very helpful. + +#### GLAD: +- KB happy with everyone pulling together to get PEARL running \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.30.md b/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.30.md new file mode 100644 index 000000000..a94cd0164 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2023.11.30.md @@ -0,0 +1,36 @@ +# 2023-11-30 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| DM | ?? | DK | + +## Items from last retrospective: + +- Wiki restructuring: catch up with ES when returned. +- Central MySQL service: SCD can host, but need requirements first. E.g. Possibly 20 instruments x 20 Gb x 5 cycles per year -> ~2Tb per year. Motor positions particularly useful to log for reinstating. Suggest separate meeting to discuss. +- Should we have Standup every day?: Been trying 30 minutes limit. Feeling is that not much quicker, but an improvement. Suggest host checks errors before meeting to avoid digging through links and logs. LC felt hosting went more smoothly when checked pages beforehand. Depends on number of errors. +- PI board for planning: Will use for planning meeting on 11/01/2024. Report back next sprint retrospective. +- Merchandise: Regarding other groups' policy on keeping jumpers, nothing back from MN yet as been on leave. No problem ordering jumpers as company in science warehouse. DK will order with JH help so that FA can approve. +- Flat GitHub structure: Add pull requests in tickets to development workflow to avoid automatic closing when merging a single PR. Need some investigation of GitHub behaviour as still seems to do this. +- Documentation: Still lacking in some areas, but external users say IBEX wiki very useful and extensive. + +*** + +## Items from this retrospective: + +- Weekly coffee rolling rota: LC has created calendar invitations. +- Migrate new project board style: Now checks are done via application rather than automation(?). May create problems for existing checks if converted. ZS will try on current PI board and report back. +- Regular Code reviews: Each Monday Standup meeting been extended to 1hr to accommodate code review. First one this week which went well. Thanks to JH for stepping in. +- JH mentioned EPICS meeting and asked how many from group would go. Includes training sessions, so good for new-starters. April 15th-18th 2024, week before a cycle. Another EPICS meeting in October and NOBUGS in September. +- KVLB highlighted next sprint is only three weeks to avoid cycle clash. + +*** + +### Other comments (Mad/Glad/Sad) + +- FAA sad that many tickets still under review after Christmas break – anomaly because of long break and people forgot which tickets they were reviewing? Suggest look at first five ProjectBoard warnings each week and ask about reviews status. Can remove and re-add 'under review' labels to reset check. +- KVLB sad about lack of responses from ARGUS instrument scientists. +- All glad that Christmas meal went well – thanks to WS and DM. +- FAA glad most tickets complete and reviewed before end of sprint, especially those for release. +- LC glad system tests green before building release next week. +- LJ glad that gap closed between tickets and reviews on burndown chart. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.08.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.08.md new file mode 100644 index 000000000..86d81ba36 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.08.md @@ -0,0 +1,26 @@ +# 2024-01-08 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| CMS | ES | IG | + +Present: +- In person: IG GR JD TW +- Online: JH KB ES CMS SC DK FA LJ + +## Previous sprint + +### No notes of significance + +## Current Sprint + +### Calendar now shows First Line Support allocations + +### The Christmas meal was excellent and well organised + +### PAT testing and SW Release +CMS raised the forthcoming scheduled PAT testing. We should consider synchronising the next software release activities around it and also how best to restore systems to their states prior to PAT testing equipment shutdown. + +## Mad/Glad/Sad +- LJ Glad - Had a good break over the Christmas holidays. +- KB Glad - People managed to extend the Christmas break somewhat. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.31.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.31.md new file mode 100644 index 000000000..791c64f7f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.01.31.md @@ -0,0 +1,33 @@ +# 2024-01-31 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| WS | ZK | DM | + +## Items from last retrospective: + +The items from the last retrospective were recapped, no additional comments were made. + +- Weekly coffee rolling rota: LC has created calendar invitations. +- Migrate new project board style: Now checks are done via application rather than automation(?). May create problems for existing checks if converted. ZS will try on current PI board and report back. +- Regular Code reviews: Each Monday Standup meeting been extended to 1hr to accommodate code review. First one this week which went well. Thanks to JH for stepping in. +- JH mentioned EPICS meeting and asked how many from group would go. Includes training sessions, so good for new-starters. April 15th-18th 2024, week before a cycle. Another EPICS meeting in October and NOBUGS in September. +- KVLB highlighted next sprint is only three weeks to avoid cycle clash. + +*** + +## Items from this retrospective: + +- Manual testing: The manual test process seems to be an improvement now that they’ve been transferred. JH suggested decoupling coupled tests and replace that by adding what the previous test was, or perhaps link them. +- Deploy Script: Discussions regarding the amount of user input required, and simplifying the deploy script. The idea of two deploy scripts or automating the user responses was discussed to retain the ability to alter deployment steps (i.e. in case of rerunning tests). Further discussion requires for this item. +- Code review: It was suggested that code review should be moved to a different day – KB will attempt to do so, if available. +- Modular Instrument Control Virtual Machines: Code-Chat on the topic to be organised by LJ. +- Korea EPICS Meeting: There would need to be justification to send more than two members of staff if there was no training at the event. The event program appears to be incomplete as of this discussion. JH, ES and DM appeared interested in attending. Item was still unresolved, it was suggested that a separate meeting should be set up to discuss signing up to the event. +*** + +### Other comments (Mad/Glad/Sad) + +- ZS is Sad that WS is leaving. +- ES is Glad the release went well, and works. +- JH is Glad Excel is no longer being used for manual system testing. +- KB mentions the next sprint duration is 5 week diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.03.06.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.03.06.md new file mode 100644 index 000000000..d12f2a6a5 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.03.06.md @@ -0,0 +1,38 @@ +# 2024-03-06 + +```{note} +This meeting is unfinished and to be continued. +``` + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| IH | DK | ZK | + +## Items from last retrospective: + +- Manual testing: Use GitHub project board now. +- Deploy Script: Automated unit tests are needed to make sure. +FA recommends maybe using a config file (predefined answers for steps), further discussion needs to take place to decide how we want to move forward with this. +- Code review: Happened and was successful. +- Modular Instrument Control Virtual Machines: Another code chat this Friday is coming up but this will be the topic of discussion for next one. +- Korea EPICS Meeting: There is a lengthier discussion in our teams channel. JH is the only one that the team and management can justify to send. The reasons include that training does not appear to be general/as useful as anticipated. Another reason is the limited budget. + +*** + +## Items from this retrospective: + +- KB: Item carried over (to the next sprint planning) were left in same order: everyone seems to be fine with that, saves a bit of time. +- ZK: move code review extension to Thursday, KB acted on it and works well +- JH recommends putting screenshots in the release notes. Ideas: + - Put more context and pictures into ticket header + - Place image links in the Release Notes (not images themselves) + Specific implementation is not clear yet. +- + +*** + +### Other comments (Mad/Glad/Sad) + +- + +> Date: 06/03/2024 diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.04.03.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.04.03.md new file mode 100644 index 000000000..d7a71e757 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.04.03.md @@ -0,0 +1,76 @@ +# 2024-04-03 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| SC | ES | LC | + +## Items from last retrospective +- items from last sprint left in same order- all good +- code reviews moved to Thursday - been finding it good +- More context and pictures in ticket headers, screenshots in release notes + - packing release notes full of images felt impractical, put links instead + - image added to ticket header by developer before putting it into review + - image will be on the branch you're working on + +## Items from the current retrospective + +### Looking at ISIS beam status page at standup in cycle +- a lot of time beam is off without us realising +- will give us context for the day with regards to possible scheduling +- could make just a PV on our dashboard, if you want more news look at the teams channel +- push back about adding another check to morning standup instead of keeping an eye on teams + +### Ticket context +- Want bad examples of tickets, action on ES for this +- Will bring this up again next retro +- Should we have a longer planning meeting to properly go over what a ticket would entail, and make sure it is written properly with correct acceptance criteria? + - Will run into problem where "the person whose domain this ticket belongs to" is absent + - This was something we used to do a bit of in pre-planning + - Is it worth recording planning meetings so we can reference the discussion later (timekeeper) + - Possibly have a note-taker for planning, will try it in next planning + +### Ordering the priority columns in planning +- People seem to pick up the tickets that they feel able to do/interested in +- Position in ready does (or at least should) influence order tickets are taken +- Feeling that it may not matter too much with 6-monthly releases, however patches change this +- Prioritising adds more time to talking about tickets, however planning meetings are already very short +- Sometimes people skip tickets because it's in an area they aren't familiar enough with, or is not clear what needs to be done +- Enforcing the order will push people into areas they're not comfortable, which would be good in the long term + - We're allowed to make mistakes, you don't have to succeed from the outset (especially if it isn't urgent) + +### Splitting PM duties +- KB to split duties with GR + +### Make sure flash reviews don;t get missed +- track on github, flash reviews column on task board +- checking the teams channel on Monday is good, but there's a chance tickets get buried + +### Finding out tickets aren't needed +- this has happened in the past, we try our best, sometimes we are given incorrect information +- When picking up a ticket, verify with scientists what exactly they want for it (add this to the dev wiki) + - Alternatively contact relevant sample environment team + +### standup length +- keep it to fewer points +- keep engaged during checking of tests/instrument health + +### github actions +- might be useful instead of jenkins builds +- spin up a windows environment for us +- started looking into a few, can be difficult +- where it's easy, try it, run it in parallel with existing builds, turn of the jenkins versions when no longer needed +- wiki check might be easy enough to move + - Friday ticket to do this + - KB already trying to move project board checks +- not sure if we can have project level actions + - actions are repo level, org level we'd need an app, however we can make our own (KB already trying this) + +### Mad sad glad +- LJ Friday went well, had both meetings, interesting tickets, completed in time +- SC glad he tried programming elsewhere in EPICS + - Some OPI standards were difficult to find, in fact most hidden in checker script + - we don't have a standards page, but we do have an example template OPI we should use + - perhaps make a Friday ticket to make the checker script more clearer +- ZK glad about description of the ticket that didn't end up being needed +- ZK glad SC bringing new tools into the meeting + - happy to try it, but many of us unsure about its used diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.01.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.01.md new file mode 100644 index 000000000..ec20f66b4 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.01.md @@ -0,0 +1,34 @@ +# 2024-05-01 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| CMS | | SC | + +## Items from last retrospective +- More context and pictures in ticket headers, screenshots in release notes + - ES pointed to two things: +1. In certain cases the ticket is otherwise well written but the requirement has changed. +2. [6053](https://github.com/ISISComputingGroup/IBEX/issues/6053) is a good example of a well written ticket. + +- Flash reviews as task + - LJ ==> Important that these aren't ignored and picked up with due urgency. On the task board if there are no updates we generally overlook the board. However with flash reviews on the board, no update could mean important reviews haven't been taken up. We need to be careful when checking the task board. + +## Items from the current retrospective +- Going to carry on not prioritising the columns in planning +- Flash reviews are on task boards. If the board isn't updated, we still need to check the flash review items. +- Moving to online meeting also from office + - SC ==> Offline meetings restrict us from using online tools. They also mean that meeting rooms are booked - sometimes much bigger than the number of members present. + - KB ==> Issues with hearing and not comfortable with multiple people talking even with noise cancellation headphones. + - LJ ==> Issues with echo and delay in delivery. + - We're going to keep having meetings as hybrid, as multiple people in a zoom call in the same room is limiting for some of the team +- Discussions on Burndown chart + - KB explained that the burndown chart isn't typical burndown chart, but customized for ISIS. + - LC suggested to invert the Y axis of the graph. +- Discussions around rotating logs off + - CMS ==> When is the file closed and available for moving/truncating/copying - also applies particularly to console logs? When to truncate the DB? + +## Mad sad glad +- LC glad that user training went well, interesting discussions. +- DK glad Chronus is working well despite some niggles. + - glad additional people on the support call is helpful + - sad there is an increase in support calls. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.22.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.22.md new file mode 100644 index 000000000..d3de4ca6e --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.05.22.md @@ -0,0 +1,87 @@ +# 2024-05-22 + +## Power Point slides + +1. **Lowri**: + + - Worked on a Friday ticket, adding component/configuration details. + - Dealt with LabVIEW remote VI for easier communication. +2. **Sudeepta**: + + - Attempted a Java 21 build for CSS. + - Developed a custom emulator for Lindy PDU changes. + - Worked on JMC-based profiling of instruments. +3. **Evan**: + + - Set up EPICS build pipeline on Windows 11. + - Dockerized an IOC for a Friday ticket. + - Updated EPICS dependencies. +4. **Freddie**: + + - Added PVAccess client library to Genie Python. + - Reviewed tasks. +5. **George**: + + - Updated Jenkins build checks. + - Managed storage for non-science data on PCs. +6. **David**: + + - Investigated Moxa RS485 settings. + - Searched for an amplifier for Sudeepta’s project. + - Handled Attocube issue and other tasks. +7. **Jack**: + + - Discussed pearl camera differences. + - Demonstrated Kepco IOC with PVAccess. + - Highlighted PVAccess benefits. +8. **Daniel:** + + - Worked on IBEX Windows 11 build. + - Reworked IBEX buttons. + - Dockerized the IOC. + - Organized demos and reviewed system tests. +9. **Isaac:** + + 1. McLennan Power cycle ticket + 2. Helped refactor IbexButtonBuilder + 3. Created several tickets for alarms based on a James Lord email + 4. Web Dashboard work + 5. IOC emulators kill processes more efficiently on ctrl+c tickets +10. **Zsolt:** + + 1. Updated PyLint + 2. Helped refactor IbexButtonBuilder + 3. Significant improvements/work done to Device Generator +11. **Ian:** + + 1. Onboarding work + +## Old Retro + +Keep flash reviews on the tasks. Leave a comment on the note saying picked up for review. + +Keep having hybrid meetings + +IP students don't have work phones so don't insist we use our phone all the time so not compulsory + +Better labelling for burndown chart + +Draw attention to impeded through a new mechanism. + +## Current Retro + +EPICS top PR Lowri says that there should be a better way to manage the submodules. Jack offered solution and Lowri updated the wiki page to reflect this. + +If discussion ticket and clear acceptance criteria for outcome of discussion whoever assigned to to create the discussion just does it. If the implementation ends up going beyond the points then make another ticket for the rest of the implementation. + +## Mad/Glad/Sad + +*Sad:* +Short but productive sprint +Some of us missed the Aurora + +*Mad:* +EPICS base has hundreds of merge conflicts + +*Glad* +Ian has joined \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.06.19.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.06.19.md new file mode 100644 index 000000000..2934806fb --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.06.19.md @@ -0,0 +1,50 @@ +# 2024-06-19 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| FA | IH | ZK | + +## Items from last retrospective + +- EPICS top PR Lowri says that there should be a better way to manage the submodules. Jack offered solution and Lowri updated the wiki page to reflect this. + + Page has been updated and device generator matches it + +- If discussion ticket and clear acceptance criteria for outcome of discussion whoever assigned to to create the discussion just does it. If the implementation ends up going beyond the points then make another ticket for the rest of the implementation. + + Create ticket if necessary only. + +## Items from the current retrospective + +- Put PyCharm on NDX-s: + - Scientists want this instead of Notepad++ + - We want to discourage use of NDX (client-server based approach) + - Don't want scripts to be run from PyCharm + - They want: autocomplete and syntax highlight, we want: them not to be able to run scripts from it. + - Bottom line: we need more context and requirements formalisation (Jack and Lilith might have a good idea on their requirements based on their recent interactions with scientists) + +- extra role for planning meetings + - notetaker will take on this responsibility + +- Ticket format: + - suggestion of reworking the ticket template with some better guidelines on what to include at certain sections (Testing is the one mentioned) + - reviewer should use what's on the ticket as a starting point and not solely rely on description + +- 15 minute rule in standup: + - fill out first 15 minutes with operations stuff + - we most of the time don't wait if we know people won't make it + - seems like it's going to stay as it is + - come up with alternative solutions and discuss next sprint + +- Joining meeting from the office through your PC: + - Some say it went well, some say it was difficult to hear conversation. + - result: ok for one-offs but do not intend to make it general + +- Certain number of people answer some Jenkins and Nagios checks and results + - could have a code chat on how to interpret status + +## Mad, Sad, Glad + +KB: Glad that Tom is back on the team +FA: Glad that the rack is in place and office looks neater thanks to Jack and others in the office helping out, Also glad many tickets went through and migration is going well. + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.07.17.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.07.17.md new file mode 100644 index 000000000..2214e69b9 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.07.17.md @@ -0,0 +1,55 @@ +# 2024-07-17 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| LC | CMS | DK | + +## Present +- in person: LC, LJ, IG, FA, TW, JD, GR +- online: SC, DK, CMS, JH, KB, DM + + +## Previous Sprint 23/05/2024 + +- PyCharm on NDX control machines - could potentially run scripts via the IDE causing confusion; would produce an extra load on NDX machine -> discourage use if/when possible + +- Extra role for planning (not sure what - timing? editing tickets?) - possibly too much for notetaker to do -> agreed can try at next planning meeting then reassess. + +- Ticket formatting/template - nothing done towards this to date -> LJ will create a ticket to rewrite the template + +- 15 minute Stand-up split -> revisit next retrospective as not enough information this time. + +- Join meeting from office PC -> OK for time being for one-off meetings when room not available. + +- Interpreting Nagios -> LJ will organise a codechat + + +## Current Sprint 20/06/2024 + +- Python standards -> JH suggested a code chat or code review to discuss the changes. LJ agreed and reminded the group that we have standards already, now we'll enforce them + +- Assign ourselves as reviewer of a PR -> all agreed good idea. + +- Have a `Pending` column in tasks board rather than two columns -> KB created mockups of potential new task board (link in `Dev-Changes-Memo` Teams channel). `View 4` agreed as best option. Discussion of labels, sorting of columns, etc. KB will write up some instructions on how to use the system, and to link them from said Teams channel. + +- Fleeces for new starters - T-shirts and polo shirts also requested -> JH will get some quotes + +- Should `email-exp-controls` Teams channel contribute to support issues effort calculations? -> all agreed to only discuss issues in this channel in 'support-issues' for accounting purposes. 'Read Only' access applied to channel? + +- New web dashboard - JH would like to further develop -> TW suggested a code-chat/review of what's been done already before deploying. Agreed would be done on 25/07/2024. UPDATE: JH gave talk & demonstration on morning of 01/08/2024. + +- NOT looking at project board in Stand-Up meeting (as new checks cover most issues) -> agreed to check weekly during Friday's meeting as a trial. + +- Adding new sub-modules to EPICS Top repository -> agreed to create and add empty submodule (to master branch) at beginning of ticket, reviewer will then populate during final merge + +- Concern over large number of 'in progress' tickets -> suggestion to use 'impeded' label more freely rather than just for external reasons. + +- 15 minute wait during Stand-Up - awkward with gap in middle while those present wait for others who've said they'll be late. Sometimes gap filled with looking further at system issues -> LC suggested to start 10:15 and do personal updates first, then look at system status. LC has checked meeting room availability. Agreed we'll try this for two sprints and review next but one retrospective. **Highlight this for reviewer of said sprint** + + +## Mad/Glad/Sad + +- KB was glad we finished the sprint with no tickets in the `Ready` column. Suggested explanation that we underestimated the number of tickets we were capable of completing. +- FA glad Beckhoff issues fixed, JH all of the above for the same reason! +- DK and LC glad Open Day demonstration went well and thanked everyone involved. +- DK and others glad JD has made very good start and fitted in to Team well. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.08.07.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.08.07.md new file mode 100644 index 000000000..faf336d8f --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.08.07.md @@ -0,0 +1,48 @@ +# 2024-08-07 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| DM | DK | IG | + +## Present +- in person: DK, JD, TW, IG +- online: LC, LJ, DM, SC, CMS, JH + + +## Previous Sprint + +- Python standards -> JH suggested a code chat or code review to discuss the changes. LJ agreed and reminded the group that we have standards already, now we'll enforce them. LJ to discuss at the forthcoming sprint planning meeting. + +- Assign ourselves as reviewer of a PR -> all agreed good idea. + +- Have a `Pending` column in tasks board rather than two columns -> KB created mockups of potential new task board (link in `Dev-Changes-Memo` Teams channel). `View 4` agreed as best option. Discussion of labels, sorting of columns, etc. KB will write up some instructions on how to use the system, and to link them from said Teams channel. -> LJ commented that it's an improvement. + +- Fleeces for new starters - T-shirts and polo shirts also requested -> JH will get some quotes -> Not done yet - Jack is happy to pass on acquisition information in case anyone else gets some time to do it. + +- Should `email-exp-controls` Teams channel contribute to support issues effort calculations? -> all agreed to only discuss issues in this channel in 'support-issues' for accounting purposes. 'Read Only' access applied to channel? -> Yes it's read-only. + +- New web dashboard - JH would like to further develop -> TW suggested a code-chat/review of what's been done already before deploying. Agreed would be done on 25/07/2024. UPDATE: JH gave talk & demonstration on morning of 01/08/2024. -> JH is planning to work on this next sprint. + +- NOT looking at project board in Stand-Up meeting (as new checks cover most issues) -> agreed to check weekly during Friday's meeting as a trial.-> all finding it a good thing. + +- Adding new sub-modules to EPICS Top repository -> agreed to create and add empty submodule (to master branch) at beginning of ticket, reviewer will then populate during final merge. -> Not yet done. + +- Concern over large number of 'in progress' tickets -> suggestion to use 'impeded' label more freely rather than just for external reasons. -> Doing alright. + +- 15 minute wait during Stand-Up - awkward with gap in middle while those present wait for others who've said they'll be late. Sometimes gap filled with looking further at system issues -> LC suggested to start 10:15 and do personal updates first, then look at system status. LC has checked meeting room availability. Agreed we'll try this for two sprints and review next but one retrospective. **Highlight this for reviewer of said sprint** -> All agree that 10:15 is much better. + + +## Current Sprint +- DK: I think it would be beneficial to the team if every member of it showed a summary slide at each sprint review, no matter how large or small their own perceived contribution to the project may be. -> Unanimous agreement generally. +- FA: Could we move sprint planning to be 10:30 to 12:30 which would be more in line with our start time to allow people travelling further to get to the meeting in time? We could also then use the existing 10:15 stand-up slot for a triage of any support issues for when planning is in cycle. -> There was a certain amount of disagreement due to potential conflicts with out of core-hours, in particular lunchtime. Needs further discussion. +- TW: Could we encourage signing off messages sent from the shared mailbox with something like "Tom (on behalf of exp controls)" or similar? Not just for our scientists actually, sometimes it's a bit tricky as a developer to find out who replied to one of the messages. -> Agreed within the team that we should always try to associate a 'human' with a communication. +- LJ: Do we want to be more strict about re-pointing tickets at sprint change-overs/when putting them into review? we started this sprint with supposedly a large amount of points in review, but often when this is the case it's actually not representative of amount of work to be done. -> There was some discussion and no conclusion was reached. Needs further discussion. +- TW: SURF caused a lot of support issues this sprint, which fell disproportionately on one or two team members. The feeling is that we are taking the blame for issues outside of our control. -> TW suggested more pairing when addressing SURF support calls. All agreed this was a good suggestion. JH/DK suggested a presentation to the dev team on the software architecture etc. to increase familiarity. + + +## Mad/Glad/Sad + +- LC: Sad that Isaac was leaving and he will be missed. +- DK: Sad about SURF issues. +- LJ: Glad that we now have coding standards in place and that we can now build on them. +- IG: Mad that had to share a meeting room with very active rodents above the ceiling. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.09.04.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.09.04.md new file mode 100644 index 000000000..0474aafe9 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.09.04.md @@ -0,0 +1,42 @@ +# 2024-09-04 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| KB | LJ | JD | + +## Present +- in person: GR, LC, LJ, TW, JD IG +- online: SC, KB, DK, DM, CMS + + +## Previous Sprint + +- Unanimously decided to keep the 10:15 stand up meeting - as it is better than having a large gap in the middle. +- People have been signing off messages sent from the shared mailbox with their names which is good. +- Decided to talk about moving sprint planning meetings (to 10:30) next retrospective as FA is not present. + +## Current Sprint + +- KB: Suggested that the deadline for proposal tickets going into next sprint should be 6pm the evening before. Unanimous agree and reminders should be sent out to enforce this, at least until it becomes a habit of the team. +- GR: Priority of tickets being ignored: + - Either don't assign priorities to tickets or take action to make sure that people are picking up the more important ones. + - The aim is for higher priority tickets to be picked up regardless of whether they're interesting or not. + - The main alternative is for it to be enforced that everyone picks from the top of the list which isn't ideal either. + - New starters should be encouraged as they progress to go for more higher priority tickets depending on ability. +- TW: Creating a release is painful: + - We ought to create a 'CI' like process for releases so that when it comes to producing a release, there aren't lots of unexpected problems to solve in a small time constraint. +- JH & TW: Code reviews at Thursday standups: + - Decided that if no one volunteers to do a 'show and tell' then the ticket at the top of review will be picked and the code will be looked at by the team. As it is a review, it should be understandable by the whole team anyways. Also helps whoever is about to pick up the review. +- LJ & KB: `Repointing` Tickets: + - Decided that with the burndown chart coming back there is no need to repoint tickets either during the sprint or between sprints. There was concern about a loss in resolution in how much time a ticket will take after end of a sprint. + + +## Mad/Glad/Sad +- DK: Glad that the deploys went well and that the manual testing process had gotten faster. +- KB: Glad that the changes to the tasks board were working well with the team and that the updated project board automation was almost back. +- Mad that CR11 hadn't been fixed regarding the microphone issue and ceiling pigeons. +- LJ: Sad that she had been ill. +- LC: Mad that she was manually updating the burndown, then became ill. + +## Other +- TW: Technician Training with scientists. David volunteered to help Tom with the training. \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.02.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.02.md new file mode 100644 index 000000000..5c407f41d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.02.md @@ -0,0 +1,49 @@ +# 2024-10-02 + +| Chair | Timekeeper | Note Taker | +| :--------: | :---------: | :----------: | +| CMS | FA | KB | + +### Present: +- in person: FA, TW, JD, IG +- online: JH, KB, GR, SC, CMS, DK, LJ + +## Previous Sprint +* Failed to remind, but 6pm on the Wednesday before the planning meeting is OK +* Possibly gone too far the other way at the moment re show and tell, some concern that we are now presenting there things which should be a code chat + +## Current Sprint +* Timekeeping when talking about own tickets + * Others can and should help in this situation +* Bluesky presentation was good +* Nicer UI for the github configs repo + * Ongoing, new authority types set up, need to keep testing + * Action: JH will follow up on this +* VMS service and controls link + * Discussed at group meeting + * Controls moving to EPICS and a future option may be something different +* Ticket labels + * Are they still useful for viewing tickets in other locations? Yes for some + * Automation will help +* Day for Ruff/Pyright fixing + * We should have one + * Have this instead of a Friday day in the Sprint + * Action: KB to find a suitable day +* Testing all IOCs + * Mainly resolved in the thread + * First task is to resolve running tests in parallel +* Planning meeting timings + * Defer a second time to when Freddie is present +* Task board view + * Use a table rather than a board, gives better information in a good order + * Need to ensure flash reviews are brought to the top + +## Mad/Glad/Sad +KB: As PM, sad that only one out of three instruments has migrated, but glad that they have been delayed for good reasons + +DK: Glad the training went well + +JH: Feels there have been fewer calls for basic items + +## Other +Nothing discussed \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.30.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.30.md new file mode 100644 index 000000000..6f7243b45 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.10.30.md @@ -0,0 +1,72 @@ +# 2024-10-30 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| CMS | KB |LC | + +Present: +- In person: FA, TW, JD, IG, LJ, LC +- Online: JH, SC, CMS, ES, KB + +## Previous Sprint +- Automation: we now have it on the sprint board, with labels being added/removed when moving a card through columns +- Ruff/Pyright day: we did this. +- Using task table instead of task board: this seems to work quite well +- Was remarked last retro that there felt like fewer calls for lat cycle, however after doing the maths it was also our most support-heavy cycle of the year + +## Current Sprint +### Moving planning meetings later in the day +- Initial proposal was to move them from 10-12 to 10:30-12:30 + - This however cuts into lunch hours, meaning that those who arrive early will have to wait even longer to eat and go hungry for a bit. + - Additionally can cause issues with people putting meeting early afternoon in diaries which might altogether remove your lunch break +- Balance is people starting early and needing to eat vs needing to travel in in the mornings later to avoid paying for peak tickets +- JD suggestion: what is stopping us from moving to an afternoon meeting? + - This technically goes against agile methodology (theoretically no work going on between review and planning, in practice that never happens) + - However we're not held to agile irrevocably, so why not? + - There used to be a lot of post-planning work, but not any more + - Could lead to there being many meetings on a single day when in-cycle which is bad for support +- Going to try it as an afternoon meeting for the next few months + +### Using Gitlab for instrument config branches +- Won't let us use it due to bad commits +- would have to rewrite history on 4 instrument branches (thankfully not on any others) +- LJ willing to take a look + +### ISIS Branded merchandise for new starters +- JH Hopefully getting around to it this week + +### NDX security +- GR highlighted the fragility of the security on NDX machines in the teams channel + - default user/pass was found on web search +- Bigger problem than was highlighted in the message +- going to revisit with GR present to make sure no points misrepresented + +### Cabin PC passwords +- Should we have scientists share their passwords with us in keeper? There were a few instances where we needed them and had to look on a whiteboard for them + - Generally don't need to use them but e.g. IMAT has a couple of machines we need to log into for cameras + - A user called us on the support phone when they needed the cabin PC password, and the local contact was neither local nor contactable, so we had to tell them to read words on the whiteboard until one of them sounded like a password to us + - Should we even be giving them out? Probably not. +- Could go round the halls once a year looking for passwords on whiteboards and adding them to keeper +- Could also just ask the scientists if they don't mind giving us the passwords + +### Release numbering +- Every release for a while has been a major release and incrementing the large version number +- Could be confusing figuring out which instruments a given release was deployed to +- Will instead move to YY.MM.Patch format from next release (now 25.02.0) from next release onwards + +### Moving sprint meetings based on availability +- Will bring this up again when GR present +- More labour to check, but LC willing to do a little rescheduling on further out days if the team is ok with it + +### Demos +- Should not be a whole cycle after release, should put it in PI template +- It's a good way to ensure whole team is visible to science groups for more than just specific devices etc. +- Even if only 3 out of the 5 demos have people show up, will still be worth it + +## Mad/Sad/Glad +Mad: +- SC mad of pyright because there's now work needed before submitting to review, but... +Glad: +- SC glad at the same time that it's teaching better python practices +- Collectively glad that old code is being updated to follow pyright + - As such less messing around with it when you change a single file \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.11.27.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.11.27.md new file mode 100644 index 000000000..d1fb02f33 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.11.27.md @@ -0,0 +1,106 @@ +# 2024-11-27 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| LC | IG | JH | + +Present: +- In person: FA IG TW LJ GR JD LC +- Online: JH KB ES CMS SC DK + +## Previous sprint + +### Instrument configurations on gitlab + +- [ ] **Action: LJ to persevere with trying to sort out the dodgy commits which prevent us from doing this.** + + +### Security - cabin pc passwords on whiteboards +- we have been adding these to keeper as we go to support calls. +- we should not give out passwords over the phone. + +### Release versioning +- we agreed to move to `YY.MM.patch` for the next release. + +### Demos +- these should be part of the PI template to have them done before a whole cycle after the release. + + +## Current Sprint + +### Security of the current NDX set up +we should set something up to talk about this in the group meeting. + +### Timing of sprint meetings +GR has meetings in advance that cause clashes. can we be a bit more flexible about sprint meetings? +KB has predicted sprint dates for the next year. It's difficult to book meeting rooms in this way because there are always clashes. +GR has agreed to take some of the labour off rescheduling by finding new rooms if need to reschedule and then updating the meeting. +This would mean just the meeting changes, but sprint deadlines will remain. we should always mark ourselves as out of office, in our own and the experiment controls. calendar, to make both of these tasks easier. + +### Multiple repos on the project board +- this is a good thing and it's working pretty seamlessly. +- this means issues are much more isolated to where the actual code is. +- it's quite easy to transfer issues now, this is great. + +### Closing old tickets +- lots of work has been happening to close old/dead tickets +- main IBEX issue repo is still useful for general tickets, we can transfer if needed. +- it takes a lot of time to go through old tickets as we have tickets that are over a decade old. +there is a big list now which is for culling, KB needs assistance from the group to look through these and has marked some with "verify" for this. + +### Professional versions of pycharm +would like the professional versions as vs2022 not quite as good + +- [ ] **Action: JH will ask Irene for numbers for half the group and all the group, we will then talk about it again** + + +### Group mailing lists +- there are a few different mailing lists for the experiment controls, we should try and make these more concise as there is a mismatch in the members of each. +- one of these is a mailbox, one is a mailing list, some are sharepoint groups. +- all of the above are showing up in outlook - we might not have control over them so may need to ask someone else? +- we should write down what we want to do about these groups and do it. + +- [ ] **Short term action: could FA or CMS make sure that the members are at least the same.** +FA: we may be able to reduce them down to one anyway. + +### Release notes numbering +- we should make the "Upcoming" release notes the next version. +- refer to previous sprint discussion on this for more details + +### Demo timing +- already discussed in previous sprint + +### Tracking of email support requests out of cycle +- out of cycle we should have a rota - will discuss in group meeting + +### Locking the office when not in use +- we should lock the office when we're not in it. no-one can get locked out as it's a keypad so there's not really an excuse. + +### Next release RC possibility +- next release is looking more complicated because of dependency updates, can we create release candidate for pre release so we can be told about all the broken stuff before we deploy in February. +- this is quite tight as we need to be creating the next release in January +- CRISP may be busy, but other instruments may be willing, though unlikely as we're currently in cycle. + + +### work experience students +do we have suitable work for students? +- yes, we can do this, and there is plenty suitable work. +- we could have 2 students and they could work together? +- we may not have enough desks for this so may have to be strategic when people are on leave etc. +- if out of cycle much easier for someone to WFH for a week. we can also create space in end office for one of them + +## Mad/Sad/Glad +Mad: +- no one is mad + +Sad: +- no one is sad + +Glad: +- Lots of people working on bluesky, great that it worked so well +- number of issues has dropped by 300 +- ruff is good as it's improved quality of code and caught bugs that may not have been found. + + +Other: +- could we be stricter about timekeeping in the review section so we're not rushed in the retrospective. we should use the time estimated section in the slide as it makes timekeeping easier. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2024.12.18.md b/doc/processes/retrospective-notes/Retrospective-Notes-2024.12.18.md new file mode 100644 index 000000000..d09dfe8bb --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2024.12.18.md @@ -0,0 +1,63 @@ +# 2024-12-18 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| FA | GR | JD | + +Present: +- In person: IG GR JD +- Online: JH KB ES CMS SC DK FA + +## Previous sprint + +### Security of NDX set up +- Decided not to talk about this in group meeting but rather have a meeting dedicated for it + +### Closing Old tickets +- Backlog has been reduced by 300 so far +- Mention of creating a list of tickets that need 'verifying'- could do this as part of standup + +### Pycharm Pro +- Waiting to hear back form Irene regarding getting the whole team set up + +## Group Mailing Lists +- There are multiple mailing list for exp controls, we should try and make them more concise +- Having a sharepoint mailing list might be useful for letting people e.g summer student access sharepoint (CMS) +- Worth holding a meeting? + +### Demos +- Muons still need demoing for (JH + SC) +- Should we still provide a demo even though the next release is right around the corner? +- Its important to be interacting with users especially muons (Agile) (KB) +- Demoing to muons whilst demoing winter release could be risky. Better off apologising to group + +### Email support rota out of cycle +- In place (KB) + +### Work Exp Students +- Application put in for 2 students. +- FA concerned about desk space +- JH asked if it would be possible to have an apprentice? +- KB was concerned that we might not be eligible for one + +### Other +- Make sure to be running latest MySQL and other components to begin 'testing' for a release candidate. (FA) + +## Current Sprint + +### Points added to project board (KB) +- Can now see a tickets points from repos and project board +- Additional to point labels +- Points allocated per ticket need to be the same so automation being worked on + +### Standup/Meeting rotas- put roles in exp controls calendar +- This would take too much time (KB + LC) + +## Mad/Glad/Sad +- FA Mad/Sad - HIFICRYOMAG memory being hogged by Sophos - could look at Microsoft alternatives? +- KB Mad/Sad - People outside of the team making decisions on our behalf where they shouldn't be e.g Admin by request +- FA Mad/Sad - Industrial Placement restrictions for next year +- CMS & most others Glad that Christmas Meal was organised so well and held. Sad that FA and SC couldn't make it. +- DK Glad - support rotas are working well. Needs review (FA) +- FA Glad - Log files more clear & rotating properly +- FA/JH Glad - Bluesky testing continues to be going well \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2025.01.07.md b/doc/processes/retrospective-notes/Retrospective-Notes-2025.01.07.md new file mode 100644 index 000000000..4a298e59d --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2025.01.07.md @@ -0,0 +1,23 @@ +# 2025-01-17 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| CMS | ES | IG | + +Present: +- In person: IG GR TW JD +- Online: JH LJ KB ES CMS SC DK FA + +## Previous sprint +- Points added to project board (KB) +- Calendar now shows first line support. +- Christmas meal was excellent. + +## Current Sprint (short) + +### PAT Testing +- Could cause issues with release, so need to try to synchronise activities. + +## Mad/Glad/Sad +- LJ Glad - Glad for a good Christmas break. +- KB Glad - People managed to extend their Christmas break somewhat. diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2025.02.04.md b/doc/processes/retrospective-notes/Retrospective-Notes-2025.02.04.md new file mode 100644 index 000000000..06b20f5e6 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2025.02.04.md @@ -0,0 +1,81 @@ +# 2025-02-04 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| TW | | JH | + +Present: +- In person: IG GR KB +- Online: JH TW ES CMS SC DK FA LJ LC JD + +## Previous sprint + +nothing - short sprint + +## Current Sprint + +### PI changeovers can be confusing +Christmas PI changeover was tricky - if there are any ideas to make this easier please suggest. + +### shared group laptop +it's very helpful (in fact it may even be another "excellent helper"?) to take to meetings and to the halls + +### ticket priority +medium and low have been picked up ahead of high priority tickets this sprint. In this case the release tickets were dependent on each other, but that doesn't explain all of them. +we should pick from the highest priority downwards regardless of if we want to do it or not. + +### putting code in the source code of the dev manual +we should move these out to the _code_ repository, not the wiki repository, as they are a bit of a pain to see otherwise as you have to clone the wiki repository. + +### manual system tests on release +are the statuses currently in place enough? should we split into `fail`, `fail - fix`, `fail - noted`? Yes. Issue proposed to do this and improve the template: https://github.com/ISISComputingGroup/IBEX/issues/8633 + +### build nodes +they are slow. we should axe off NDW1757, it's terrible. Can we have faster build nodes? +we could build over the network and split things client/server as servers have caches. The main bottleneck for build nodes at the moment is disk speed, perhaps we need more node. +we could consider using dev machines overnight as jenkins nodes. Would have to be careful about soak testing etc. +RIP NDW1757 + +### feedback from scientists - confusing "instrument patching" / "ibex deployment" email +can we combine these better so that its less confusing? +we had the opposite request last time. +we don't normally send out "done" emails as it's just extra admin. perhaps we should start doing so? + +### galil-old is bad +yes, it's bad, we should test the new one, it's hard to maintain and keep features up to date. +galil-old caused issues over the last release as it relies on VS2010. +the new galil driver isn't quite there yet, we need to test on CRISP (they're happy for us to do so) + +### email sent to unclear +it was an old FIT email - we'll delete it. + +### nice that we can pip install genie_python now +yes + +### first line support +we should remind people who are on first line support the next week - we'll start doing this at standup. + +### zipping up builds +Sophos is horrible and tries to read the millions of files that e.g. client builds are made up of - we should zip these up. +we should also do this with the ibex backup from NDX to data-old. +turns out we already do this! but we should make our nightly builds not bother with leaving the unzipped build. that will save a lot of files + +### TwinCAT on developer machines +we should not bother installing it on developer machines, and leave NDXMOTION to run TwinCAT XAR so it can simulate Beckhoffs. + +## Mad/Glad/Sad + +glad - release done! + +glad - pat testing done! + +glad - interviews done! + +glad - we won't get timing system issues this cycle (allegedly) + +mad - the release was not fun + +mad - partial derivatives and asymmetry is hard + + + diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2025.03.05.md b/doc/processes/retrospective-notes/Retrospective-Notes-2025.03.05.md new file mode 100644 index 000000000..bf4ba5706 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2025.03.05.md @@ -0,0 +1,155 @@ +# 2025-03-05 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| GR | ??? | TW | + +## Previous sprint + +PI changeovers confusing + +Shared laptop helpful + +Ticket priority - probably not enough data to tell whether there was any change. + +Source code in dev manual - yes move it - ticket exists + +Manual system tests on release + +Build nodes + +Windows patching / IBEX deployment - note added to draft email by KB + +GALILOLD is bad +- Should we update GALILNEW to be actually new, otherwise GALILNEW is a bit old. + +Remind people when they are on first line support + +Zipping up builds, Sophos is horrible. Tickets. Many tickets. Some of which are conflated with each other. + +Use NDXMOTION to do Beckhoff stuff, don't install XAR locally. JH removed from install & build wiki page. + +## Current Sprint + +### Defunct emails that looks like it's associated with us but isn't. + +FA has sorted this. + +### Instrument Demos + +FA: how to balance too early (scientists on holiday) vs too late (not enough test time) + +KB: think we should book demos week after deploy. + +Conclusion: try to book 2 weeks before cycle + +### Instant awards scheme + +LC: Yes put people forwards for awards. Nominating whole teams / more people will tend to get a bit more scrutiny. + +### Tips/tricks teams channel + +ES: Minor tips & tricks that aren't worth a whole wiki page, maybe a lightweight way to share these little things. + +KB: creating the teams channel as we speak + +FA: more curation/organization might be good long term if it gets too big. + +KB: if channel gets too big then we can move things out of teams channel to somewhere else + +IG: OneNote? + +KB: Onenote in teams not very good + +GR: Maybe let's go for a channel this sprint and then review + +Various: is it searchable enough? + +FA: how categorizable is it? + +LJ: Make replies to top-level "theme" posts? Similar to retrospective channel? + +Conclusion: try teams channel, evolve it over time as needed. + +### Release timeline + +GR: Let's not spend too much time discussing this + +### The end of 🐐 is nigh + +DK: 🐐💀 at end of financial year? + +KB: No. End of project is September 2025. But most people won't be booking significant time to ibex post June. + +GR: Just to be clear this is the 🐐 project, not the 🐐 product + +KB: next PI we might be looking at doing things differently. August. + +DK: Can we have more time for internal team/technical priorities + +FA: Have a formal 80/20 split for tickets which are scientist-driven/not-scientist-driven, but there's also scope for "personal development" type time outside the ticket framework. + +FA: Post ibex finish we can review some of our IBEX tech choices, some bits of 🐐 are looking a bit dated. + +### Wikis + +GR: 3 wikis exist, sometimes with duplicate content. Happy to tinker. + +IBEX: scientist facing +Dev: dev facing +User: how to use ibex + +Conclusion: go for it George. + +JH: repository-specific info - migrate to `README` or docs of each repo. + +LJ: Searching? + +ES: Search at org level + +### Pyright + +JH: people will be upset but it's a good idea + +TW: didn't actually cause too many issues in practice + +KB: still some instruments to migrate in summer + +### Standup + +CMS: Move "Friday" standup tasks to "Thursday"? + +LJ/KB/GR: Discussion about whether we might lose code review time + +LJ: should we be stricter about actually doing the code reviews + +### Staff updates vs standup + +JH: Big announcements (UKRI/STFC/NatLabs level) that various people in the team miss if clashes with standup + +KB: All staff meeting might take priority anyway + +Various: discussion about tangentially related things + +ES: is standup actually important? + +KB: standup is important to connect to others + +GR: We can probably justify missing one standup every so often + +Conclusion: don't know and/or don't care, other meetings might take priority, as long as ops stuff e.g. nagios gets checked. "Someone" will take care of it. + +### + + +## 😠😢😄 + +- FA 😄 we made changes to lots of stuff in release and got away with it +- LJ 😄 about stress(full) rig +- ES 😄 about OPCUA +- GR 😄 about 🌞 in the evenings when leaving work +- CMS 😄 that we've run EMMA on newer windows version/hardware + * CMS 😄 OpenGENIE & SECI work on windows 11 (?!) +- FA & KB & others 😄 about SECI 💀 +- IG 😠 that his 💻 got wiped +- GR 😄 that this meeting wasn't stupidly long \ No newline at end of file diff --git a/doc/processes/retrospective-notes/Retrospective-Notes-2025.04.03.md b/doc/processes/retrospective-notes/Retrospective-Notes-2025.04.03.md new file mode 100644 index 000000000..730eb5574 --- /dev/null +++ b/doc/processes/retrospective-notes/Retrospective-Notes-2025.04.03.md @@ -0,0 +1,146 @@ +# 2025-04-03 + +| Chair | Timekeeper | Note Taker | +|-------|------------|------------| +| JD | ??? | SC | + +## Previous sprint +### Defunct emails that looks like it's associated with us but isn't. + +FA has sorted this. + +### Instrument Demos + +FA: how to balance too early (scientists on holiday) vs too late (not enough test time) + +KB: think we should book demos week after deploy. + +Conclusion: try to book 2 weeks before cycle + +### Instant awards scheme + +LC: Yes put people forwards for awards. Nominating whole teams / more people will tend to get a bit more scrutiny. + +### Tips/tricks teams channel + +ES: Minor tips & tricks that aren't worth a whole wiki page, maybe a lightweight way to share these little things. + +KB: creating the teams channel as we speak + +FA: more curation/organization might be good long term if it gets too big. + +KB: if channel gets too big then we can move things out of teams channel to somewhere else + +IG: OneNote? + +KB: Onenote in teams not very good + +GR: Maybe let's go for a channel this sprint and then review + +Various: is it searchable enough? + +FA: how categorizable is it? + +LJ: Make replies to top-level "theme" posts? Similar to retrospective channel? + +Conclusion: try teams channel, evolve it over time as needed. + +### Release timeline + +GR: Let's not spend too much time discussing this + +### The end of 🐐 is nigh + +DK: 🐐💀 at end of financial year? + +KB: No. End of project is September 2025. But most people won't be booking significant time to ibex post June. + +GR: Just to be clear this is the 🐐 project, not the 🐐 product + +KB: next PI we might be looking at doing things differently. August. + +DK: Can we have more time for internal team/technical priorities + +FA: Have a formal 80/20 split for tickets which are scientist-driven/not-scientist-driven, but there's also scope for "personal development" type time outside the ticket framework. + +FA: Post ibex finish we can review some of our IBEX tech choices, some bits of 🐐 are looking a bit dated. + +### Wikis + +GR: 3 wikis exist, sometimes with duplicate content. Happy to tinker. + +IBEX: scientist facing +Dev: dev facing +User: how to use ibex + +Conclusion: go for it George. + +JH: repository-specific info - migrate to `README` or docs of each repo. + +LJ: Searching? + +ES: Search at org level + +### Pyright + +JH: people will be upset but it's a good idea + +TW: didn't actually cause too many issues in practice + +KB: still some instruments to migrate in summer + +### Standup + +CMS: Move "Friday" standup tasks to "Thursday"? + +LJ/KB/GR: Discussion about whether we might lose code review time + +LJ: should we be stricter about actually doing the code reviews + +### Staff updates vs standup + +JH: Big announcements (UKRI/STFC/NatLabs level) that various people in the team miss if clashes with standup + +KB: All staff meeting might take priority anyway + +Various: discussion about tangentially related things + +ES: is standup actually important? + +KB: standup is important to connect to others + +GR: We can probably justify missing one standup every so often + +Conclusion: don't know and/or don't care, other meetings might take priority, as long as ops stuff e.g. nagios gets checked. "Someone" will take care of it. + + +## Current Sprint +### On call. +Jack H: I much prefer doing the weekend on call first as we are now, as opposed to the weekdays-then-weekend we were doing previously, feels like the worst bit is out of the way at the start. + +David K: Also better for managing cover for the weekend before cycle and the short final week of cycle - which was the original idea for changing the on-call period, IIRC. + +### Centrally-hosted MySQL Database - Status? +David K: What's the status of this? The ticket was last proposed two years ago: [IOC Log Server: Push to a central MySQL instance · Issue #5820 · ISISComputingGroup/IBEX](https://github.com/ISISComputingGroup/IBEX/issues/5820). Is this dependent on the move to Archive Appliance? +We were recently asked to extract data after a local database had been backed-up and truncated on RIKENFE, so had to import it from the network share, which took several hours, then create CSV files of the requested data using the 'IOC log query' script. The instrument scientist could have done all of this themselves from the Log Plotter perspective using the 'Data Export' panel if a central MySQL database had existed. + +### On-site rota +George R: a) for (very good reasons on the whole) we have a situation where the only people on site to day are the ones who only work on site. I accept this might occasionally happen, but I would like to keep an eye out for recurrences of this, as it may suggest that the rota is not working and needs revisiting. +b) Can we agree a process for how people permanently change days and advertise it + +Kathryn B: I thought the rota was the guaranteed days on site, and extras as necessary or swap with someone else to maintain the minimum cover - so if you need to be in an onsite meeting on a day you would normally not be on site, you just come to site and work there instead of at home. + +David K: the up-to-date rota is in a tab in the announce channel: [Onsite Rota](https://teams.microsoft.com/l/entity/1c256a65-83a6-4b5c-9ccf-78f8afb6f1e8/_djb2_msteams_prefix_2670613932?context=%7B%22channelId%22%3A%2219%3Aeaf1bd106e2d4df78f4ea9f7aa3d003d%40thread.skype%22%7D&tenantId=3f66361c-a87e-4158-8f61-99e82db3cac8) + +### EPICS Collaboration on site +George R: It was great to see Evan working with a colleague from Accelerator Controls today. Can we do more to build links with other RAL EPICS users and share knowledge/expertise across groups? +Freddie A: We have had meetings with accelerator controls in the past, but these dropped off at some point, we can restart. I'd offered to host the next "EPICS Oxfordshire" meeting onsite for start of this year, but then as STFC were hosting a full collaboration meeting it was decided to postpone until later in year. + + +## 😠😢😄 +- KB 😄 **Bluesky scripting** has gone really well so far, and I think the message from Diego asking for it on zoom is a great example of this ([diego.alba-venero@stfc.ac.uk via email: bluesky on Zoom](https://teams.microsoft.com/l/message/19:4e381ff6b5674230a74878b1355eec22@thread.skype/1743159129191?tenantId=3f66361c-a87e-4158-8f61-99e82db3cac8&groupId=d9946ec3-a454-424f-b673-5ffcb9f9ade0&parentMessageId=1743159129191&teamName=IBEX%20Developers&channelName=email-exp-controls&createdTime=1743159129191) +posted in IBEX Developers / email-exp-controls on 28 March 2025 10:52) + +- TW 😄 We eventually had SECI free cycle!! + +- CMS 😄 Teams for planning seemed to go very smoothly, should we be moving other meetings? \ No newline at end of file diff --git a/doc/script_generator/DynamicScripting.drawio.png b/doc/script_generator/DynamicScripting.drawio.png new file mode 100644 index 000000000..13fe1c66d Binary files /dev/null and b/doc/script_generator/DynamicScripting.drawio.png differ diff --git a/doc/script_generator/Existing-script-generators.md b/doc/script_generator/Existing-script-generators.md new file mode 100644 index 000000000..ada0fa080 --- /dev/null +++ b/doc/script_generator/Existing-script-generators.md @@ -0,0 +1,158 @@ +# Existing script generators + +### MkScript + +Used by south side muons and MARI + +### MuonScript + +Used by south side muons + +### MACS + +MACS is probably the most interesting muon script generator. A brief overview of functionality: +- Keeps a table-based view of operations on the screen. An "Operation" could be: + * A run at a single combination of temperature/field + * A scan over temperature or field (in three modes), counting at each point. These scans can be defined as linear or logarithmic. + * An arbitrary block set to an arbitrary value +- Sends commands to OpenGENIE via the SECI API as it goes - means that scripts can be edited (up to the point when they have started running). This is similar in spirit to NICOS script server. +- Includes some basic time estimation (it displays the total count time of everything in the script window) + +**Note:**
+MACS can only run on a real instrument, with a copy of SECI and working communication to hardware. It _**cannot**_ be run offline. +Furthermore, because the instruments (ARGUS & CHRONUS) are used in-cycle, the development team can only use MACS out-of-cycle. + +### INTER + +INTER has a script generator called MaxScript. It has fallen out of use because it is not easy to integrate with the SECI system. Scientists from Inter expressed the desire for the script generator to be able to talk to the script server directly, with feedback (for example, greying out completed rows where a row represents an experimental measurement). + +The INTER script generator can also display the script in a "tree" view - this collapses each run into a short recognisable string, and the details can be seen by expanding out the tree item. + +The INTER script generator provides some time estimation. + +INTER script generator is available at: `\\isis\shares\ISIS_Experiment_Controls\external_code\inter script generator` (copied from https://github.com/jonasgitt/ScriptMax with required DLLs and extra files added). Launch via the Qt IDE (download latest version from web; takes a while). I haven't been able to build it successfully to an exe, but only spent half an hour on it. + +### LOQ + +LOQ uses a script generator "written by Matt Clarke in VB or C#". The scientists have expressed a desire for the script generator to be simple from a user's perspective. + +## Commonalities of old systems / feature requests for new system + +### Configurable + +There is a clear need for the script generator to be highly configurable. Common actions that scientists want to take are: +- `cset` a block +- run an arbitrary instrument script (with parameters) + +There is also a need for "hooks" into which the instrument scientists can insert arbitrary code. For example, if the script generator (/server) are told to stop, some instruments may want to simply terminate execution of all code, whereas others may want to perform some "cleanup" (for example, setting the beamline back to a known state). + +### Offline mode + +Being able to use the script generator offline (in advance) as well as alongside/integrated with IBEX is important. + +### Spreadsheet / table based + +Most of the existing script generators are table based. Some have additional alternative means of displaying equivalent information (such as a tree view). At a SAG meeting, there was broad consensus that a table-based UI would be appropriate. + +### Parameters + +All of the script generators I've seen have essentially been setting a small number (2-10) of parameters and then performing some kind of run. The functions to set parameters and do the runs are different for each beamline, so each beamline will need to be able to configure this. It would be advantageous for a scientist to be able to configure this by themselves, without necessarily requiring input from computing. + +### Parameter checking + +All of the scientists have brought up a desire for parameters to be checked for implausible values (e.g. setting a dilution fridge to furnace temperatures). Many of the existing script generators already do this. How the limits are defined should be configurable by instrument scientists. + +### Output + +All of the existing script generators generate a plain text file. However, the overwhelming consensus is that a script written using the new generator should be able to be executed "in one click", without requiring copying/saving a script and loading it explicitly. + +However, the *option* to explicitly generate and load a script is still important, as some users like to edit the generated script before it is sent. + +### Execution Feedback + +Scientists have expressed a desire for live feedback visible in the script generator while the script is running. For example, rows should be greyed out once that part of the script has been executed. + +### Time estimation + +Some of the existing script generators already do some form of time estimation - given instrument-specific assumptions. This is a feature that is quite heavily requested by all the different groups. + +### Dry runs + +Some scientists have expressed an interest in dry runs/simulated runs of scripts to detect errors. None of the existing script generators that I've seen contain this functionality. + +At a SAG meeting, the scientists suggested a graph of time (or rather, counts) against arbitrary parameters so that they can check the the graph looks as they expect. + +### Scans + +Some of the existing script generators already support "scanning" a parameter (that is, setting a parameter to a value and taking a measurement at each step. There is broad consensus that this is useful functionality. + +There is slightly less consensus about nested loops - some groups see these as very useful, while others would prefer this "advanced" functionality to be handled by writing the python themselves. + +Logarithmic scans (as opposed to linear) are important for the Muon group, but there was not substantial interest from other groups. + +### Loading scripts + +The script generator should be able to load/save scripts in it's own format. In addition, several groups expressed interest in loading from other formats, for example: +- Excel +- CSV +- custom formats (e.g. ENGINX) +- Raw python code + +### Readability + +There is broad consensus that the generated scripts should be human readable, to facilitate a workflow where they may generate the skeleton of the script using the script generator and then edit it to suit some more advanced requirements for a more complex experiment. + +## Suggested implementations + +All of the implementations below assume that the NICOS script server will be used as the back-end, when the script generator is running on an instrument. Using the script server is logical as it already allows for queuing/re-ordering/editing scripts in a queue on an instrument. + +### Approach 1 + +One suggestion to implement the script generator as a Java eclipse plugin that can be run either within the IBEX GUI or standalone. This plugin would be implemented as an MVVM stack in a similar style to the rest of the GUI. + +Advantages: +- It facilitates integration within the GUI while not being too difficult to package as a standalone application. +- We can re-use some of the NICOS code from the GUI +- It is a UI technology which IBEX developers already use and are already familiar with + +Disadvantages: +- Using Eclipse RCP might constrain us unnecessarily +- It might be easier to work with generating python code in python itself + +### Approach 2 + +Another suggestion would be to implement the script generator as a standalone Python program (using tkinter or a similar UI framework). As above this would be implemented as an MVVM stack. + +Advantages: +- It might be easier to interface with Nicos as nicos is written in Python +- It may be easier to generate python code using python than java (for example, access to the `ast` module) +- Python is a technology which IBEX developers already know and use (minus caveat about UI toolkits below) + +Disadvantages: +- IBEX developers are not currently familiar with Python UI toolkits. This would be another technology to learn. +- Some of the NICOS functionality which we've implemented in the GUI would need to be reimplemented in Python +- It would be more difficult or perhaps impossible to integrate this into the main GUI later + +### Approach 3 + +Approach 3 would be to attempt to adopt and adapt one of the existing script generators, and retrofit script server support to it and add any other features that are requested. + +Advantages: +- Some functionality is already implemented in the other script generators. + +Disadvantages: +- The quality of the existing code is largely unknown +- The existing script generators are written in technologies which were not necessarily familiar with, or which may be obsolete +- It is unknown how generic/extensible the existing script generators are. Most are fairly specific to a small number of beamlines. + +### Approach 4 + +The script generator could be written as a web application, enabling it to be used both from within the user interface and externally. + +Advantages: +- The script generator can be accessed from anywhere without needing to be downloaded + +Disadvantages: +- The IBEX team as a whole do not have significant expertise in web technologies +- It is not clear how this would interface with the NICOS script server (and what the security implications of doing this would be) +- Running the application in a web browser may constrain us unnecessarily. \ No newline at end of file diff --git a/doc/script_generator/Gotchas-and-Troubleshooting-for-The-IBEX-Script-Generator.md b/doc/script_generator/Gotchas-and-Troubleshooting-for-The-IBEX-Script-Generator.md new file mode 100644 index 000000000..5e558fdbe --- /dev/null +++ b/doc/script_generator/Gotchas-and-Troubleshooting-for-The-IBEX-Script-Generator.md @@ -0,0 +1,48 @@ +# Troubleshooting + +## Script generator tab is not shown + +By default the script generator is disabled in the IBEX GUI. To enable it [Follow this step](#scriptgenerator_enable_perspective) + +## Cannot open generated script in notepad++ from the FileMessageBox + +Currently, we search for notepad++ in "Program Files" and "Program Files (x86)". It may be that notepad++ on the computer being used isn't stored in those two. For the time being copy and paste the file location to open with their preferred editor. + +## Warning: Could not load any script definitions from `` + +This could be due to 4 things: + +- The script definitions have not been loaded into the correct place in the filesystem + - Indicated in the log by `Error loading from ` + - To fix simply move them into the location that is given in the warning message +- The IBEX preference is not pointing to the correct location in the file system + - Indicated in the log by `Error loading from ` + - For a temporary fix you can move the script definitions into that location, but for a better fix (see gotcha below) ensure that the preference is pointing to the correct location (The preference can point to multiple locations by separating them with commas) +- The script definitions all have errors on them and cannot be loaded + - Indicated in the log by `Error loading : ` + - The script definitions are maintained by the instrument scientists, however, obviously provide support and guidance for this +- There is a chance that Python has crashed due to an error + - This will likely be in the log + +## Some of my script definitions have not loaded + +- We can point a preference to where to load script definitions from. This can be a combination of different places separated by commas. One of these preferences may be slightly incorrect and the script definitions from that location not loaded. + - Indicated in the log by `Error loading from ` + - Find out where the preference should point (see gotcha below), adjust it and reload the GUI +- Some of the script definitions may have errors in them and so failed to load + - Indicated in the log by `Error loading : ` + - The script definitions are maintained by the instrument scientists, however, obviously provide support and guidance for this + +## The script generator renders as a blank view + +- Delete the `workspace` folder (next to the executable for the script generator) and then retry launching it. If the workspace is owned by a different user, this can cause problems with file permissions. + +## The script generator is constantly loading + +- This is likely based on loading a dependency + - Failing to load Python for some reason, check logs for Py4J exceptions (check if a C:\Instrument\Apps\Python3 is interfering) + - Failing to load git, check log files (the possible solution is to install git separately so we pick up the system git) + +## Preferences + +Currently, we cannot load the main ibex client preferences from the script generator. So the script generator has it's own plugin_customization.ini containing some duplicate preferences i.e. script_generation_folder and script_generator_config_folder. When editing the preferences these should be changed in both files. \ No newline at end of file diff --git a/doc/script_generator/IBEX-Script-Generator-Dynamic-Scripting.md b/doc/script_generator/IBEX-Script-Generator-Dynamic-Scripting.md new file mode 100644 index 000000000..67ed5d187 --- /dev/null +++ b/doc/script_generator/IBEX-Script-Generator-Dynamic-Scripting.md @@ -0,0 +1,20 @@ +# Dynamic Scripting + +Dynamic scripting refers to requirements that allow a user to control the execution of actions whilst a script is running. + +## Design + +Dynamic scripting can be in 4 possible states: Playing, Paused, Stopped, Error. The design makes use of four design patterns: [adapter](https://refactoring.guru/design-patterns/adapter), [state](https://sourcemaking.com/design_patterns/state), [observer](https://refactoring.guru/design-patterns/observer) (using the ModelObject and PropertyChangeListener Java classes) and [factory](https://refactoring.guru/design-patterns/factory-method) (note: most articles actually talk about the abstract factory pattern we are using a more traditional factory pattern). + +Three of these design patterns have been modified to fit our purpose. State, Factory and Observer are used in conjunction to achieve state changes that are reactive to changes in the script server and model changes (handled asynchronously with observers over py4j bridge). State changes are not done by a return value of a method but by firing a property change from a `DynamicScriptingState`. This property change is then received by the `DynamicScriptingManager` and the `DynamicScriptingStateFactory` is then used to create the next state. + +The adapter pattern is used twice: once as an adapter to the `NicosModel` and the other as an adapter to the `ScriptGeneratorSingleton`. + +Dynamic scripting also makes use of the [dependency injection technique](https://www.freecodecamp.org/news/a-quick-intro-to-dependency-injection-what-it-is-and-when-to-use-it-7578c84fa88f/). See article for more details but this has allowed for ease of testing. + +![UML Class Diagram for Dynamic Scripting](DynamicScripting.drawio.png) + + +## Implementation + +Dynamic Scripting code is stored in the `uk.ac.stfc.isis.ibex.scriptgenerator.dynamicscripting` module of the [ibex_gui](https://github.com/ISISComputingGroup/ibex_gui). As well as code for the UI in the `uk.ac.stfc.isis.ibex.ui.scriptgenerator`. diff --git a/doc/script_generator/Script-Generator-Testing-Environment.md b/doc/script_generator/Script-Generator-Testing-Environment.md new file mode 100644 index 000000000..04a7f9afd --- /dev/null +++ b/doc/script_generator/Script-Generator-Testing-Environment.md @@ -0,0 +1,42 @@ +# Script Generator testing + +After the first full release of the script generator testing on personal computers revealed a problem with our current testing standards i.e. running on dev machines ensures we have things like git, Python and the JRE already installed and available impeding our ability to simulate a "home environment". + +A few options: + +- Run up a VM with windows and copy a built version into it for testing (see VM Testing below) +- Have a "clean" windows server to remote desktop into and copy a built version for testing + - We will have to maintain the cleanliness by preventing setup of Java, Git and Python in it + +## VM Testing + +https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/ + +Unzip download and open ova file with virtualbox. + +Create shared folder to put script generator in. Auto mount and make permanent selected. + +Put built script generator into shared folder then copy from there onto VM (Doesn't like running from shared folder). + +Then test whatever you want. + +Things to check: + +- You are using the bundled python + - Start task manager and look at the python running the script generator script +- You are using the bundled JRE +- You are using the bundled git + +## Licensing + +- https://windowsdev.azureedge.net/eulas/WindowsDeveloperVirtualMachineEula20151118.pdf +- `Section 7` describes what we are not allowed to use it for + - This does not include any of our currently intended purposes +- `Section 1 part c` worries me slightly + - I think we are using it to demonstrate and internally evaluate software on windows + - I am not sure whether or not this would be classed as a commercial purpose (I think this means as commercial hosting so I believe it is fine if you take into account `Exhibit A 2cv`) +- `Section 4` + - We can have a backup copy and reinstall it + - Does this mean we can keep reinstalling it? +- `Exhibit A 13dii` "you may not use it after the evaluation period" + - Does this mean that we cannot reinstall it or use the same VM? \ No newline at end of file diff --git a/doc/script_generator/Script-generator-high-level-design.md b/doc/script_generator/Script-generator-high-level-design.md new file mode 100644 index 000000000..8d40be4f4 --- /dev/null +++ b/doc/script_generator/Script-generator-high-level-design.md @@ -0,0 +1,143 @@ +# Script Generator Design + +Meeting to discuss this happened. Summary of the main points that were discussed is below: + +## High level design + +A high level design to use **Eclipse RCP** was agreed. We will build this in two configurations: +- As a perspective in the IBEX GUI, similar to other existing perspectives +- As a standalone application
+**Note:** these two configurations will be using the same code (except for build-specific files). + +The following general design points were agreed: +- The GUI will be written in Java (eclipse rcp). It should have minimal logic in it other than pure display logic +- It will talk to a python process embedded in the GUI, whose responsibility it is to generate a python script based on the input that a user has provided + * Py4j is already included in our GUI build and allows for Java-Python communication + * We need this process to be implemented in Python because it needs to be able to run arbitrary user-supplied python parameter validators (see also "The `ScriptDefinition` class", below). + +{#scriptgenerator_script_definition} +## The `ScriptDefinition` class + +Scientist defined script definitions must be python 2/3 compatible to work with genie python currently. + +A `ScriptDefinition` is the base building block that the script generator will use to build up scripts. It is essentially a wrapper around a Python function that can tell us: +- Whether a given set of inputs is valid +- The types of the parameters +- How long each action will take (optionally) +- Possibly other metadata in future + +As an example, consider a class that looks something like: + +```python +from genie_python.genie_script_generator import ScriptDefinition, cast_parameters_to + +def mytype(string_input): + if string_input == "default": + return 0.0 + else: + return float(string_input) + + +class DoRun(ScriptDefinition): + + @cast_parameters_to(temperature=float, field=float, uamps=mytype) + def run(self, temperature=0.0, field=0.0, uamps=0.0): + g.cset("temperature", temperature) + g.cset("field", field) + g.begin() + g.waitfor_uamps(uamps) + g.end() + + @cast_parameters_to(temperature=float, field=float, uamps=mytype) + def parameters_valid(self, temperature=0.0, field=0.0, uamps=0.0): + errors = "" + if not 0.1 <= temperature <= 300: + errors += "Temperature outside range\n" + if not -5 <= field < 5: + errors += "Field outside range" + if not -20 <= uamps <=32: + errors += "uamps outside of range" + if errors != "": + return errors + return None + + def get_help(self): + return "Help string" + + @cast_parameters_to(temperature=float, field=float, uamps=mytype) + def estimate_time(self, temperature=0.0, field=0.0, uamps=0.0): + return temperature * 10 + uamps * 100 +``` +(this is not necessarily the final API) + +We would probably supply some default `ScriptDefinition`s, but instrument scientists would be responsible for writing and maintaining their own instrument-specific script definitions (in a similar way to the current instrument scripts). + +Currently, a default value has to be provided for every argument of `parameters_valid` and `run` due to the way we are passing kwargs. The parameters that are passed are all currently strings and must be converted to whatever type the user wants. We are providing a decorator `cast_parameters_to` to allow scientists to easily cast parameters and also define their own converters (see example `mytype` above). The custom converters means they can handle edge cases they are expecting from users. + +The default values are used by the script generator to populate initial values for an action. However, this does not rely on the presence of a default value. In the absence of a default value the default is set to the name of the parameter. + +## User interface + +There are two main types of user interfaces that support slightly different workflows. We have decided that we will eventually need to support both, but as part of the MVP we should only implement the table-based view. + +### Table based (part of MVP) + +This interface is a simple table. It is configured for one (and only one) type of `ScriptDefinition`. It configures it's columns to match the parameters of the script definition. + +For example, for the `ScriptDefinition` given above, the table in the script generator would look like this: + +| temperature | field | uamps | Validity | +| --- | --- | --- | --- | +| 50.0 | -1 | 100 | ✘ | +| 80.0 | 2 | 100 | ✔ | + +Each row in the table corresponds to exactly one `Action`. The type of action must be the same for all rows (otherwise, the columns would no longer make sense). + +A row will be highlighted red if invalid and have a heavy cross mark against it in the validity column. If the row is valid it will be the regular table colour with a validity column cell that is green and has a heavy tick mark. + +A user is able to get the reason for invalidity (provided by the script definition) by hovering over the row or by clicking the 'Get Invalidity Errors' button. + +### List/tree view (not part of MVP, but likely to be requested in future) + +This interface is a list of Actions and their parameters. In this UI, each action can be of a different type. The actions are executed sequentially. This is a more flexible approach, however it looks closer to the final python script, so is slightly more complex to use (as a user) than the table-based approach. The UI would look more like a set of bullet points: +- `doRun - temperature=50, field=5, uamps=100` +- `changeSomething - value=1000` +- `doRun - temperature=100, field=5, uamps=100` + +In this sense, it is quite a thin wrapper over a python script. The main advantage it gives you is parameter type/value checking (and arguably. In future we may be able to extend this approach to allow for loops or other types of more complex statements. + +## Script Definitions + +The script generator will be configured by loading in a script definition file. This script definition file can initially be written in Python - though we have not necessarily ruled out moving to another format in future if the need arises. + +This script definition file will define the available `Action`s, and will be editable by the instrument scientists. Each instrument that uses the script generator will need at least one script definition file (maybe more if they have different experimental setups which require very different scripts). + +### Workflow for editing script definitions + +The workflow for creating or modifying script definitions is: +1. The instrument scientist creates a new or edits an existing a script definition using a text editor (e.g. Notepad++). +1. The instrument scientist saves the script definition: + 1. if using his/her office (or cabin or home) PC, the script is saved locally (to a directory of the scientist's choosing) + 1. if using the instrument control PC, the script is saved `` (to a directory of the scientist's choosing?) +1. The instrument scientist transfers the saved script definition to the script definition repository. + * the scientist will use a command (in practice a python or batch script) to perform the transfer + * the command will print a message to inform the instrument scientist of success or failure of the save operation +1. If the transfer: + 1. succeeds: the scientist need do nothing more + 1. fails: the scientist should contact the Experiment Controls group for support. + +Notes: +1. We expect instrument scientists to create/modify script definitions. We do not expect users to create/modify script definitions. We should advise instrument scientists not to allow users to create/modify script definitions. +1. We will create a python/batch script to manage transfer of saved scripts to the git repository +1. We will prevent instrument scientists from committing as `spudulike`, the script will have to ask for a username (e.g. `fedID`) and password to commit with +1. Script will generate a commit message that identifies that the push was done by the script (i.e. to help us to identify situations in which someone has tried to circumvent the script by using git commands directly) +1. Script will prevent the user from setting username and password to use all the time for pushing to this repository +1. The script will do: pull, add, commit and push to master +1. Must have git and python installed +1. Must have access to repository origin + +### How Script Definitions Repo is Updated on Start + +![](UpdateRepoFlowDiagram.png) + +On start up we want to get the current repository of script if possible and the user lets us. If there is a problem because they have edited their code in the repository we assume this is because they know what they are doing. To edit the code they will need to know how to commit it back to the central repo anyway so we don't need to cover that case except to warn them (they are power users). For normal users they just want it up-to-date or not. diff --git a/doc/script_generator/UpdateRepoFlowDiagram.png b/doc/script_generator/UpdateRepoFlowDiagram.png new file mode 100644 index 000000000..304a4975b Binary files /dev/null and b/doc/script_generator/UpdateRepoFlowDiagram.png differ diff --git a/images/scriptgen.png b/doc/scriptgen.png similarity index 100% rename from images/scriptgen.png rename to doc/scriptgen.png diff --git a/doc/scripting/Bluesky-scanning.md b/doc/scripting/Bluesky-scanning.md new file mode 100644 index 000000000..3a23286a2 --- /dev/null +++ b/doc/scripting/Bluesky-scanning.md @@ -0,0 +1,17 @@ +# Bluesky + +For information about scanning using bluesky, including information about: +- The bluesky run engine, `RE` +- Bluesky ophyd-async devices (for example `BlockR`, `BlockRw`, `BlockRwRbv`, `ReflParameter`, or `SimpleDae`) + * If you are looking at bluesky code which uses `async def` and `await`, and imports `ophyd_async`, you're probably looking at a device. +- Bluesky plans, (for example `scan`, `adaptive_scan`) + * If you are looking at bluesky code with lots of `yield from` statements, you're probably looking at a plan. +- Bluesky callbacks (for example `LivePlot`, `LiveFit`, `ISISCallbacks`, file-writing) + +**See [ibex_bluesky_core docs](https://isiscomputinggroup.github.io/ibex_bluesky_core).** + +--- + +For troubleshooting, see [ibex_bluesky_core troubleshooting guide](https://isiscomputinggroup.github.io/ibex_bluesky_core/dev/troubleshooting.html#). + +Please add troubleshooting information to that guide (via a pull-request on `ibex_bluesky_core`) rather than this page. \ No newline at end of file diff --git a/doc/scripting/Genie-Python.md b/doc/scripting/Genie-Python.md new file mode 100644 index 000000000..a52b1f9e9 --- /dev/null +++ b/doc/scripting/Genie-Python.md @@ -0,0 +1,17 @@ +# Genie Python + +```{note} +These pages document user-facing scripting workflows. Looking for developer information about python? See [Python](../system_components/Python). +``` + +```{toctree} +:glob: +:titlesonly: + +genie_python/* +``` + +### User documentation + +- [Scripting in the user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Scripting) +- [genie_python reference](http://shadow.nd.rl.ac.uk/genie_python/sphinx/genie_python.html) \ No newline at end of file diff --git a/doc/scripting/Matplotlib.md b/doc/scripting/Matplotlib.md new file mode 100644 index 000000000..fdde2be5a --- /dev/null +++ b/doc/scripting/Matplotlib.md @@ -0,0 +1,38 @@ +# Matplotlib + +```{seealso} +[User-facing docs](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Matplotlib) on user manual. +``` + +We allow plotting in IBEX via `matplotlib`. These plots will appear in the IBEX client. + +As of [ticket 6719](https://github.com/ISISComputingGroup/IBEX/issues/6719), we are using a custom matplotlib backend. This backend comprises two main components: +- The websocket backend in `genie_python\matplotlib_backend\ibex_websocket_backend.py`, which is based on matplotlib's `WebAgg` backend. This serves plots over a websocket connection. +- The front-end code in `/uk.ac.stfc.isis.ibex.ui.graphing` in the GUI. This reads data from the websockets published by the backend. It is effectively a Java reimplementation of `WebAgg`'s front-end code, which is in `python\Lib\site-packages\matplotlib\backends\web_backend`. + +## Python (backend) component + +Our custom backend is defined in `genie_python\matplotlib_backend\ibex_websocket_backend.py`. It uses/builds on `WebAggCore` under the hood. + +Fundamentally, it is a relatively simple backend which publishes a websocket. This websocket can send or receive: +- Binary messages, send only. These are PNG-encoded frames which should be published by the frontend. +- String messages, send & receive. These are encoded using JSON. They can be used, for example: + * To notify the front-end that a new frame is available + * To allow the front-end to request a new frame + * To allow the front-end to resize a plot + * To allow the front-end to pan/zoom the plot, via mouse handlers. Note the front end simply sends mouse events to the backend, `WebAgg` then interprets these and redraws an appropriate plot. + +We have modified the backend to: +- Be non-blocking, so that scientists can update their plot and still continue running scripts in the background +- To notify the IBEX client on calling `plt.show()`, so that IBEX can show the plot windows. + + +## GUI (java) component + +`WebAgg` is published by matplotlib as a browser-based backend. IBEX previously rendered plots in an embedded web browser view within an OPI, but this had significant reliability problems. + +We have since moved to reimplementing the javascript based frontend code in java, as it is very simple. When matplotlib is updated, we should verify that plots still work and update the protocol in the GUI if the underlying protocol in `WebAgg` has changed or been extended. However this should be relatively rare as WebAgg is designed to be embedded in other applications (per [mpl docs](https://matplotlib.org/stable/gallery/user_interfaces/embedding_webagg_sgskip.html)). + +The front-end receives binary messages from the backend, and displays the binary message content as a PNG in the user interface. The user interface also occasionally forces plots to be redrawn, so that any missed updates will not cause a plot to completely freeze/fail to update. + +The user interface implements a buffered redraw mechanism to prevent very fast updates from the matplotlib backend from overwhelming the UI thread of the client. This is defined in `/uk.ac.stfc.isis.ibex.ui.graphing/src/uk/ac/stfc/isis/ibex/ui/graphing/websocketview/MatplotlibFigureViewModel.java` and currently limits the UI to drawing at 250ms intervals. **Forced** redraws happen at an interval of 2000ms, this helps us to "catch-up" if any updates have been missed and is required in order to get updates to some dynamic plots. \ No newline at end of file diff --git a/doc/scripting/genie_python/genie_python-Troubleshooting.md b/doc/scripting/genie_python/genie_python-Troubleshooting.md new file mode 100644 index 000000000..8720c710e --- /dev/null +++ b/doc/scripting/genie_python/genie_python-Troubleshooting.md @@ -0,0 +1,104 @@ +# `genie_python` Troubleshooting + +## Logging + +### Where are the logs? + +Genie_python writes its logs to `C:\Instrument\var\logs\genie_python`. + +### What does `ERROR: CAException` mean when it is in the log? + +The following is written to the log when a virtual circuit disconnects from the IOC side: + +``` + 2018-05-21T15:21:31 (CA) (15712) ERROR: CAException: type=-1 state=192 op=5 file=..\cac.cpp lineNo=1223 + 2018-05-21T15:21:31 (CA) (15712) ERROR: CAException: type=6 state=192 op=0 file=..\getCopy.cpp lineNo=92 +``` + +The time stamp on these are for the first `get_pv` call or equivalent after a disconnect. + +## Command problems + +### Can not set or get a block reports disconnected + +There can be multiple problems, check: + +1. Block exists +1. Block is spelt correctly, use `b.` and autocomplete +1. Try getting the underlying PV `g.get_pv("IN::CS:SB:", line 1204, in _gcd_import + File "", line 1176, in _find_and_load + File "", line 1147, in _find_and_load_unlocked + File "", line 690, in _load_unlocked + File "", line 940, in exec_module + File "", line 241, in _call_with_frames_removed + File "C:\Instrument\Apps\Python3\Lib\site-packages\setuptools_dso\__init__.py", line 10, in + from .dsocmd import DSO, Extension, install, build, build_dso, build_ext, bdist_egg + File "C:\Instrument\Apps\Python3\Lib\site-packages\setuptools_dso\dsocmd.py", line 729, in + _needs_builddso(_build, right_before='build_clib') + File "C:\Instrument\Apps\Python3\Lib\site-packages\setuptools_dso\dsocmd.py", line 715, in _needs_builddso + assert issubclass(command, Command) +``` + +When trying to import channel access libraries (`CaChannel`, `pcaspy`, `aioca`, `p4p` etc), this may be because `import pip` has previously been run in the same python session. See https://github.com/pypa/pip/issues/8761 for some details/links. `import pip` is not generally supported/recommended by python and is a bad idea - if wanting to install packages dynamically, use `subprocess` instead to explicitly spawn a new process which calls pip. + +It's also technically possible to replace `import pip` with: + +``` +import _distutils_hack +_distutils_hack.remove_shim() +import pip +``` + +But this is not recommended for obvious reasons. + +### Pyright completely fails to run + +Pyright keeps a cache directory in `c:\Users\\.cache\pyright-python`, this can get corrupted, if it does get corrupted pyright will entirely fail to execute. This cache directory can be deleted (at the cost of the next script-check operation being much slower). + +Error from `g.load_script` will be a `json.decoder.JSONDecodeError` as pyright does not return JSON in this case (but rather, returns some non-JSON error message). diff --git a/doc/scripting/genie_python/genie_python-linter.md b/doc/scripting/genie_python/genie_python-linter.md new file mode 100644 index 000000000..56f8d879d --- /dev/null +++ b/doc/scripting/genie_python/genie_python-linter.md @@ -0,0 +1,19 @@ +# `genie_python` script checking (linting) + +On `g.load_script` genie_python now runs pylint on the scripts. This help the user see errors before the script is run. However sometimes this causes its own issues. There is a [page in the user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Error-Checking-Troubleshooting) describing what the user should do. + +Here are some more tips that we don't necessarily want to encourage. + +Warnings can be removed from a line by adding a comment + +` # pylint disable=` + +e.g. `from IMAT_library import * # pylint: disable=wildcard-import, unused-wildcard-import` + + +### Linting dynamically defined variables +Python allows programmers to set attributes dynamically using expressions like `locals()['foo'] = 1`, which creates a local variable called `foo` with value `1`. Pylint doesn't support dynamic assignment, so if `foo` was subsequently referenced in the code it would be counted as an undefined variable. + +In cases where we need to lint scripts containing dynamic assignments, we can write a Pylint [transform plugin](http://pylint.pycqa.org/en/latest/how_tos/transform_plugins.html) to let Pylint know that the dynamically assigned variables are OK and should not be counted as undefined. This has been done [here](https://github.com/ISISComputingGroup/genie_python/blob/0a5f5093486e85e550b8168810e3d5cd762e34ff/Lib/site-packages/genie_python/scanning_instrument_pylint_plugin.py) to support dynamic assignments in the [SCANS library](https://github.com/ISISComputingGroup/IBEX/issues/5214), where some scripts were dynamically adding all methods of classes derived from `ScanningInstrument` to the local module. + + diff --git a/doc/scripting/instrument-scripts.md b/doc/scripting/instrument-scripts.md new file mode 100644 index 000000000..db411d58b --- /dev/null +++ b/doc/scripting/instrument-scripts.md @@ -0,0 +1,3 @@ +# Shared Instrument Scripts + +The general instrument scripts are in this [repository](https://github.com/ISISNeutronMuon/InstrumentScripts) and include scripts like `do_sans` and `do_trans`, as well as the (old) scans library. \ No newline at end of file diff --git a/doc/specific_iocs/Cells.md b/doc/specific_iocs/Cells.md new file mode 100644 index 000000000..da39ff960 --- /dev/null +++ b/doc/specific_iocs/Cells.md @@ -0,0 +1,9 @@ +# Cells + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +cells/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Choppers.md b/doc/specific_iocs/Choppers.md new file mode 100644 index 000000000..9349255fa --- /dev/null +++ b/doc/specific_iocs/Choppers.md @@ -0,0 +1,9 @@ +# Choppers + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +choppers/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Cryogenics.md b/doc/specific_iocs/Cryogenics.md new file mode 100644 index 000000000..578bd31cf --- /dev/null +++ b/doc/specific_iocs/Cryogenics.md @@ -0,0 +1,145 @@ +# Cryogenics + +## IOCs + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +cryogenics/* +``` + +```{seealso} +See also [Temperature controllers](Temperature-Controllers) for other temperature controllers which may be used in cryogenic setups. +``` + +## Cryostats (T>1K) + +### Closed Cycle Refrigerators (CCRs) +These are controlled using Eurotherms. No need to communicate directly with the CCR itself. + +### Orange Cryostats +These are controlled using Eurotherms. No need to communicate directly with the Cryostat itself. + +### Variox (Blue) Cryostats +These are manufactured by Oxford Instruments (OI). They can be controlled by either: + * Oxford Instruments ITC503 (See [#1389](https://github.com/ISISComputingGroup/IBEX/issues/1389), [#3189](https://github.com/ISISComputingGroup/IBEX/issues/3189)) + Oxford Instruments ILM200 (See [#1390](https://github.com/ISISComputingGroup/IBEX/issues/1390)) + +or + * Oxford Instruments Mercury ITC (with level meter expansion card) + +or + * Oxford Instruments Mercury ITC + standalone level meter such as an Oxford Instruments ILM200 + +### ISISStat + +Controlled by a Mercury ITC (See [#1199](https://github.com/ISISComputingGroup/IBEX/issues/1199)) + +Note: a "Mercury ITC" and an "ITC503" are two different devices (with different drivers). They are both manufactured by Oxford Instruments and control the temperature of a cryostat. The Mercury ITC is the newer model, the ITC503 is the older model. Mercury ITC controllers can take various expansion boards, which I do not believe ITC503 controllers are capable of. + +Our support for Mercury ITC units is not yet fully complete: +* See [#1286](https://github.com/ISISComputingGroup/IBEX/issues/1286) +* See [#1287](https://github.com/ISISComputingGroup/IBEX/issues/1287) +* See [#1392](https://github.com/ISISComputingGroup/IBEX/issues/1392) + +### Oxford Flow through Cryostats + +EMU use a flow cryostat controlled by an ITC503 temperature controller. There is no special setup of ITC503 required from a computing perspective. + +Other beamlines may use different setups, check with the scientists if unsure. + +## Low temperature inserts (T<1K) + +All of the inserts in this category are run in a "parasitic" fashion - i.e. they sit inside an outer cryostat which provides cooling to low temperature, and then the insert provides the final stage of cooling to very low temperature. + +The outer cryostat for these inserts can be either a blue cryostat (see above for details) or a cryomagnet (see below for details) + +### Mercury Heliox (He3 sorption refrigerator) + +This insert is capable of reaching temperatures of ~300mK. The H and the outer cryostat use separate physical controllers, which need separate moxa connections and EPICS drivers. + +For further details see [mercury Heliox](cryogenics/Mercury-Heliox) + +### Triton Dilution Fridge + +Triton systems are dilution refrigerators capable of reaching ~30mK. They are controlled by a dedicated PC which sits in a rack. Cryogenics group do a significant amount of setup with these controllers when they arrive on a beamline. + +See [Triton](cryogenics/Triton) for further details about the triton system + +### ICE dilution fridge. + +This dilution fridge is only used on muon beamlines - usually on MuSr, but occasionally borrowed by EMU. The RIKEN muon instruments (Argus/Chronus) may also use this fridge in the near future. This is currently the only dilution fridge at ISIS which is not controlled by an Oxford Instruments Triton controller. + +For details see [ICE dilution fridge](cryogenics/ICE-Dilution-Fridge) + +## Cryomagnets (cryogenically cooled superconducting magnets) + +Most of the magnets in this section can be used instead of a cryostat when using a "parasitic" cooling insert (e.g. Heliox, triton). This is because the magnets must be cooled to low temperatures anyway so that they superconduct, so they also act as the outer cryostat. + +### 7.5T magnet + +The 7.5T magnet is controlled by 3 devices: + * [Intelligent Power Supply (IPS)](cryogenics/Oxford-Instruments-IPS) (See [#1391](https://github.com/ISISComputingGroup/IBEX/issues/1391)). It could also be controlled by a [Mercury IPS](cryogenics/Oxford-Instruments-Mercury-IPS) but cryogenics group are not using this option at present due to hardware issues with the Mercury IPS. + * [Intelligent Temperature Controller (ITC503)](cryogenics/ITC-503) (See [#1389](https://github.com/ISISComputingGroup/IBEX/issues/1389), [#3189](https://github.com/ISISComputingGroup/IBEX/issues/3189)). A [Mercury ITC](cryogenics/MercuryiTC) may be used as a fallback if the ITC503 controller fails, but this is not usual. + * Intelligent Level Meter (ILM) (See [#1390](https://github.com/ISISComputingGroup/IBEX/issues/1390)) + +For details including configuration settings, see [Oxford Instruments IPS](cryogenics/Oxford-Instruments-IPS). + +See note at the bottom of this page for differences between IBEX and SECI in terms of how this device must be connected. + +### 9T "chopper" magnet + +This system appears to IBEX exactly like the 7.5T magnet discussed above, but with a different maximum field. It is currently dedicated to LET (but not permanently installed). + +Please see the documentation and notes linked above for further details. + +### 2T 3D Vector magnet + +This is primarily used on SANS and reflectometry beamlines. + +The 2T 3D vector magnet is manufactured by Scientific Magnetics. It has its own control system for which the manufacturer has provided LabVIEW VIs. We interface with these via lvDCOM (See [#1398](https://github.com/ISISComputingGroup/IBEX/issues/1398)). Detailed information [here](cryogenics/2T-3D-Vector-magnet). + +### 5T HIFI magnet + +This is a dedicated system permanently installed on HiFi. + +See [here](cryogenics/Cryogenic-SMS-PSU) and [here](/processes/instrument_details/HIFI_CRYOMAG-Instrument-Details) for details + +This magnet does *not* act as an outer cryostat for any other sample environment that may be used. + +### 17T "Birmingham" magnet + +This is a cryomagnet owned by the University of Birmingham, which occasionally travels to various scattering facilities, including ISIS. + +IBEX does not have support for this device at present. Ticket [#4523](https://github.com/ISISComputingGroup/IBEX/issues/4523) contains some details about this device. + +From a brief look at the command set and manual, it appears that all of the magnet control logic occurs in the Birmingham computer, and we simply send it ascii-formatted commands to go to a field. In theory this may be controllable via SDTEST (although a dedicated driver would of course be better). + +### 4T "E18" dilution fridge/magnet combination + +This system is a combination of a Oxford Instruments [triton dilution fridge](cryogenics/Triton) and a 4T cryomagnet driven by a [Mercury IPS](cryogenics/Oxford-Instruments-Mercury-IPS). Similar to the distinction between an old-style Oxford instruments ITC503 and a Mercury ITC, the Mercury IPS is the newer model. It is a physically different device to the old-style IPS, but performs the same task. + +For details of the Mercury IPS, see [Mercury IPS](cryogenics/Oxford-Instruments-Mercury-IPS). Some further details are also in [#4339](https://github.com/ISISComputingGroup/IBEX/issues/4339). + +## Level meters + +### Helium Level Gauge +Does what it says on the tin - measures the level of helium in a cryostat. + * support for Cryogenics Ltd He Level Gauge - See [#2350](https://github.com/ISISComputingGroup/IBEX/issues/2350) + +### Oxford Instruments Intelligent Level Meter (ILM200) + +This is another type of level meter, manufactured by Oxford instruments. It is often used in conjunction with ITC503 temperature controllers. It has two channels which can be set up to independently measure Helium and Nitrogen levels. + +Note: the ILM200 only physically measures the levels every 30 minutes, so if looking at the log plotter, there will be steps rather than a smooth downward curve - this is expected. + +See note at the bottom of this page for differences between IBEX and SECI in terms of how this device must be connected. + +## Note about Oxford Instruments ITC/ILM/IPS controllers + +These controllers used to be run in a rack with a single serial connection under SECI (this used an ISIS isobus expander crate to split the signals). This is because, in SECI, there was only one process which talked to all of the pieces of equipment (in various combinations). + +In IBEX, support for the individual devices has been written into separate IOCs to increase the flexibility of the drivers. Under Windows, this means that they cannot talk to the same COM port (because each IOC has exclusive access to the port). Therefore, under IBEX, separate connections from the moxa to each bit of kit are required. + +The kit takes 25-way serial cables. diff --git a/doc/specific_iocs/DAE-and-the-ICP.md b/doc/specific_iocs/DAE-and-the-ICP.md new file mode 100644 index 000000000..c4788e57f --- /dev/null +++ b/doc/specific_iocs/DAE-and-the-ICP.md @@ -0,0 +1,84 @@ +# DAE (Data Acquisition Electronics) + +The Data Acquisition Electronics (DAE) is the physical hardware that reads the neutron events out of the detectors. IBEX communicates with this hardware via the Instrument Control Program (ICP). This program is also responsible for combining the neutron and sample environment data into the NeXus file. + +## Running the DAE in Parallel + +In some cases you may want your DAE to mimic the state of a DAE on another machine. This is particularly useful when testing the DAE3 hardware as it will give comparable data files to the ones created by a DAE2. + +To do this you must: +* Switch to the dae3_pull_dae2_data [branch](https://github.com/ISISComputingGroup/EPICS-isisdae/tree/dae3_pull_dae2_data) +* Add `dbLoadRecords("$(ISISDAE)/db/dae3_pull.db","P=$(MYPVPREFIX),Q=DAE:,O=XXX")` to the st.cmd of your DAE, where XXX is the prefix of the instrument which you wish to mimic. +* Copy across the wiring, detector and spectra tables from the other instrument. +* Set your DAE with the same tables and the same general settings. +* Wait until the other instrument is in SETUP +* Start your DAE + +The DAE will mimic the run state, title and period of the DAE it is point to. + +{#dae_event_histogram_modes} +## Event and Histogram Mode +The DAE/ICP has two main 'modes' of operation, event mode and histogram mode. In event mode the position and timestamp of every single neutron that the detector finds is passed through the DAE to the ICP and saved into the NeXus file. In histogram mode the DAE will intercept these events and put them into bins based on when they hit the detector. Due to the binning of events histograms lead to some lost information however historically histogrammed data was the only option due to limited data rates. Instruments in general are moving more towards event data but it still has the disadvantage that it will lead to large and unwieldy NeXus files and in some cases offers little benefit as histogramming would be the first step of data analysis anyway. + +In reality this mode of operation is actually set per detector card rather than over the whole DAE. However instruments will tend to run with a majority one way or the other and so scientists tend to think of them as distinct modes. The exception to this is monitors where instruments will nearly always histogram them. This is for two reasons: + +* They have higher flux than other detectors so will dramatically increase file size if put as events +* They're used mainly for normalisation and diagnostics so loss of precision is not that much of a big deal + +{#dae_configuration} +## Configuring the DAE/ICP +There are two settings files inside `EPICS/ICP_Binaries` that are used to configure the ICP at start up, these are `icp_config.xml` and `isisicp.properties`. They contain information that is usually quite fixed for an instrument such as whether to start up in simulation mode, how much memory to use etc. **NOTE:** on in instrument these files live in a different location `c:\labview modules\dae`. The `isisicp.properties` file is used to override settings in `isisicp.default.properties`. + +There are three main files that can be set at runtime to change the behaviour of the DAE or used for later analysis, they are collectively known as tables and need to be selected from the "Experiment Setup/Data Acquisition" tab in the GUI: +* *Detector Table*: Files containing the word detector specifying the physical location of each detector. The second line contains the number of entries followed by the number of user parameters. The table consists of a detector id, it's offset, it's L2 (distance from the sample), an id code and then as many user specified parameters as specified in the second line. +* *Spectra Table*: Files containing the word spectra specifying how detector id (which has little meaning to an instrument scientist) maps to spectrum number (which the scientists use for analysis). The second line contains the number of detectors. +* *Wiring Table*: Files containing the word wiring and specifying how the detectors are connected to the DAE. The second line specifies how many detectors there are and how many of those are monitors. The table contains an index, detector id, time regime (see time channels below), physical crate, module and position numbers, the monitor that the detector corresponds to (0 for none) and the monitor prescale. +These files are picked up from `Instrument/Settings/config/inst/configurations/tables`, have the *.dat suffix and are all human readable. They can be edited by hand but must be done carefully to ensure that there are no logic errors within them. For example if a detector is listed in one file but not in another. + +### Time Channel Binning - setting to event/histogram mode + +The way that data is binned by the DAE is set by changing the time channel boundaries (TCB). Scientists generally want different amounts of detail depending on when the neutron hit the detector, which they can do by setting a smaller bin in the time range that they care more about. They can do this through the GUI manually or by creating a tcb file that is loaded into the ICP/ibex GUI. It may be that scientists would like different levels of binning for different banks of detectors they can do this by setting up a number of different `time regimes` and then specifying the regime they would like the detector to go into in the wiring table. + +Time regimes defined in the GUI are assigned an integer from 1 to 99. There is a special convention for the time regime number in a wiring table file. If the integer 1-99 is used, this means collect in histogram mode using binning created from that time regime. Specifying a time regime number of > 100 in a wiring table file is a convention for event mode - specifying YYXX will collect events using regime XX regime but create 'on the fly' histograms in YY time regime for quick spectra view. Normally you will see `1` for histogram mode and `102` for event mode in a wiring file, this mean histograms will use time regime 1 and event mode will use time regime 2 for collecting the data at high resolution but create histograms based on time regime 1 resolution for quick spectrum views + +By convention on instruments, wiring tables enabling event mode have end in `_event.dat`. histogram and event mode wiring tables for a new instrument/new DAE are normally created by Detector System Group as they will know the physical wiring layout. An existing histogram mode table can be converted to enable event mode by anybody by following the rules above. + +By default simulated data will head into one or two spectra, if you want to see simulated data across all banks e.g. for an areaDetector simulation then look for the following lines in the `isisicp.properties` file mentioned above in [Configuring the DAE/ICP](#configuring-the-daeicp) +``` +isisicp.simulation.neventssim = 50 +isisicp.simulation.spreadsimevents = false +``` +change `spreadsimevents` to `true`, you probably don't need to increase `neventssim` unless you had loaded a custom larger wiring table and spectra aren't accumulating data fast enough for you. + +## Performing arbitrary actions on run start/run end + +The ISISDAE ioc can be configured to process PVs just after starting and ending runs. It is easiest to configure this in `globals.txt` like the following example for the SANS2D fast shutter: + +``` +ISISDAE_01__POST_BEGIN_1=TE:NDW1799:FINS_VAC:SHUTTER:OPEN_IF_AUTO.PROC CA PP +ISISDAE_01__POST_RESUME_1=TE:NDW1799:FINS_VAC:SHUTTER:OPEN_IF_AUTO.PROC CA PP +ISISDAE_01__POST_END_1=TE:NDW1799:FINS_VAC:SHUTTER:CLOSE_IF_AUTO.PROC CA PP +ISISDAE_01__POST_ABORT_1=TE:NDW1799:FINS_VAC:SHUTTER:CLOSE_IF_AUTO.PROC CA PP +ISISDAE_01__POST_PAUSE_1=TE:NDW1799:FINS_VAC:SHUTTER:CLOSE_IF_AUTO.PROC CA PP +``` + +There are currently 4 macros available for each of `POST_BEGIN_x`, `POST_END_x`, `POST_ABORT_x`, `POST_PAUSE_x` and `POST_RESUME_x`. + +Note that if the PV you want to process is not in the ISISDAE ioc, you will need to specify the `.PROC` field explicitly for the link to work. + +## ISISDAE environment variables + +These were added for https://github.com/ISISComputingGroup/IBEX/issues/6560 but testing proved inconclusive, so may need to be revisited: + +* `ISISDAE_TIMER_PRIORITY` controls if the spectra read timer queue thread is same (0), higher (1) or lower (-1) priority than other scan threads, defaults to higher +* `ISISDAE_TIMER_SLEEP` how long to sleep after a spectra read to allow a thread yield (default: 1ms) + +## Further information + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +dae/* +``` diff --git a/doc/specific_iocs/DAQ.md b/doc/specific_iocs/DAQ.md new file mode 100644 index 000000000..eb6c38289 --- /dev/null +++ b/doc/specific_iocs/DAQ.md @@ -0,0 +1,94 @@ +# DAQ (NI cDAQ) + +## Individual devices + +```{toctree} +:glob: +:titlesonly: + +daq/* +``` + +## General + +If a DAQ unit is power cycled, it may need to be reconnected from the National Instruments "MAX" (measurement and automation explorer) software. To do this, open MAX, expand "network devices", right click the relevant device, and use the "self-test" option, then try reconnecting from IBEX. + +## Connecting to a cDAQ using DAQmx and NI-MAX +You will need: +- Latest version of NI-MAX +- DAQmx driver installed (see "Device support in NI-DAQmx" section of DAQmx readme for compatibility with cDAQ devices) + +To connect to the device from the instrument/your machine see the [National Instruments documentation](https://www.ni.com/getting-started/set-up-hardware/data-acquisition/compactdaq#Configuring%20NI-DAQmx%20for%20CompactDAQ%20Ethernet%20Chassis) + +You can also connect to a simulated device, see the instructions at http://www.ni.com/tutorial/3698/en/. + +Notes: +- Opening "Devices and Interfaces" in the sidebar typically takes several minutes on a computer connected to the RAL network. This is possibly because of the amount of NI hardware on the network. +- You will need to know the hostname of the cDAQ for the "Find Network NI-DAQmx Devices" dialogue + +## Configuring a DAQ IOC +The Asyn port for each channel for the daq is defined using the `DAQmxConfig` iocsh command. This is typically located in a st-daq.cmd file next to the device's st.cmd. The DAQmxBase documentation is here: https://github.com/ISISComputingGroup/EPICS-DAQmxBase/tree/master/documentation + +``` +DAQmxConfig ( Asyn_port_name, physical_channel_address, channel_number, data_type, options) +``` +- `Asyn_port_name` is name of the Asyn port to be created for this channel. +- `physical_channel_address` is the address of the channel on the device you wish to connect to. To obtain this find which input/output you wish to address on the DAQ card. This can be found on the pinout of the specific NI card by clicking the `Device Pinout` button in NI-MAX. For example, `cDAQ9181-1234MOD3/ai0` addresses the third card in chassis `cDAQ9181-1234`, and the second AI channel from that card. + +- `channel_number` should start from zero, this is used to address each channel within an Asyn port. +- `data_type` is the data type of the channel (AI, AO, BI, ...) +- `options` Some common options are described here. All options can be found in [the documentation for DAQmxBase](https://github.com/ISISComputingGroup/EPICS-DAQmxBase/tree/master/documentation): + - Read mode can be continuous (default), OneShot (get data when record processed), or MONSTER (see below) + - `F` is the frequency (speed) at which samples will be collected + - `N` is the number of samples DAQmxBase will collect and then return to the IOC. + +A sample call to DAQmxConfig looks like (taken from separator): +`DAQmxConfig("R0", "cDAQ9181-1234MOD3/ai1", 1, "AI","N=1000 F=1000")` + +This configures the third card of chassis `cDAQ9181-1234`, ai input 2. Data acquisition is continuous at a rate of 1000 samples/second collection. The sample size is 1000, so will return around once per second. + +To address this channel in a record, your input field will look like: +``` +field(INP, "@asyn(R0 1 5.0) DATA") +``` +This accesses channel 1 of port R0, with a 5.0 second timeout. + +## Design of the DAQmx Driver +The DAQmx driver was originally written by Diamond but is no longer in use by them, meaning it is effectively ours to maintain. It's written using the old C API for asyn rather than the newer C++ one. At it's core the driver uses a state machine for configuring and acquiring data and a messaging system for moving between states. The state machine for the configuration is roughly described by the diagram below: + +![DAQmx State Machine](DAQmx_state.png) + +There are a number of other states after the start state that actually do the data acquisition but these are simpler. The state machine is incredibly hard to reason about as there are two mechanisms to move between states: + +* At the beginning of the main loop a message is picked off the queue (if `ignoreMsg=0`). This message is then used to change the state. +* During the execution of a state the state for the next loop can be changed. This change is regularly overwritten later on in the state's execution or when the message handling is performed on the next loop. + +This means the diagram above is only a rough idea of what is going on. The driver is in dire need of a rewrite, see https://github.com/ISISComputingGroup/IBEX/issues/5386. + +## Data bottlenecking in the DAQmxBase driver (Monster mode) +When developing the zero field magnetometer IOC we found that there are significant overheads in the EPICS DAQmx driver when running it in one shot and continuous modes. In these modes the NI task used to take the data is started and stopped with every point/array of data requested (as applicable). This means that each point/array takes ~150 ms to acquire, around 100ms of this is from starting the task and ~50ms to actually take the data. + +We addressed this issue for the zero field magnetometer (which needed consecutive readings very close to each other) by running in 'monster' mode. In monster mode the NI data acquisition task is never closed, so the data can be captured at a much faster rate. However, if the requested data rate is too high this can cause a buffer overflow. For the zero field magnetometer on a developer's machine this occurred at ~100000 data points/second. + +To change the read mode of the DAQ to monster mode, add `MONSTER` to the options in its DAQmxConfig call, for example: + +`DAQmxConfig("R0", "cDAQ9181-1234MOD3/ai1", 1, "AI","MONSTER N=1000 F=1000")` + +Unlike the other two modes, monster mode needs a call to `DAQmxStart(portname)` at the end of the st.cmd file for the IOC, see the ZFMAGFLD st.cmd for a reference. + +## Common errors +The DAQmxBase driver used in IBEX can throw a number of errors to the log file. After resolving the issue, the IOC will need rebooting. + +``` +### DAQmx ERROR (CreateAI): Device cannot be accessed. Possible causes: +Device is no longer present in the system. +Device is not powered. +Device is powered, but was temporarily without power. +Device and/or chassis driver support may have been removed. +``` +If this happens immediately on IOC boot, check if the DAQ is connected in NI-MAX. Check the device is reserved in NI-MAX and can be reached over the network. + +``` +### DAQmx ERROR (ReadAnalogF64): The specified operation cannot be performed because a task is in the process of being aborted or a device is in the process of being removed from the system. Wait until the abort operation is complete and attempt to perform the operation again. +``` +This error usually occurs when the connection between the IOC and DAQ has been interrupted after the IOC has started running normally. Check that the machine running the IOC still has the device reservation and can connect to the DAQ. Restart the IOC. \ No newline at end of file diff --git a/images/DAQmx_state.png b/doc/specific_iocs/DAQmx_state.png similarity index 100% rename from images/DAQmx_state.png rename to doc/specific_iocs/DAQmx_state.png diff --git a/doc/specific_iocs/Fluxgates.md b/doc/specific_iocs/Fluxgates.md new file mode 100644 index 000000000..02d701936 --- /dev/null +++ b/doc/specific_iocs/Fluxgates.md @@ -0,0 +1,13 @@ +# Fluxgates & Magnetometers + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +fluxgates_magnetometers/* +``` + +```{seealso} +For information about zero-field systems, see [Magnets](Magnets) +``` diff --git a/doc/specific_iocs/Gas-And-Liquid-Handling-Systems.md b/doc/specific_iocs/Gas-And-Liquid-Handling-Systems.md new file mode 100644 index 000000000..53eca9b5a --- /dev/null +++ b/doc/specific_iocs/Gas-And-Liquid-Handling-Systems.md @@ -0,0 +1,11 @@ +# Gas and liquid handling systems + +Systems for moving gasses and liquids on the beam line + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 0 + +gas_liquid_handling/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Imaging-Cameras.md b/doc/specific_iocs/Imaging-Cameras.md new file mode 100644 index 000000000..b6bb4d75b --- /dev/null +++ b/doc/specific_iocs/Imaging-Cameras.md @@ -0,0 +1,36 @@ +# Imaging Cameras + +## IOCs + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +cameras/* +``` + +## Imaging Cameras + +These are often controlled via their own PCs running bespoke software. Running an IOC on the camera's control PC allows the instrument control machine to communicate with it. + +In general the IMAT instrument scripts create folders on the relevant remote PC for the data to be stored in (using the mapped drives) and then communicates with the IOCs on the remote PC to take data. The remote PCs are mapped to the following drives on the NDX: + +* M: `IMAT-MESSINA-DETECT` +* E: `IMAT-UCB` + +### Pixelman + +This has been run on the `IMAT-UCB` machine. + +[Pixelman](cameras/Pixelman) + +### Andor + +The Andor camera is run on the `IMAT-MESSINA-DETECT` machine. It uses an areaDetector IOC to communicate. + +## Troubleshooting + +If the script is run and it report `Folder does not exist` then this could be because the mapping to the drives (see above) is incorrect. + +On the Andor camera computer, there are two scripts on the desktop, start_epics and start_css which should be run in sequence (note the first will not finish but should sit waiting after a few seconds). If the start_epics fails with an `ERROR: VxD not loaded` it may be a problem with camera power or USB connection. Note the Andor camera PSU has two lights, both of which should be on. \ No newline at end of file diff --git a/doc/specific_iocs/Loading-Rigs.md b/doc/specific_iocs/Loading-Rigs.md new file mode 100644 index 000000000..477c622c5 --- /dev/null +++ b/doc/specific_iocs/Loading-Rigs.md @@ -0,0 +1,9 @@ +# Loading rigs + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +loading_rigs/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Magnets.md b/doc/specific_iocs/Magnets.md new file mode 100644 index 000000000..579a21aaa --- /dev/null +++ b/doc/specific_iocs/Magnets.md @@ -0,0 +1,15 @@ +# Magnets & Large Power Supplies + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +magnets/* +``` + +```{seealso} +For information about: +- Cryomagnets, see [cryogenics:magnets](Cryogenics) +- Other power supplies, see [power supplies](Power-Supplies) +``` diff --git a/doc/specific_iocs/Motors-Extensions.md b/doc/specific_iocs/Motors-Extensions.md new file mode 100644 index 000000000..7281b0dec --- /dev/null +++ b/doc/specific_iocs/Motors-Extensions.md @@ -0,0 +1,9 @@ +# Motor Extensions + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +motor_extensions/* +``` diff --git a/doc/specific_iocs/Motors.md b/doc/specific_iocs/Motors.md new file mode 100644 index 000000000..1a4367581 --- /dev/null +++ b/doc/specific_iocs/Motors.md @@ -0,0 +1,83 @@ +# Motors + +The basis of the motor IOCs at ISIS is the [motor record](https://www3.aps.anl.gov/bcda/synApps/motor/index.html), which aims to provide a generic interface to all motors regardless of make or model. The motor record is a single PV that contains a number of fields for things like motor speed, position, encoder readback etc. Note that this is very different from the way other IOCs would normally be set up, where each of these values would be its own PV. + +Naming conventions for motor PVs are also different in that all motors are of the form `%PVPREFIX%MOT:MTR0101` where the first number gives the controller and the second gives the axis. These controller/axis numbers are what dictates the axis position in the table of motors. + +For most motor IOCs the motor record itself is pulled in from `support\motor\master\motorApp\Db\motor.db`. However, the Galil contains its own copy of the motor record in `support\galil\master\GalilSup\Db`. + +## IOC Structure + +## Physical Hardware + +![Physical hardware](motor_hardware_structure.JPG) + +In general one motor controller will control a number of different axes (the actual motors themselves). Each axis will most likely give some feedback to the controller, the most common types of feedback are: +* Encoder: This is some form of external device that is measuring where the axis actually is. This is normally fed back to the controller in _steps_, the motor record will convert this to a distance using a supplied resolution. +* Limits: A binary signal that tells the controller that the motor has reached the edge of it's travel. There will be a high and a low limit for each direction of travel. +* Home: A binary signal that tells the controller when the axis is at a known _home_ position. + +## Software structure + +All motor IOCs have a macro called `MTRCTRL` which defines the controller number. The controller number is used to [create the PVs for each axis](/iocs/conventions/PV-Naming) and gives the row where each axis will appear on the table of motors. + +Motors will tend to have a subset of the following *.cmd files for loading dbs: + +| Filename | Description | Location | +| --- | --- | --- | +| `st.cmd` | Should be very simple, only loading the st-common.cmd | One in each iocBoot folder for each IOC | +| `st-common.cmd` | The main cmd for the IOC | A single version in the first IOC | +| `st-ctrl.cmd` | Creates IF macros for which controller we're loading | A single version in the first IOC | +| `st-max-axis.cmd` | Calculates the maximum axis we need to load | A single file in the first IOC | +| `st-axis.cmd` | Loads each individual axis | A single file in the first IOC | +| `st-motor.cmd` | Loads the actual db files for each individual axis | A single file in the first IOC | +| `galilXX.cmd` | Sets up the physical connection for a Galil | A file for each galil in `configurations\galil` | + +## Motors used at ISIS + +The motors currently used at ISIS are: + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +motors/Attocube +motors/Beckhoff +motors/EnginX-Sample-Positioner +motors/Galil +motors/Linmot +motors/McLennan-motors +motors/Newport +motors/PI-Rotation-Stage-setup +motors/SM300 +motors/SMC100 +motors/SXD-Attocube +``` + +```{seealso} +See [Motor Extensions](Motors-Extensions) for information about extra configuration which can be applied to motion controllers, for example jaws. +``` + +## Troubleshooting information + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +motors/Motors-Trouble-Shooting +motors/Adding-motor-IOC-to-global-motor-moving-and-stop-all +motors/Clearing-the-Motors-are-Moving-state +motors/Set-the-raw-position-of-the-motor-without-moving-it +motors/Add-support-for-motor-extras +``` + +## See also + +* [IMAT Lens Adjustment](motor_extensions/IMAT-Lens-Adjustment) +* [Motion Setpoints](motor_extensions/Motion-Set-points) +* [Creating soft motors to control real motors](motor_extensions/Creating-soft-motors-to-control-real-motors) This is useful if you need some logic for transformations between axes. +* [Portable Eulerian Cradle](motor_extensions/Portable-Eulerian-Cradle) see on ENGINX +* [SANS2D Front Beam Stop inhibit movement](motor_extensions/SANS2D-Front-Beam-Stop-inhibit-movement) +* [Disabling motor from view in the motor table](/iocs/testing/Disable-records) diff --git a/doc/specific_iocs/Multimeters.md b/doc/specific_iocs/Multimeters.md new file mode 100644 index 000000000..2af372d7e --- /dev/null +++ b/doc/specific_iocs/Multimeters.md @@ -0,0 +1,9 @@ +# Multimeters + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +multimeters/* +``` diff --git a/doc/specific_iocs/Other.md b/doc/specific_iocs/Other.md new file mode 100644 index 000000000..6c9584505 --- /dev/null +++ b/doc/specific_iocs/Other.md @@ -0,0 +1,9 @@ +# Other / Miscellaneous + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +other/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/PLCs.md b/doc/specific_iocs/PLCs.md new file mode 100644 index 000000000..3af9e6fdf --- /dev/null +++ b/doc/specific_iocs/PLCs.md @@ -0,0 +1,34 @@ +# PLCs + +## Introduction + +A PLC (programmable logic controller) is a device that is capable of doing near real-time control of a number of inputs and outputs. They're used throughout ISIS to do control that needs to happen very quickly or very reliably/safely e.g. if a vessel is under vacuum make sure it cannot move. In general IBEX doesn't need to worry about what logic the PLC is performing but just needs to read and write to various memory addresses on the device to make sure it performs the correct logic. + +## What is a PLC? + +PLCs, or programmable logic controllers, are specialised, ruggedized computers used in the control of manufacturing processes. They replaced previous electrical relays, which were large, heavy, unreliable and had to be physically rewired if there needed to be a change in the logic of the control process. Since PLCs are a type of computers, they can be reprogrammed by writing new software. + +PLCs are usually composed of a CPU, Input modules, output modules, communication equipment, and a power supply. Input modules are responsible for taking input from devices (such as sensors, etc), translating that signal to a digital signal that the CPU can understand , as well as removing electrical noise. The output modules do a similar job, but instead they handle the output signal coming from the PLC towards devices such as solenoid valves, on/off buttons, and others. They translate the binary output of the CPU to a digital or analogue signal that the output device can understand. Each input/output port or module enable the PLC to take input and give output to a wide array of specific devices that may use different communication protocols. The communication equipment is an interface that allows the computer to be controlled and reprogrammed by users. It can be as simple as a USB port that allows a regular laptop to connect to it. Through this communication equipment the PLC can also have remote input or output, that comes from a different place that at the point of use. + +All of these components are installed in a chassis or rack. There are two main types of PLCs in this regard: fixed and modular. Fixed PLCs are all-in-one machines that come with a fixed set of inputs and outputs. They are fairly small, can be installed easily, cheap and also include a small screen that can be used for manually programming the PLC. Modular PLCs are formed out of a series of connected modules that are bought separately. They are installed on a rack or clipped on a rail attached to a wall. A modular PLC has at least a CPU module and a power supply. The input and output modules can be added sideways and removed when not needed. Therefore, modular PLCs can be very flexible as you can configure their I/O to be exactly what you need. + +The computer software used to program PLCs is often proprietary and supplied by the manufacturer. The same is for the removable I/O modules. +Although the programming software is often proprietary, the programming language used usually follows one of five IEEE standards: ladder diagram, sequential function charts, function block diagram, instruction list, or structured text. Ladder diagrams are the most popular, as it is a graphical programming language based on ladder logic. This type of logic was used when designing electrical relays, and thus ladder diagrams are easy to use for electrical engineers. Structured text is a text based programming language, with syntax similar to C. Instruction list is a text based language that is similar to assembly programming languages. + +As mentioned above, a PLC is a type of computer that is specialised for controlling industrial processes. A PC can also be used for this. However, a PLC is specially designed for this type of job, has specialised modules from manufacturers with experience in this area, it does not have normal peripherals of a PC that are unnecessary for its job such as mice, keyboards, monitors, and is more rugged so that it can operate in environments with electrical noise, vibrations, etc. It is designed with more reliability on both a hardware and software level. PLCs have a minimal Operating System whose only job is to run the PLC code sequentially. PLC programming languages are more suited for applications in controls engineering, and more accessible for engineers in this area. + + +## Models + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +plcs/* +``` + +```{seealso} +A [Beckhoff](motors/Beckhoff) is also a generic PLC, though we usually treat it as a motion controller. +``` + diff --git a/doc/specific_iocs/Polarisers-and-Analysers.md b/doc/specific_iocs/Polarisers-and-Analysers.md new file mode 100644 index 000000000..6c2a0ae15 --- /dev/null +++ b/doc/specific_iocs/Polarisers-and-Analysers.md @@ -0,0 +1,9 @@ +# Polarisers & Analysers + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +polarisers_and_analysers/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Power-Distribution-Unit-(PDU).md b/doc/specific_iocs/Power-Distribution-Unit-(PDU).md new file mode 100644 index 000000000..8fdec1416 --- /dev/null +++ b/doc/specific_iocs/Power-Distribution-Unit-(PDU).md @@ -0,0 +1,22 @@ +# Power distribution units + +A PDU, or Power Distribution Unit, is a large power strip with multiple power outlets to supply electricity to a number of attached devices. They may or may not have surge protection. Advanced PDUs provide surge protection, real-time monitoring and remote access. + +### LINDY PDU + +The Lindy PDU has been implemented on IBEX GUI using an IOC that uses the SNMP interface to view, monitor and update outlets. There are 8 outlets on the strip and they can be individually turned on or off from IBEX. Lindy also comes with a web interface and there is also separate configuration software that can be downloaded if required from the web site, which would probably be needed to set a static IP address (however DHCP can be enabled via a button press - see below). To run on the IBEX the IP of the PDU is required to be configured on the IOC macros. +Further info [here](https://www.lindy.co.uk/power-c8/power-distribution-unit-pdu-c347) for user manual etc. The manual is also in `\Manuals\Lindy_ipower` + +### IP address + +As described in the manual, you can press the function button on the device to see the IP address or change its mode between dhcp/static +``` +Function Button: +a. Press and release this button to turn off the warning audible alarm. The overload alarm can’t be stopped by pressing this button. +b. Pressing the button and releasing it after 2 beeps will show up the unit’s IP address. +c. Pressing the button and releasing it after 4 beeps changes the IP address mode from fixed to DHCP and vice versa. +d. Pressing the button and releasing it after 6 beeps restarts the network interface. +``` + + + diff --git a/doc/specific_iocs/Power-Supplies.md b/doc/specific_iocs/Power-Supplies.md new file mode 100644 index 000000000..f860a15a4 --- /dev/null +++ b/doc/specific_iocs/Power-Supplies.md @@ -0,0 +1,15 @@ +# Power supplies + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +power_supplies/* +``` + +```{seealso} +- For large power supplies (e.g. driving magnets), see [Magnets](Magnets) +- For cryogenic power supplies (e.g. driving cryomagnets), see [cryogenics:magnets](Cryogenics) +- For multimeters which may also act as current sources, see [Multimeters](Multimeters) +``` diff --git a/doc/specific_iocs/Pressure-Monitors.md b/doc/specific_iocs/Pressure-Monitors.md new file mode 100644 index 000000000..174734a9d --- /dev/null +++ b/doc/specific_iocs/Pressure-Monitors.md @@ -0,0 +1,9 @@ +# Pressure monitors + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +pressure_monitors/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Reflectometry-IOC.md b/doc/specific_iocs/Reflectometry-IOC.md new file mode 100644 index 000000000..c73731974 --- /dev/null +++ b/doc/specific_iocs/Reflectometry-IOC.md @@ -0,0 +1,103 @@ +# Reflectometry + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +reflectometry/* +``` + +## High level Requirements + +As described in the physics background page ([Reflectometers Science](reflectometry/design_docs/Reflectometers-Science)), we need to calculate where a beamline should physically be based on higher level parameters, and track a changing beam path with multiple components. The user workflows we need to support are: + +1. Delayed move (the user wants to move one or multiple devices on the beamline and they want to confirm the entered value(s) before committing to the move): + 1. Set a parameter (or multiple) + 1. Ensure the value is correct by reading it + 1. Move to the set value – either: + 1. Trigger move for a single parameter – the new value is applied for this parameter, all others retain their current values. + 1. Trigger move for all parameters – any new values are applied for all parameters +1. Immediate move (the user knows the value is correct and just wants to move): + 1. Set a single parameter + 1. Beamline immediately moves parameter to that position, while keeping all other parameters at their current value +1. Readback values (so the user knows where the beamline actually is) + 1. When any component moves, the actual position of the parameters should be reported +1. Visual indicators (so the user has extra information on the state of parameters without thought, usually indicated on the GUI via a colour) + 1. If a parameter setpoint is changed but not moved to (default colour yellow) + 1. If an underlying motor of a parameter is moving (default colour green) + 1. If a setpoint readback and readback are different from each other and not moving (default colour red) + 1. If a setpoint and setpoint readback are different and the setpoint is not changed (default colour red) + 1. TBD - we are not sure, we don't just want to copy values up from lower components. +1. Synchronised moves: To avoid clash conditions it is desirable that most axes move together so that they reach their final destinations at the same time. This means setting the velocity of the axis before a move is started. There is no initial requirement to actively monitor the motors themselves to ensure continual crash avoidance, e.g. in the event of a motor stall. + - a caveat on this is that some axes should not be synchronised. Either because the motor velocities should not be changed (piezo motors) or they take a long time and have no clash conditions (e.g. multi-detector heights). + +## Design Details + +Reflectometry IOC sits on top of: +- **Motor Driver Layer:** This layer is responsible for communicating with the galil and other motors. This is complete. +- **Motor Group Layer:** This layer groups together motors from the motor driver layer into more natural groups. For instance a jawset, which allows a user to set jaw gaps and height, not just positions of individual blades. + +The reflectometry IOC is composed of layers, each layer talking to the layer below and above it: + +- **[Beamline Parameters](reflectometry/Reflectometry-Beamline-Parameters):** In this layer, the user is specifying where they want a component to be positioned in relation to the incoming beam. The user will specify theta, for instance, which will then set the position of the geometry component related to theta. They will also read whether the geometry component has changed and report a readback of the positions on the actual beamline. The beamline parameters are calculated in a strict order starting with those closest to the source to ensure the path is correct. Parameters can be disabled in which case they do not automatically track the beam. + +- **[Geometry Components](reflectometry/Reflectometry-Geometry-Components):** This layer calculates the beam path, and handles the conversion of positions between parameter values (relative to current beam path) and motor values (absolute room coordinates). The beamline parameters set the positions from above and the composite driving layers sets the parameters from below. + +- **[Composite Driving Layer](reflectometry/Reflectometry-Composite-Driving-Layer):** This layer pushes values from the parameters into the motor group and motor driver PVs. They also push the readbacks up to the geometry layer. This layer contains some logic to allow concurrent moves with very simple synchronization (complete all moves in time with the slowest axis). + +The whole system is coordinated by the [Beamline object](reflectometry/Reflectometry-Beamline-Object), which represents a complete model of the beamline from a motion point of view. This model is defined for each individual instrument by the [Reflectometry Configuration](reflectometry/Reflectometry-Configuration). + +## Set Point Propagation + +The values that the user sets on the reflectometry IOC only ever get transferred downward between the layers. This is true of setpoints and set-point readbacks. The only time this is violated is on start-up where the state is read from the underlying hardware for initial values. This means if a user sets a motor position it will not be reflected in the setpoint and the next time the reflectometry IOC is moved it will set it back to its original position even if it hasn't been changed. +Readbacks flow in the opposite direction they are always read from the lower levels and set upwards. To help identify where set points and readbacks are not the same indicators on the GUI will be set. +This is controversial but is signed off as per [ticket 4307](https://github.com/ISISComputingGroup/IBEX/issues/4307). + +## Server Status Feedback + +The reflectometry server provides feedback on its status to the user at 3 different levels: +- **Error Log:** At this level, we just collect error messages in the reflectometry server and expose them via a PV in addition to writing them to the log file. In order to do this, we should conventionally use `SERVER_STATUS_MANAGER.update_error_log()` in place of `logger.error()`. The PV with the list of errors is then displayed in a tab on the reflectometry front panel OPI. The log is stored in a 10,000 element char waveform and thus the string returned is truncated to that limit. +- **Problems:** These are slightly higher level and intended to be fairly general just as an indication of the area where a problem occurred. For more detailed information, the user should refer to the error log. Problems are comprised of a type of issue, a severity and a list of sources that are reporting it. Examples: + - Problem: Configuration is invalid; Severity: Major; Source(s): Configuration + - Problem: Velocity cannot be restored; Severity: Minor; Source(s): MTR0304, MTR0305 +- **Status:** This is the overall server status. The status is derived from the problems (the highest error level is the error level of the server) + +The intention of this system is to provide useful feedback to the user in case things go wrong **in addition to** handling exceptions programmatically in a sensible way as one would usually. + +The status, log and active problems get cleared every time a "move" is issued. This keeps the log PV size reasonable and the reported status up to date. E.g. if an error was thrown because of a disconnected motor IOC, and the motor has since been reconnected, the errors should be cleared on a successful move. If there are any persistent problems, they will just be thrown again. + +## Other Concepts + +{#reflectometry_footprint_calculator} +#### Footprint Calculator + +The footprint calculator calculates the resolution and footprint on the sample based on the slit gaps, distances between slits and sample length. This object is owned by the beamline. Currently, the footprint and resolution values are read-only. + +**For more implementation details see the [Beamline Configuration page](reflectometry/Reflectometry-Configuration)** + +#### Long Axis + +The long axis can be set like any other parameter to control movement of a component along the beam. It is currently set assuming the provided `IOCDriver` moves exactly parallel to the natural beam. Moving a component so far in the long axis that it reorders components in the beam is not currently supported. + +## Design Changes + +Potential design changes: + +1. Add component for slit gap parameter +1. Move displacement from movement strategy and place it in beam path calc + +## Specific Reflectometers + +Information pertaining to specific reflectometry setups on the instruments. + +- [CRISP](reflectometry/beamlines/Reflectomtery-IOC-CRISP) +- [INTER](reflectometry/beamlines/Reflectomtery-IOC-INTER) +- [POLREF](reflectometry/beamlines/Reflectomtery-IOC-POLREF) +- [SURF](reflectometry/beamlines/Reflectomtery-IOC-SURF) + + +## Troubleshooting + +see [Reflectometry Troubleshooting and FAQ](reflectometry/Reflectometry-Troubleshooting-and-FAQ) diff --git a/doc/specific_iocs/Rheometers.md b/doc/specific_iocs/Rheometers.md new file mode 100644 index 000000000..85e0b4a34 --- /dev/null +++ b/doc/specific_iocs/Rheometers.md @@ -0,0 +1,11 @@ +# Rheometers + +A rheometer is a laboratory device used to measure the way in which a liquid, suspension or slurry flows in response to applied forces. + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +rheometers/* +``` diff --git a/images/Rotating_sample_changer.png b/doc/specific_iocs/Rotating_sample_changer.png similarity index 100% rename from images/Rotating_sample_changer.png rename to doc/specific_iocs/Rotating_sample_changer.png diff --git a/doc/specific_iocs/Sample-Changers.md b/doc/specific_iocs/Sample-Changers.md new file mode 100644 index 000000000..8604ded62 --- /dev/null +++ b/doc/specific_iocs/Sample-Changers.md @@ -0,0 +1,15 @@ +# Sample Changers + +## Generic Sample Changers Support Module + +See [Sample Changers Support Module](motor_extensions/Sample-Changer-Support-Module) + +## Rotating Sample Changer + +There a couple of specific ISIS built rotating sample changers that are mostly used on GEM, POLARIS and HRPD. The version on HRPD has slightly different firmware but basically the same hardware. The hardware consists of a carousel that can hold 20 samples. When a new sample if requested the carousel rotates to the specified position and a sample arm will push the sample down into the beam. + +### Dropped samples + +Occasionally a sample can fall down below the beam, called a 'dropped' sample. If the device drops a sample it will tell you that it has done so as it moves on to the next sample. There is a retrieve command to collect the sample again, IBEX will attempt to retrieve the sample (as well as perform some other error checking) using the following state machine: + +![state machine](Rotating_sample_changer.png) \ No newline at end of file diff --git a/doc/specific_iocs/Sensors.md b/doc/specific_iocs/Sensors.md new file mode 100644 index 000000000..810b640ae --- /dev/null +++ b/doc/specific_iocs/Sensors.md @@ -0,0 +1,9 @@ +# Sensors + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +sensors/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Signal-Generators.md b/doc/specific_iocs/Signal-Generators.md new file mode 100644 index 000000000..e966ea6d5 --- /dev/null +++ b/doc/specific_iocs/Signal-Generators.md @@ -0,0 +1,9 @@ +# Signal Generators + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +signal_generators/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/Temperature-Controllers.md b/doc/specific_iocs/Temperature-Controllers.md new file mode 100644 index 000000000..0c34341cf --- /dev/null +++ b/doc/specific_iocs/Temperature-Controllers.md @@ -0,0 +1,13 @@ +# Temperature Controllers + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +temp_controllers/* +``` + +```{seealso} +See also [Cryogenics](Cryogenics) for low-temperature specific controllers. +``` diff --git a/doc/specific_iocs/cameras/Pixelman.md b/doc/specific_iocs/cameras/Pixelman.md new file mode 100644 index 000000000..85db2550c --- /dev/null +++ b/doc/specific_iocs/cameras/Pixelman.md @@ -0,0 +1,27 @@ +# Pixelman + +## IOC + +This IOC runs on a separate computer, **not on NDXIMAT**. + +The computer's name is `IMAT-UCB`, the password is listed on the passwords page. The PV Prefix is `MO:IMAT-MCP`. + +To launch Pixelman you have to run a special batch script in `C:\Instrument\Apps\EPICS\support\pixelman\master\run`. The file is `run_pixelman.bat` You should see some log entries in the camera software and the Pixelman GUI will start itself as it loads the EPICS plugins. If the .exe is just double-clicked the camera software will look like it loaded OK but not publish any PVs. If you get a message box suggesting you remove a line from a file, please click "NO" + +For more information see the README file in `C:\Instrument\Apps\EPICS\support\pixelman\master\run` + +## Setup + +The PC is set up and looked after by FIT and the instrument scientists. The person from Berkeley set up the camera software and network card; the scientists may have instructions about how to restart various components. + +We installed the IBEX server **32-bit** into `c:\instrument\apps\epics` using the normal installer but for 32-bit (we literally only installed the server, no git, client, SQL or config). The server does **not** run, we just use the two programs detailed in IOC section. + +To help with this we created two shortcuts on the desktop to start Pixelman and gateways as above. + +## Troubleshooting + +If you see errors on the IMAT screen from scripts that say things like connection failed to IMAT:...:SP: you may need to start the EPICS gateway. To do this type `C:\Instrument\Apps\EPICS\gateway\start_gateways.bat`. You may also need to restart the Pixelman software. + +It seems that if the contents of `ShutterValues.txt` are not liked by Pixelman, then the program will not start with EPICS support and you either get nothing or a Microsoft C++ error box. If you run `pixelman.exe` and dismiss the errors about not being able to load `epics.dll` then Pixelman will start, but you have no EPICS PVs. If you use the desktop shortcut `run_pixelman.bat` it fails as described. The EPICS DLL does not read `ShutterValues.txt` so I am unsure of the reason behind this, we do not have the source code for Pixelman so I cannot debug this. + +UPDATE: after investigation by camera developer, it looks like the maximum number of time slices is now lower than it used to be (now ~2900, previously 3100), which means previous working `ShutterValues.txt` files may fail. At the moment it is unclear what has caused this. diff --git a/doc/specific_iocs/cells/TiZr-Sample-Can.md b/doc/specific_iocs/cells/TiZr-Sample-Can.md new file mode 100644 index 000000000..a13a59a8a --- /dev/null +++ b/doc/specific_iocs/cells/TiZr-Sample-Can.md @@ -0,0 +1,21 @@ +# TiZr Sample Can + +The main project is one that sees the building and integration of a sample can such as was used in an experiment in the mid-1990s. The sample container is designed to contain steam at 580K and 95 bar. This will be used for an experiment on NIMROD. + +The assembly will consist of a stainless steel reservoir for the sample (either H20 or D20) which will then be evaporated into the TiZr cell. The pressure will generally be managed by the temperature. A manually controlled needle valve will be used during setup to limit the pressure should the reservoir be overfilled. A pressure gauge which outputs 0V to 10V will be used to report the pressure. + +From an IBEX perspective, the standard Triple Eurotherm on NIMROD will be used to read and control the temperature of the reservoir and the cell, with readings for the cell from both the top and bottom of the cell. Another Eurotherm crate will be brought in to read the temperature of the pipework after the cell and the pressure within the cell. + +In a meeting on the 20th of April 2020, no other data collection or recurring calculations were identified, which means that the IBEX setup is limited to configuration items, and as such, no tickets have been created at this time. + +In a meeting on the 19th of June 2020, an awareness of a need to reduce the impact of potential human error was realised and IBEX issue #5494 was created to handle that + +## TiZR cell inhibitor IOC + +Takes two PVs as macros (PV-A and PV-B). Each PV has a maximum value set on them (PV-A-Max, PV-B-Max) and PV-A has a safe value (all of these are defined as macros). Each of PV-A and PV-B are allowed to exceed their max individual values separately but not at the same time (i.e. the exclusive or is allowed: PV-A > PV-A-Max XOR PV-B > PV-B-Max). If both exceed their maximum values (i.e. PV-A > PV-A-Max and someone sets PV-B > PV-B-Max or vice versa) then the given safe value is written to PV-A. + +When the safe value is written the TIZRWARNING will be set into alarm for 5 seconds and then the alarm is cleared, unless, the pv is still out of range in which the alarm is persisted in which the safe value would be continuously written to the PV (this could occur if writing the safe value is rejected or the safe value > PV-A-Max). + +Currently, if PV-B > PV-B-Max and a value is written to PV-A where value > PV-A-Max, the value will be set to PV-A for a brief period and then the safe value will be written to PV-A again overwriting this. We could change this by setting the DRVH on PV-A whilst inhibiting based on the PV-B value. + +The support module is here https://github.com/ISISComputingGroup/EPICS-TiZr and the ioc at https://github.com/ISISComputingGroup/EPICS-ioc in the TIZR folder. \ No newline at end of file diff --git a/doc/specific_iocs/choppers/Astrium-Chopper.md b/doc/specific_iocs/choppers/Astrium-Chopper.md new file mode 100644 index 000000000..62d0ada42 --- /dev/null +++ b/doc/specific_iocs/choppers/Astrium-Chopper.md @@ -0,0 +1,48 @@ +# Astrium Chopper + +## These are installed on LET as Choppers 1 and 5 + +Note that these controllers are to be retired end of 2019. + +## Software Architecture ## + +The IOC for the Astrium choppers works similarly to that of the [Mk3](MK3-Chopper) in that it uses a .NET C# DLL to communicate with the servers running the chopper control programs. The architecture can be described as: + +![](Astrium.svg) + +The red part of this diagram is written in C++/CLI to communicate with the .NET AstriumComms.dll and builds to another dll called Astrium.dll, which lives in the support directory. The green part is the asyn driver that communicates with Astrium.dll. StreamDevice is then used to communicate with this driver so as to leverage the formatting abilities of protocol files. + +## Hardware quirks ## + +* Sending a calibrate when the device has already been calibrated causes the chopper to get into a state that must be physically restarted to fix. Disallowing this is implemented into the IOC. +* When a frequency is set the device multiplies it by 10 e.g. sending 1Hz to the device will cause the chopper to run at 10Hz +* There is a resonance that means the chopper cannot run at 180Hz, this is reflected in the control scripts on LET and in the IOC itself. +* There is a resume command in the C# dll. This does nothing. +* The frequency SP_RBV from the device always reads zero. +* Sending a frequency setpoint may cause the frequency setpoint to be "corrupted" (the chopper will report that it is phasing to one value, but actually phase to an entirely different value). This is mitigated in the driver by resending the phase any time the frequency is set. + +## Chopper 1 (NCS016) ## + +If the control program on the chopper PC is restarted for any reason, a reset will need to be performed before the chopper can be calibrated (which is also needed after a program restart). This is done with a physical toggle switch on the monitoring system crate in the top of the left-hand cabinet. (Fault LED 8 on the same card will be illuminated showing the error). + +This cabinet is normally locked and so an instrument scientist will need to be contacted to gain access. + +### Photograph of the actual crate + +![](Astrium_Monitoring_Crate.JPG) + +The toggle switch is labelled _quit failure_ and LED 8 can be seen to the right of the potentiometer labelled _adjust threshold_. + +## Chopper 5 (NCS017) ## + +It is believed that the reset is not required on this system. + +## Troubleshooting + +### Build Issues + +**We are now using Visual Studio 2019 which allows a target of "latest W10 sdk" as the version, so we no longer need to use a specific W10 for our builds. This error was seen briefly on mk3 build too. If you see it, post to technical on Teams.** + +If you're having trouble building the Astrium chopper with error messages relating to the Boost library, you need to update visual studio 2017 (15.9.21 works) as there is a compiler bug in earlier versions of VS. + +If you're getting `fatal error LNK1104: cannot open file 'MSCOREE.lib'` you need to add WindowsSDK 10.0.15063.xxx (or possibly ALL available Windows SDKs that Visual studio has available, but try the one mentioned first). Run the Visual Studio Installer, Select modify, and scroll through the sidebar and/or the 'Individual Components' tab looking for Windows SDK versions. I (Liam, with Tom's help) ended up installing 50GB+ of extra packages to get this to work. If you solve the issue in a more streamlined way (i.e. _only_ installing the above version of the SDK, rather than everything) _please_ update this troubleshooting tip. \ No newline at end of file diff --git a/images/Astrium.svg b/doc/specific_iocs/choppers/Astrium.svg similarity index 100% rename from images/Astrium.svg rename to doc/specific_iocs/choppers/Astrium.svg diff --git a/images/Astrium.xml b/doc/specific_iocs/choppers/Astrium.xml similarity index 100% rename from images/Astrium.xml rename to doc/specific_iocs/choppers/Astrium.xml diff --git a/images/Astrium Monitoring Crate.JPG b/doc/specific_iocs/choppers/Astrium_Monitoring_Crate.JPG similarity index 100% rename from images/Astrium Monitoring Crate.JPG rename to doc/specific_iocs/choppers/Astrium_Monitoring_Crate.JPG diff --git a/doc/specific_iocs/choppers/FZJ_DD_Fermi_chopper.md b/doc/specific_iocs/choppers/FZJ_DD_Fermi_chopper.md new file mode 100644 index 000000000..1504200ef --- /dev/null +++ b/doc/specific_iocs/choppers/FZJ_DD_Fermi_chopper.md @@ -0,0 +1,16 @@ +# FZJ Digital Drive Fermi Chopper + +IOC name: `FZJDDFCH`. + +Prototype digital drive Fermi chopper controller made by Forschungszentrum Jülich (FZJ). A single "one-off" device unlikely to be used anywhere other than on MAPS. + +## Gotchas +- Only one TCP connection allowed at a time. Other TCP connections will appear to connect but not give any responses (this can cause the IOC to read "zero" for everything). +- Message length limit at 19 chars. For some commands (e.g. setting phase setpoint) this means the device cannot handle more than 1 d.p. If the message is too long the device will respond with an error message and not accept the setpoint at all. This length limit *includes* the `\r\n` terminator so really the message length limit is 17 characters. +- The manual does not match the command set. For example, the order of responses in the long response packet is incorrect and several of the commands simply don't exist. The best (most up-to-date) place to get the command syntax from is the Jülich VIs which are running on the chopper control machine (David has a copy of these). +- Be careful about renaming/removing PVs, this IOC is used by a CaLAB wrapper and that will need to be updated if the PV names change. +- Errors on all PVs are propagated from a single source so may not accurately reflect which command(s) have gone wrong + +## Connection + +Connected to a computer on the network which is looked after by the chopper group. If this computer is not pingable, contact the chopper group to restart it (the chopper needs to be stopped before restarting the control PC). \ No newline at end of file diff --git a/doc/specific_iocs/choppers/FZJ_Fermi_chopper.md b/doc/specific_iocs/choppers/FZJ_Fermi_chopper.md new file mode 100644 index 000000000..87bf90871 --- /dev/null +++ b/doc/specific_iocs/choppers/FZJ_Fermi_chopper.md @@ -0,0 +1,5 @@ +# FZJ Fermi Chopper + +Used on MERLIN, MAPS, MARI. + +This material is now in the [user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/FZJ-Fermi-Chopper). \ No newline at end of file diff --git a/doc/specific_iocs/choppers/MK3-Chopper.md b/doc/specific_iocs/choppers/MK3-Chopper.md new file mode 100644 index 000000000..7afba2d9f --- /dev/null +++ b/doc/specific_iocs/choppers/MK3-Chopper.md @@ -0,0 +1,17 @@ +# MK3 Chopper + +The MK3 Chopper is a standard chopper and used by multiple beamlines. The chopper is run from a system pc which is running server software and is connected to use .Net remoting. The code on the server is controlled by the electronic systems design group. + +To connect to the server the IOC instantiates a remote object of type IBeamLine and uses this to interrogate the server. This is held in `MK3ChopperSkeleton.dll` held in the support directory. To set up the remote a configuration file is needed and this is currently held in `c:\LabVIEW Modules\Drivers\ISIS MK3 Disc Chopper\MK3_Chopper.config`. There is a test server which can be found in the passwords file along with an instrument setting. + +The IOC connects from the driver to the Mk3BridgeLib.dll (in support) using a c interface. The Mk3BridgeLib uses the Chopper C# object which uses the skeleton lib. + +## User Control + +The user can only start/stop/park and unpark the mk3chopper using the panel on the chopper itself. It also has to be put into remote mode before any commands can be sent to it. [This is limited by the hardware control but may be expanded by the chopper group at some point. Email has been sent to them Sep 2019](https://github.com/ISISComputingGroup/IBEX/issues/4389) + +## Troubleshooting + +### User cannot set Frequency + +When trying to set a frequency if the frequency readback does not change start by looking at the log file. If the file read `Current User Demand Delay Too Large for Angle , Direction & Speed` it means that the phase for the chopper is too large for the current frequency. I believe that the phase must be smaller than `1/frequency` e.g. for the instrument running at 50Hz it needs to be smaller than 20,000us. diff --git a/doc/specific_iocs/choppers/SKF-MB350g3-Chopper.md b/doc/specific_iocs/choppers/SKF-MB350g3-Chopper.md new file mode 100644 index 000000000..ff5244cda --- /dev/null +++ b/doc/specific_iocs/choppers/SKF-MB350g3-Chopper.md @@ -0,0 +1,80 @@ +# SKF MB350g3 Chopper + +The SKF MB350g3 chopper is a fermi chopper located on the EMMA beamline and was previously used on the LET beamline. The chopper is mounted on a lifting mechanism detailed [here](../motor_extensions/Fermi-Chopper-Lifter). + +It's possible that the controller may need to be reset (press `RESET` button on front panel) after power-on or other condition. The display will show `INHIBIT` in this case and `READY` after resetting. The indicator marked `READY` on the OPI will be initially unlit and then lit afterwards. + +## Physical setup + +Original configuration: +NDXLET <---Ethernet---> AnyWhereUSB <---USB/RS232---> SeaLevel RS232-RS485 converter <---RS485---> MB350PC + + +New configuration: +NDXLET <---Ethernet---> MOXA NPort <---RS232---> Patton RS232-RS485 converter <---RS485---> MB350PC + + +Notes: +Originally, the passive RS232-RS485 converter could not be made to work correctly despite much effort, so an alternative, more convoluted solution was adopted in the hope that the original design could be reinstated at a later date. This alternative solution involved a "SeaLevel SeaLink USB serial adapter" (RS232-RS485 converter) connected to a "Digi USBAnywhere" (remote USB hub over Ethernet). + +This worked well for many cycles, however, the USBAnywhere box suddenly developed a regular problem in that it appeared to lose connection to the chopper controllers every couple of days. Suspected causes are a network scan (either for security or inventory) or a driver issue for the USBAnywhere or SeaCOM converter. The remedy was to reboot the USBAnywhere box either using the utility on NDXLET or by power-cycling the unit. + +New solution follows the original design to use a passive converter (Patton 2085) in circuit with the serial line between the MOXA and chopper controller. This removes the dependency on additional Windows drivers for extra hardware (AnyWhere USB & SeaLink) and simplifies the cabling as a MOXA is already present. + + +Switch settings for Patton 2085 Converter: + +DCE / DTE switch: DCE selected + +DIP switches: (All off originally, but this configuration seems to work also) +1: Off +2: Off +3: On +4: Off +5: On +6: Off +7: Off +8: Off + + +DB9M Pin to 2085 terminal connections: +1 -> RCV- +2 -> RCV+ +3 -> XMT+ +4 -> XMT- +5 -> G + + +## Software setup + +From a software setup point of view this can be treated as a standard serial device. It needs a COM port and all the usual COM settings to talk to (Baud, parity etc.). From testing on EMMA, the defaults in st.cmd are ok. + + +## Software + +The driver for this chopper has a couple of peculiarities: + +### Serial breaks + +The device requires the changes in https://github.com/ISISComputingGroup/EPICS-asyn/pull/6 and the following lines in its st.cmd: + +``` +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "break_duration", "20") +$(IFNOTDEVSIM) $(IFNOTRECSIM) asynSetOption("L0", -1, "break_delay", "20") +``` + +These lines tell it to sleep for 20ms and then send a serial break for 20ms after every message. If these items are not done then the chopper will not respond to messages. + +### Custom checksum + +The device implements a custom checksum algorithm which has been implemented in https://github.com/ISISComputingGroup/EPICS-StreamDevice/pull/3 and also in the lewis emulator + + +### Packet format + +The device sends and receives a modbus-like (but not actually modbus) packet. This is a packet of binary data which contains various data from the chopper + + +### Addressing + +The chopper can be set up on a multi-drop setup. The first byte of every packet to/from the chopper contains the address of the chopper which is an integer between 0 and 15. diff --git a/doc/specific_iocs/choppers/SKF-MB4150g5-Disk-Chopper-Controllers.md b/doc/specific_iocs/choppers/SKF-MB4150g5-Disk-Chopper-Controllers.md new file mode 100644 index 000000000..354e7327a --- /dev/null +++ b/doc/specific_iocs/choppers/SKF-MB4150g5-Disk-Chopper-Controllers.md @@ -0,0 +1,40 @@ +# SKF MB4150g5 Chopper + +These controllers are a development of the [MB350g3](SKF-MB350g3-Chopper) previously used on LET for its disk choppers and now repurposed on EMMA for its Fermi. The G5 is an Ethernet device using a MODBUS protocol to communicate, and are generally installed on the instrument private networks. Spare/test controllers may exist on the ISIS network. + +The IOC for this device is the **SKFCHOPPER**, this IOC has to have macros set in the globals.txt + +## Current Installations + +* **Six** controllers on LET (original single-disk choppers 3 & 4, and the previously Astrium double-disk choppers 1 & 5. Chopper 2 is an [ISIS MK3](MK3-Chopper)) +* **Five** on IMAT (two double-disk choppers and a T0) +* **Two** on SANDALS (a single double-disk chopper) + +## Addressing Schema on the instrument's private network + +LET - These controllers have been allocated _static_ addresses in the range _xxx.xxx.xxx.8x_, which is under the control of the Experiment Controls Group. Since installation, the addresses have been in the original range of xxx.xxx.xxx.18x, but problems were encountered when installing two additional controllers (see ticket [5033](https://github.com/ISISComputingGroup/IBEX/issues/5033) for details). + +IMAT - These controllers are still in the original range of _xxx.xxx.xxx.18x_, but it is planned to reallocate them for consistency. + +SANDALS - Also still in the original range of _xxx.xxx.xxx.18x_. + +**N.B.** The address ranges and subnets on the private network haven't formally been divided between the various groups (Experiment Control, Electronics, Detectors) to date. A coordinated effort is required to produce a document, ideally before any further use of the private network is made. + +## Communications + +The controllers use a MODBUS protocol to communicate over Ethernet. Their IOCs use a small subset of the complete command set to provide the minimum functionality required for remote control by the scientists. Due to differences in parameter definitions between SKF and ISIS, some interpretation (and trial and error) was needed to produce this reduced set of commands. + +The MODBUS TCP protocol is used, which slightly differs to MODBUS RTU used by Eurotherms and other serial devices. +This means there are some extra fields in the header of MODBUS requests/responses, as well as MODBUS TCP using a `crc16` or similar checksum. +A MODBUS TCP packet consists of an MBAP (Modbus Application) Header and a PDU (the same as MODBUS RTU) +The MBAP consists of: + +Transaction Identifier: 2 bytes +Protocol Identifier: 2 bytes +Length: 2 bytes +Unit Identifier: 1 byte + +https://ipc2u.com/articles/knowledge-base/detailed-description-of-the-modbus-tcp-protocol-with-command-examples/ shows some examples of a MODBUS TCP request compared against MODBUS RTU. + +Some of the SKF chopper controllers at ISIS implement the transaction identifier part of the MBAP header incorrectly. We have catered for this in the MODBUS interpose framework, skipping the check for this ID can be done by setting the `SKIP_TRANSACTION_ID` macro in globals to `1`. + diff --git a/doc/specific_iocs/cryogenics/2T-3D-Vector-magnet.md b/doc/specific_iocs/cryogenics/2T-3D-Vector-magnet.md new file mode 100644 index 000000000..36e9f3e7a --- /dev/null +++ b/doc/specific_iocs/cryogenics/2T-3D-Vector-magnet.md @@ -0,0 +1,67 @@ +# Scientific Magnetics 2T 3D Vector Magnet + +The system comprises: +* Temperature controllers (optional Lakeshore 336) +* Temperature monitor (Cryocon 14, Ethernet device) +* 3 power supplies (one for each axis) +* Helium level gauge +* Rotation stage (controlled separately by a Galil axis) +* Motorised needle valve (controlled by Galil analogue output, or Thurlby-Thandar EX355P benchtop PSU) + +### Large log files + +See [here](/system_components/labview/LabVIEW-errors) for mitigations - this is a known LabVIEW issue/bug. + +### VI + +There is a manufacturer supplied VI and our own LVDCOM IOC. There are two versions of the manufacturer's VI and both are stored in SourceSafe at the following location: `\LabVIEW Modules\Drivers\Scientific Instruments\3D Magnet\Source Code\`. One VI has been used on LARMOR and ZOOM and uses the TTi PSU for needle valve control, and the other on POLREF when an analogue output from the Galil was used. See below for more information. + +The settings for the manufacturer VI are stored (and can be edited) in the plain text file: `settings.txt`. The COM ports are defined in the first section of the file, but it shouldn't be necessary to change these. + +### Ethernet Switch + +The magnet control rack has a small Ethernet switch inside to provide network access to the Lakeshore 336 and MOXA (see below). This will need to be connected to the ISIS network to enable remote control of the magnet. + +### MOXA NPort + +The rack has its own dedicated MOXA NPort 5616 due to the number of devices it uses. When the system is moved between instruments, this MOXA is added to the NPort configuration on the local control machine and its ports assigned to COM101-COM116 (so as not to interfere with common equipment). The control machine IP address should be added to the "accessible IPs list" of the MOXA via the administration webpage (and subsequently removed when the magnet is finished with). Currently, the settings of the dedicated MOXA are: + +IP address : DHCP reserved and visible on front screen when connected to ISIS network + +* Port 1 / COM101 : SMC X PSU +* Port 2 / COM102 : SMC Y PSU +* Port 3 / COM103 : SMC Z PSU +* Port 4 / COM104 : Cryocon 32 temperature controller +* Port 5 / COM105 : Helium level meter +* Port 6 / COM106 : TTi EX355P PSU (when required for needle valve control) + +### Rotation Stage + +Standard unit controlled by a Galil axis on both SECI and IBEX instruments. Details: + +* +- 370 degrees +* Motor with encoder +* **_No_** limit switches - Do **not** rotate more than 360 degrees too many times or cables will be damaged. +* Home position defined by an index pulse on the encoder + +When this magnet was last installed on ZOOM, one of the slit axes was used for the rotation stage as there are no spare cables in the sample area. A batch file exists in the `configurations\galil` directory of the `settings` area to set the required PV values for the axis (edit if different axis is used), and a corresponding batch file in the `user\users\ZOOM` directory of the 'U-drive' will reinstate the settings for the slit axis when the magnet rotation stage is removed. + +**NB** There are plans to install additional cabling into the ZOOM sample area to connect extra motion stages, so this _borrowing_ of an axis may not be needed any longer. Check before altering PV values. + +### Needle Valve + +This is a motorised valve which is controlled via an analogue output from a Galil on SECI instruments (POLREF at least) and a Thurlby-Thandar Instruments EX355P benchtop PSU on IBEX instruments (LARMOR and ZOOM at least). There are two front panel VIs for this purpose, one for each scenario: `Vector-control-v16isis.vi` for the TTi-EX355P controlled valve and `Vector-control-v16isis-Galil-Valve.vi` for analogue output controlled valve. Both are in the SourceSafe location mentioned above: `\LabVIEW Modules\Drivers\Scientific Instruments\3D Magnet\Source Code\`. + +If the PSU is to be used, first it will need to be configured via its own setup VI located in `\LabVIEW Modules\Drivers\Thurlby EX355P\TH EX355P - System Functions.llb\TH EX355P - Setup Dialog.vi`, with the port from the `settings.txt` file mentioned above. It will then be possible to check the communication to the PSU via its own front panel VI here: `\LabVIEW Modules\Drivers\Thurlby EX355P\TH EX355P - System Functions.llb\TH EX355P - Front Panel 1.vi`. This VI and the main magnet VI cannot be run simultaneously. + +### Lakeshore 336 Temperature Controller + +The Lakeshore 336 is an Ethernet device and should therefore be connected to the switch in the magnet control rack. Its hostname can be found in a settings menu accessed from the front panel. Communications can be checked by pinging the hostname, or more thoroughly by configuring its VI or IOC/OPI. + +### Miscellaneous Information + +* The system takes approximately 3 days to reach base temperature + +### Schematic of system + +![Schematic of system](3d_vector_magnet_schematic.png) \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/3HeFridgeOps.png b/doc/specific_iocs/cryogenics/3HeFridgeOps.png new file mode 100644 index 000000000..75c22463e Binary files /dev/null and b/doc/specific_iocs/cryogenics/3HeFridgeOps.png differ diff --git a/images/Scientific Magnetics 3D Vector Magnet System.png b/doc/specific_iocs/cryogenics/3d_vector_magnet_schematic.png similarity index 100% rename from images/Scientific Magnetics 3D Vector Magnet System.png rename to doc/specific_iocs/cryogenics/3d_vector_magnet_schematic.png diff --git a/doc/specific_iocs/cryogenics/CP2800-Compressors.md b/doc/specific_iocs/cryogenics/CP2800-Compressors.md new file mode 100644 index 000000000..29c746526 --- /dev/null +++ b/doc/specific_iocs/cryogenics/CP2800-Compressors.md @@ -0,0 +1,32 @@ +# CP2800 Compressors + +There are 2 independent CP2800 Compressors on HIFI_CRYOMAG. These provide high pressure helium gas to 2 independent pulse tube cold heads via a valve which allows high pressure gas in and low pressure gas out. The valve is driven by the control circuits in the compressors. +Each cold head has 2 stages, with the hot end of stage 2 linked to the cold end of stage 1. The 1st stages of the cold heads get to around 40K and are used to cool the thermal shields surrounding the magnet coils. The 2nd stages cool the coils to around 4K, just below the superconducting threshold. + +### Command data packet + +This gist is that a command looking something like ` ` is sent. +- `STX` is the start of text character, or hex 0x02. This serves to sync the receiver to receive a new message, and purges any data received since the previous `STX` or carriage return was received. +- `ADR` is the address field, which is a macro in the proto file. This is a one byte field that can range from `0x10` to `0xFE`. +- `CMD_RSP` is the command response field. Essentially this tells the protocol and the device which type of message is being received/sent, a command or a response status. +- `DATA` is the optional data. `STX` and carriage returns cannot appear in the data field. to get around this, the format converter `%!` is used. See below for more info. +- `CKSUM1,2` are checksum characters for the message. +- `CR` is a carriage return. + +The reference manual for the communications is the 'data dictionary' on the CP2800 manuals share. + +### SMDP Converter + +CP2800 communications use a format converter `%!`, used to remove escape characters from commands sent to the device. Details of this can be found in the CP2800 manuals folder in the controls group manuals share in the file `Sycon Multi Drop Protocol II`. + +### Error codes +The machine can send the following error codes: + +|Code|Error | +|----|---------------------------------------------------------------------------------------------------| +| 0 | No Error | +| 1 |IC2 bus failure. Cycle power if this occurs. If this persists, contact factory. | +| 2 |5V power too high - occurs if power is above 5.25V. Contact factory | +| 3 |5V power too high - occurs if power is below 4.75V. Contact factory | +| 4 |Compressor is in lockout mode due to too many errors in a certain time. Cannot restart until a code is entered. Contact factory | +| 5 |Order of phase power is wrong. Rewire the compressor input by switching around 2 of the power wires| \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/Cryogenic-SMS-PSU.md b/doc/specific_iocs/cryogenics/Cryogenic-SMS-PSU.md new file mode 100644 index 000000000..101b0dea8 --- /dev/null +++ b/doc/specific_iocs/cryogenics/Cryogenic-SMS-PSU.md @@ -0,0 +1,31 @@ +# Cryosms PSU + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +cryosms/* +``` + +```{seealso} +See also [design notes](cryosms/Cryogenic-SMS-PSU-design) for background on the implementation of this IOC. +``` + +It was originally designed to be used for the HIFI main instrument cryomagnet, but has also been used to control the Oxford Instruments 9T magnet. + +## Troubleshooting + +### Driver gets stuck in `Processing` at a ramp rate transition, at high field + +Firstly, check which ramp file is in use and check at which fields the ramp rate transitions to verify the driver is stuck at one of these transition points. + +The lowest allowable ramp rate in the power supply is `0.0008 A/s`. Other allowable ramp rates are discrete values - when a ramp rate is sent to the power supply, it will select the next lowest possible ramp rate. Unfortunately, the absolute lowest ramp rate (`0.0008 A/s`) seems to be broken on the hardware, which reports a rate of `2^30` in this case. Therefore, the lowest ramp rate in the ramp table must be higher than the **second** lowest ramp rate allowable by the hardware. + +**Always check with cryogenics before increasing any ramp rates, as it is possible to cause a magnet quench by selecting a too-high ramp rate.** + +### PSU COM Ports disappearing + +Symptom: The COM ports for the Cryogenic Inc. Power Supplies are missing from the Device Manager (these are inbuilt USB to serial adapters within the devices, they install a driver which allows them to appear as COM ports) +Solution: Turn the PSU off at the main switch, unplug the USB cable from the PSU, unplug any connections via hubs and their power as well, reconnect everything and power everything on, and the COM ports should be available again diff --git a/doc/specific_iocs/cryogenics/Cryogenics-Ltd-Helium-Level-Gauge.md b/doc/specific_iocs/cryogenics/Cryogenics-Ltd-Helium-Level-Gauge.md new file mode 100644 index 000000000..a158cc148 --- /dev/null +++ b/doc/specific_iocs/cryogenics/Cryogenics-Ltd-Helium-Level-Gauge.md @@ -0,0 +1,4 @@ +# Helium Level Gauge (Cryogenics Ltd) + +Helium level Gauge, which attaches to a single helium level probe on circuit A and reads out the level. Has configurable alarm limits for the levels. + diff --git a/doc/specific_iocs/cryogenics/HLM-General.md b/doc/specific_iocs/cryogenics/HLM-General.md new file mode 100644 index 000000000..e768ba2f7 --- /dev/null +++ b/doc/specific_iocs/cryogenics/HLM-General.md @@ -0,0 +1,43 @@ +# Helium Level Monitoring + +```{note} +This is not a typical EPICS IOC. + +For details & background about the HLM Project as a whole, see [here](https://github.com/ISISNeutronMuon/HLM_PV_Import/wiki/Intro-to-the-HLM-Project). +``` + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +hlm/* +``` + +**HLM PV Import** - service and manager for updating the [HLM Database](https://github.com/SampleEnvironment/He-Management/wiki#helium-level-monitoring-database) with PV data from the [Helium Recovery PLC](../plcs/Helium-Recovery-PLC) ([Omron FINS](../plcs/Omron-FINS)). + +**HLM View** - data display website running with Django on the HLM server machine (for more info check [deployment notes](hlm/HLM-Web-Server-deployment)) + + + +### Links +* [HLM PV Import Wiki](https://github.com/ISISNeutronMuon/HLM_PV_Import/wiki) +* [HLM GAM Database](https://github.com/SampleEnvironment/He-Management/wiki#helium-level-monitoring-database) - the gas management database PV values are being imported into as object measurements +* [Helium Recovery PLC](../plcs/Helium-Recovery-PLC) - [FINS PLC](../plcs/Omron-FINS) used for monitoring various parameters related to the helium gas recovery system +* [HLM Project Sharepoint](http://www.facilities.rl.ac.uk/isis/projects/heliummgmt/_layouts/viewlsts.aspx?BaseType=1) - project management docs and other useful info +* [HLM View](https://github.com/ISISNeutronMuon/HLM_View) - data display website repo + +## Jenkins + +Automated building & testing is running on the IBEX Jenkins instance as the ["HLM PV Import" multi-branch pipeline job](https://epics-jenkins.isis.rl.ac.uk/job/HLM%20PV%20Import/). + +It is running on **NDW1757** and using the Python in `C:/HLM_PV_Import/python.exe`. To update Python, manually download a newer version and install it in there. + +The `Jenkinsfile` is in the project root, together with the `setup_jenkins_settings_file.py` script required (and called by the Jenkinsfile) when running the tests. + +**Notes**: +* In `requirements.txt`, make sure to keep `cryptography` at `3.1.1` to avoid build errors due to newly added Rust dependencies. +* Use `call "myvenv\\Scripts\\activate.bat"` instead of only `myvenv\\Scripts\\activate.bat` in the Jenkinsfile bat commands or it won't work. + + diff --git a/doc/specific_iocs/cryogenics/ICE-Dilution-Fridge.md b/doc/specific_iocs/cryogenics/ICE-Dilution-Fridge.md new file mode 100644 index 000000000..eee9f5eda --- /dev/null +++ b/doc/specific_iocs/cryogenics/ICE-Dilution-Fridge.md @@ -0,0 +1,60 @@ +# ICE Dilution Fridge + +## Introduction + +This dilution fridge is made by a company called ICE and is no longer supported. + +This dilution fridge can not be compared to the Triton dilution fridge, and is the only one not controlled by Triton gas handling systems. + +There is only one ICE dilution fridge which is used on MUSR mainly. It is available for EMU and I believe there is an expectation that it will be an option for the RIKEN beam lines also. + +## Connection + +The ICE dilution fridge itself is controlled through a LabView VI running on a PC called ICECube. That PC runs Windows XP and should absolutely not go on the ISIS network. That PC runs the VI that contains the complicated logic for making dilution happen. That PC is then controlled by an IOC running under IBEX, which replaces a much simpler VI found at `C:\LabVIEW Modules\Drivers\ICEOxford\ICECube` . The IOC talks to ICECube over a serial connection. + +Settings: + +1. Baud rate: 9600 + +### Command set + +The complete command set for the ICE dilution fridge is detailed in the Remote Interface document on the share network folder. + +**Note:** The IOC has been developed to have the same functionality as the ICECube VI. As a result, it only has the PVs and only supports the commands that are used by the VI (with one exception detailed in Gotchas). There are some commands in the Remote Interface that are never used by the VI and are thus not implemented in the IOC. These are: `IDN?`, `1KPUMP?`, `HE3PUMP?`, `ROOTS?`, `STATUS?`, `SVx?`, `Vx?`, `Px?` (where x is the number of the solenoid valve/valve/pressure), `T?`, `T-R?`, `cryo-r?`, `STILL-R?`, `1K-R?`, `1K`, `CLOSEALL`, `LOG=TRUE/FALSE`, `File Path`, `MC-PID`, `MC-TSET`, `MC-HTR-RGE`, `STILL_PWR`, `CRYO-HTR`. + +The dilution fridge temperature at low temperatures is controlled via a Lakeshore 370 remotely. This can be found in the documentation command set and the two commands are `LS-DIRECT-SET` and `LS-DIRECT-READ`. These two commands will pass any command that you wish on to the Lakeshore 370 directly - as though you were talking to the device itself. This is probably the reason why the MC- commands listed above are not used, since those parameters are controlled via the LS 370. On the share network folder you can find the manual for the Lakeshore 370 and it gives detailed information about all the commands the Lakeshore accepts. + +The cryostat temperature control is via a CRYOCON 24C. There is a remote direct command also for this namely CRYO-DIRECT-SET and CRYO-DIRECT-READ. They are listed in the Remote Interface but never used by the LabView VI and thus not implemented in the IOC. + +### Channels + +The ICE dilution fridge has 4 distinct channels: + +| Physical sensor location | Channel name | Notes | +| --- | --- | --- | +| Outer Cryostat | T1 | The T1 channel represents the temperature of the outer cryostat and is controlled through VTI Loop 1. VTI Loop 1 has been used by scientists regularly in the past, and is the main temperature on the VTI tab that the instrument scientists are interested in. | +| Auxiliary equipment | T4 | The T4 channel represents a temperature sensor that can be attached to any auxiliary equipment and is controlled by VTI Loop 2. VTI Loop 2 has been used by scientists rarely. | + +The T2 and T3 channels are seldom if ever used, therefore the temperature sensors they monitor are almost always disconnected. + + +## Configuration + +In order to run this IOC, you should have the ICECube remote PC running and connected via serial. No further configuration should be needed. + +## Gotchas + +- Some PVs are set to scan every second, some every 5 seconds and some every 10 seconds. Because of this, some of the readbacks in the OPI take a bit to update after you set the setpoint. Because there are a lot of read requests since the IOC has many PVs, sometimes it takes even longer for the pv to update, up to 8 seconds for 5 second scan PV and up to 12 seconds for a 10 second scan pv. There are forward links in the setpoint corresponding to these PVs so that the readback is updated immediately after changing the setpoint, but the command is queued among others so it does not help much. +- There are some graphical elements in the VI code that are hidden and that do not appear during use of the VI as far as we know. This is a readback for 1K (different from readback of 1K stage, uses the 1K? command), a readback for the state of the device (that is in included in the diagnostics tab of the OPI) and proportional valve 4. Proportional valve 4 is implemented in the IOC but does not appear in the OPI. The 1K readback has not been implemented because we looked through the blocks on the configurations of MUSR and EMU on SECVI and found no block on 1K, so we assumed it was not important. +- The roots pump indicator and button do not show up in the VI when not running, but appear once you run it against an emulator. +- When you send a set command, the ICECube PC sends back a reply of "OK". The exception is for any set commands sent to the Lakeshore 370 directly, where it replies with "Set LakeShore 370". +- When you set the temperature setpoint for the mixing chamber in the Lakeshore OPI tab, it also sets the scan and cmode Lakeshore 370 settings. If the setpoint is zero, scan is set has arguments 6 and 1, and cmode is set to 4. If it is not zero, scan receives arguments 6 and 0 and cmode is set to 1. +- When setting the Lakeshore MC PID values, you need to set all of them at once. The IOC does this by updating all MC PID readbacks then sending the values replacing one of them with the new setpoint. +- The Lakeshore MC Integral and Derivative are always integers. The ICECube sends them back when reading the PID values as integers and not as floats. +- When reading the voltage range the device sends 5 values. The second is the voltage range, and the last three are ignored when reading for the readback of the voltage range of a channel. But when you want to set the voltage range setpoint, you need to send the device 6 values, the last three being the Voltage Range Resistance Range, the Voltage Range auto-range mode and voltage range CS-Off mode. Therefore, when you change the setpoint, the IOC first reads the voltage range and saves those three values to some private PVs. Those are then used for the set command sent to the Lakeshore 370. +- The Proportional and Needle Valves' OPI images are either green or red depending on whether or not their value is more than zero, or zero respectively. This is done through a calc record for each valve that checks if the readback is greater than zero. +- On the Mimic tab of the ICECube Client vi, the M/C(K) readback displays one of two different values. It displays the mixing chamber temperature if that is over or equal to 0.051 K. If it is under 0.051 K, it displays the readback of the Mixing Chamber Resistance. Initially, this was also implemented in the IOC and OPI. However, after testing it with the instrument scientist we decided that it was a bit confusing. As a result, we agreed to always display the two readbacks separately on the Mimic tab of the OPI. +- During testing with the hardware, we discovered that the Mixing Chamber Resistance readback was 1000 times higher than the readback on the ICECube. No division is done in the ICECube client vi, so we assume it is done in the ICECube vi. A calc record was added to divide the readback by 1000 and that calc record value is shown in the OPI. +- On the mimic tab, the "Temperature set" setpoint is sent to the device only when you press Start and if the Sequence is set to Temperature Control. +- The sequence dropdown ad the Start, Stop and Skip buttons are only enabled when the Mode is set to Semi Automatic. +- The ICECube Client does not read or set the mode. The Remote Interface of ICECube has no commands to read or set the mode of the dilution fridge. The mode drop down on the OPI only enables or disables the Start/Stop/Skip buttons. The OPI drop down is Alarm sensitive to make it clear that in the beginning the value in the drop down does not reflect what the hardware does. diff --git a/doc/specific_iocs/cryogenics/ITC-503.md b/doc/specific_iocs/cryogenics/ITC-503.md new file mode 100644 index 000000000..98024ec6f --- /dev/null +++ b/doc/specific_iocs/cryogenics/ITC-503.md @@ -0,0 +1,29 @@ +# ITC503 + +It is an oxford instruments Piece of kit. It is used as the cryostat as part of the 7.5T magnet (see also [ips](Oxford-Instruments-IPS) which has more detailed documentation about the 7.5t magnet). It is also used as part of the MuSR 3He Refrigerator (though that lives in its own IOC see [here](ICE-Dilution-Fridge)). + +## Gotchas + +- Scans very (very) slowly. This is necessary for communication with device. +- It is stolen from diamond. +- Occasionally device will ignore first character of command, e.g. + +``` +2018/05/16 16:57:02.169 \\.\COM2 write 3 +R9\r +2018/05/16 16:57:02.190 \\.\COM2 read 3 +?9\r +2018/05/16 16:57:02.189831 L0 TE:NDW1799:ITC503_01:I: Input "?9" mismatch after 0 bytes +2018/05/16 16:57:02.189842 L0 TE:NDW1799:ITC503_01:I: got "?9" where "R" was expected +``` + +Usually it should reply with something like `R+5.000` to that command. The `?9` response suggests that the device has dropped the first character. I tried various things to get this to work but it is just a fail on the device's part. + +## Getting weird stuff on serial line + +- Check serial settings (9600 baud/8 bits/No parity/2 stop bits) +- Check you only have one device on the end of the line. ITC/ILM systems can be addressed, if they both share a line then you can get interleaved replies from both devices which looks like nonsense on a serial line + +## ISOBUS addresses + +ITC and ILM devices can be addressed by an ISOBUS address. There is a utility in `C:\LabVIEW Modules\Oxford Software\COMMDEV.llb\Instruments?.vi` which will scan all ISOBUS addresses on a COM port and tell you what device(s) are connected to each one. diff --git a/doc/specific_iocs/cryogenics/ITC503-Heliox.md b/doc/specific_iocs/cryogenics/ITC503-Heliox.md new file mode 100644 index 000000000..ba4dfc9dd --- /dev/null +++ b/doc/specific_iocs/cryogenics/ITC503-Heliox.md @@ -0,0 +1,186 @@ +# ITC503 (Old-style Heliox) + +Original implementation for use on MuSR 3He refrigerator. The device is a Heliox/ITC503 based refrigerator controlled by the HLX503 IOC. + +## Physical operation and situation + +![3He Fridge Diagram](3HeFridgeOps.png) + +### Situation + +The 3He Refrigerator is also known as the 3He Insert because it is inserted into an outer cryostat. This document is only related to the insert not the outer cryostat, but it is useful to know that the outer cryostat cools the 3He insert to ~1.5K. From the IBEX point of view the insert and outer cryostat are separate pieces of kit. However, it is useful to know that the `1KPot` in the insert is wired to the outer cryostat to control its temperature (we do not control the temperature via the insert). The insert is controlled by 1 ITC503 connected to a “magic box” that is wired to help read and control the various parts of the insert. + +### Thermometers and heaters + +TL;DR 4 thermometers, 2 heaters. We can read 3 thermometers at a time and the ITC503 controls two heaters based on the temperature setpoint and control channel we give it. + +On the insert itself, there are 4 thermometers: the sorb thermometer, the 1K Pot thermometer, the He3Pot “High Temperature” thermometer and the He3Pot “Low Temperature” thermometer. Both the He3Pot thermometers are measuring the same temperature, one is good at measuring low-temperature values the other at high-temperature values. + +There are 3 ports on the magic box to connect the thermometers to (as there are 3 sensor channels in the ITC503 itself). The first sensor is connected to the sorb thermometer, the second to either the He3Pot Low-temperature thermometer or the 1K Pot (these are physically switched on the magic box) and the third is wired to the He3Pot high-temperature thermometer. + +The ITC503 has 1 output and the insert has two heaters (the sorb heater and the he3pot heater). By using the magic box this 1 output is at all times connected to the sorb heater, but only if the ITC503 is controlling based on the He3Pot “High Temperature” sensor then this output is also wired to the he3pot heater. + +### Operation/Behaviour + +TL;DR When you heat the sorb this reduces the pumping efficiency of He3 which cools the He3 pot more. + +The temperature of the He3Pot is defined by how much Helium-3 is being pumped onto the pot. The ITC503 controls the sorb temperature which affects the pumping efficiency of the He3 and as such the temperature of the He3Pot. When the Sorb temperature increases in heat the pumping efficiency is reduced which consequently warms the He3Pot. When the sorb reduces in heat the pumping efficiency is increased so the He3Pot temperature reduces. + +The Sorb sits in the outer cryostat so has a base temperature of about 1.5k that it will sit at when not heated. The ITC503 output is wired at all times to a sorb heater which allows the ITC503 to control the heat of the sorb and when we are controlling based on the He3Pot “High Temperature” sensor then this output is also wired to the he3pot heater. + +### Regeneration + +TL;DR After 24-48 hours of operation the He3 is no longer pumped onto the pot and we have to regenerate it by heating the sorb to ~30K. + +The system is closed-loop because of how expensive Helium-3 is. However, at some point there will not be enough Helium-3 in the pot to create cooling power (it is all in the pump). The symptom of this is that the temperature of the He3Pot to rise to the temperature of the 1K Pot. + +The process that causes the He3 to be pumped onto the He3 Pot is called absorption. This process is reversed when the sorb is at ~30K. To regenerate the He3 we need to heat the sorb to ~30K wait for the He3Pot temperature to reach a certain value and then fast cool the sorb to resume operation. + + +## HLX503 IOC Features + +### Current + +- Communication to one ITC503 with control channels defined for each of the `Sorb`, `He3PotHi` and `1KPot`/`He3PotLo` +- Set He3Pot temperature setpoint +- Set Sorb temperature setpoint +- Manual recondense +- Setting sorb and he3pot pids via tpar file +- Turn He3Pot Heater Off +- Graph the He3Pot temperature +- Monitor He3Pot setpoint, control channel, heater power, heater auto/manual + +### Future + +- [Autorecondense](https://github.com/ISISComputingGroup/IBEX/issues/6292) +- [IOC Restart State](https://github.com/ISISComputingGroup/IBEX/issues/6349) + +## Control Logic + +### Setting He3Pot Temperature + +- If set temperature < macro for Maximum temperature to operate the Heliox at + - If set temperature > macro for maximum temperature to use He3 cooling + - Change control channel to `He3PotHi` control channel + - Else + - Change control channel to `He3PotLo` control channel + - If Lookup PIDs is on: + -Look up He3Pot PIDs for the given setpoint and write them to the device + - Write temperature setpoint +- Else + - Temp setpoint in alarm + +### Setting Sorb temperature + +- Change control channel to Sorb control channel +- If Lookup PIDs is on: + - Look up SORB PIDs for the given setpoint and write them to the device +- Write temperature setpoint + +### Turning the He3Pot Heater off + +- Change control channel to He3Pot +- Set He3Pot autoheat to False +- Set He3Pot heater output % to 0 + +### Recondensing + +#### Autorecondensing + +Not to be implemented in first pass. + +All of the first set of conditions must be met and at least one of the second set of conditions. If this is the case then a recondense is required. + +##### First set of conditions + +All must be met before checking second set of conditions. + +- ITC503 is using the control channel chosen for the `He3PotLo` in macros +- Temp setpoint < Maximum temperature to use He3 cooling macro +- Auto condense enabled by macro + +##### Second set of conditions + +Only one of these conditions needs to be met. + +- Sorb temperature check is set to on in macros and Sorb temperature > condense threshold for the sorb temperature set in macros +- He3Pot temperature check is set to on in macros and He3Pot temperature > condense threshold for the He3 Pot temperature set in the macros +- Temperature error check is set to on in macros and difference between He3Pot temperature setpoint and He3Pot actual temperature > condense threshold for temperature error set in the macros + +#### Recondense Logic + +TL;DR Heat up the sorb to cause recondense, wait for He3Pot and Sorb to reach given values (showing recondense complete), fast cool the sorb and set the He3Pot to a user given temperature. + +The logic is split into 4 parts. Part 1 initialises the correct state, sets some states on the device, and waits for the Sorb and He3Pot temperatures to match certain criteria. Part 2 does mostly the same but using user-specified values from the macros specifically for part 2. Part 3 fast cools the Sorb, and part 4 is a final clear up. + +##### Part 1 +- Entry + - Set the last start of condense time to now + - Set recondensing status to true + - Set status of skipped, cancelled and timed out to False + - Set success status to False + - Set status to part 1 + - Set to manual heating and 0% heater output + - Set that we are not using tpar PIDs + - Set autopid to false + - Write Sorb PID values given by the user in the macros + - Change control channel to use Sorb + - Set temperature setpoint (sorb) to value given by the user in the macros +- Wait until one of the following conditions is met + - The user skips the part + - Set skipped status to True + - Set part success status to False + - The user cancels the operation + - Set part success and overall success to false + - Skip to not recondensing + - The operation times out + - Set part success to false + - Skip to the finish + - Sorb temp > User specified sorb condensing temp (from macros) – 0.5 and He3 Pot temperature < User specified condense He3Pot target for part 1 + - Set that this part was successful + +##### Part 2 (Now not needed - removed) + +- Set skipped to false +- Set status to part 2 +- Wait for the same conditions as in part 1 but using the He3Pot target specified for part 2 in macros + - Do same actions for each case +- If part successful then wait for the number of seconds given by the user in the macros + +##### Part 3 +- Set skipped to False +- Set status to fast cooling the sorb +- Change control channel to use Sorb +- Set sorb heater output to 0% +- Set sorb temperature setpoint to 0K +- Wait for the same skipped, cancelled, and timeout conditions and do the same actions or wait for this condition: + - Sorb temperature < final sorb temperature specified by the user in macros + - Wait for 2 seconds + +##### Finish + +- This should execute even if timed out, skipped or previous parts are successful, but not cancelled +- Set status to finishing +- If all parts have been successful then set overall success to true +- Set last finish of condense to now +- If the He3Pot temperature setpoint > the user-specified temperature threshold for He3 operation + - User will be notified by the He3Pot temperature setpoint PVs being in alarm if this happens + - Set He3Pot temperature to default post condense he3pot temp macro +- Else + - Set He3Pot temperature to the user-specified temperature setpoint +- Set condensing status to False +- Reset back to using autopid/tpar file/manually entered values as before recondense + +## Comms + +Controls one ITC503 on an RS232 connection direct into the MOXA. + +Macros with defaults for the ITC503: +- Baud rate e.g. `BAUD` +- Bits e.g. `BITS` +- Parity e.g. `PARITY` +- Stop bit e.g. `STOP` + +Set control channel and sensor details using macros e.g. SORB_CHANNEL and SORB_SENSOR. + +Further configuration of settings for recondensing and general operations are available (see macros). diff --git a/doc/specific_iocs/cryogenics/Mercury-Heliox.md b/doc/specific_iocs/cryogenics/Mercury-Heliox.md new file mode 100644 index 000000000..c2785e046 --- /dev/null +++ b/doc/specific_iocs/cryogenics/Mercury-Heliox.md @@ -0,0 +1,164 @@ +# Oxford Instruments Mercury Heliox + +The mercury Heliox systems are a type of refrigerator which cools using Helium-3. Using this technique, temperatures of around ~300mK can be achieved. + +The Heliox systems, like dilution fridges, are parasitic, which means they must sit in an outer cryostat or cryomagnet which provides the initial stage of cooling to low temperature (a few degrees Kelvin) before the Heliox provides the final stage of cooling (to ~300mK). The outer cryostat is usually controlled by a standard Mercury ITC controller. + +## Physical process - background + +A sorption cooling stage works by evaporative cooling of Helium-3. To understand the process, consider the following picture: + +![sorb](heliox.png) + +The components which are interesting for us are: +- Helium-3 pot. The sample is mounted to the bottom of this. +- Helium-4 pot (also called "1K pot"). +- Sorption pump + +The Helium-3 pot and the sorption pump form a closed system. Helium-3 can move between these two areas, but cannot escape. This is done because Helium-3 is extremely expensive, so pumping it away would be very wasteful. + +The sorption pump is a "pump" in the sense that it can control the pressure of the Helium-3 surrounding it. The physical process that achieves this is adsorption of Helium-3 onto the material (charcoal) in the pump. The pumping efficiency (i.e. pressure) of the pump can be changed by controlling it's temperature. The lower the temperature, the more Helium-3 the pump will adsorb per unit time. If the pump is warmed to ~30K, it will release all of the previously adsorbed Helium-3. + +The Helium-3 pot is where the lowest temperatures (~260mK) are achieved. In this pot, Helium-3 cools by evaporation. The rate of evaporation of Helium-3 (i.e. the cooling power) is changed by varying the temperature of the sorption pump (and therefore the vapour pressure above the Helium-3 in this pot). + +The Helium-4 pot is used to recondense Helium-3 back into a liquid after being released by the sorption pump (to release Helium-3, the sorption pump is warmed to 30K, and Helium-3 is a gas at this temperature). + +## Communications + +### Settings + +The Mercury can be physically configured to use serial, ethernet or GPIB as it's transport layer. At ISIS, we use serial. The serial settings can be changed on the physical device, but we typically use 57600 baud as this is usually an acceptable compromise of transmission speed against line length restrictions. If there is a very long connection from the moxa to the Heliox unit, and there is noise on the line, it may be useful to reduce the baud rate to see if this helps. + +### Command set + +The Heliox systems are physically based on the mercury ITC temperature controllers, but they do not use the same command set and cannot be controlled by the same drivers. The mercury Heliox systems have their own driver. + +The devices use an SCPI-like command syntax. There are two approaches to getting data using this protocol: +- Ask for everything in one go, e.g. `STAT:DEV:HelioxX`. This will return a huge status string containing every measurement under that category (~30 items). This approach is used by the LabVIEW driver (although it only actually looks at the data for a few measurements). It is also useful for enumerating the valid pieces of data that you can ask for individually. +- Ask for one thing at a time, e.g. `STAT:DEV:HelioxX:TEMP:SIG:TEMP`. This will only return the one measurement which we asked for. This approach is what the IOC uses. + +### Channels + +The mercury ITC driver essentially reads data from 5 distinct channels: + +| Physical sensor location | Channel name on LET Heliox | Channel name on Muon Heliox | Notes | +| --- |----------------------------|-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| He3 Pot | `HelioxX` | `HelioxX` | This is the "main" Heliox control channel. According to the OI manual this is the only channel which we should need to use to control the Heliox's temperature. In the IOC and LabVIEW driver, this is the only channel on which we allow setting setpoints. It is a hardware alias of either `HeHigh` or `HeLow` depending on temperature (see below). | +| He3 Sorption pump | `He3Sorb` | `MB1_He3_Sorb` | Dedicated channel for the (helium-3) sorption pump. Monitoring only. Note: the He3 sorption pump and the He3 Pot are not the same! | +| He4 Pot | `He4Pot` | `DB6_He4Pot` | Dedicated channel for the (helium-4) 1K-pot cooling stage. Monitoring only. Note: the way that ISIS run the Mercury Heliox systems means that this channel will read a constant value all the time, as there is no actual hardware present on the Heliox to read this. This hardware would only be present if a single Mercury ITC unit was used to control both the main cryostat and the sorption stage. | +| He3 Pot (high sensor) | `HeHigh` | `DB7_He3_Pot_CRN` | Monitoring only. This is a "high" (~2-80K) temperature thermocouple, used for measuring the temperature of the He3 Pot when the temperature is in it's range of validity. This channel will give invalid temperatures if the Heliox is running at "low" temperature. | +| He3 Pot (low sensor) | `HeLow` | `DB8_He3_Pot_Low` | Monitoring only. This is a "low" (~0.2-2K) temperature thermocouple, used for measuring the temperature of the He3 Pot when the temperature is in it's range of validity. This channel will give invalid temperatures if the Heliox is running at "high" temperature. | + +Because the channel names vary between the Muon Heliox and the LET Heliox, they must be supplied as IOC macros. (Apart from the `He3 Pot`, which is the same for both devices) + +If a new Heliox turns up on another beamline, the following is the process to figure out the required channel names: +- Connect to the device via your favourite terminal emulator (HTerm/PuTTY/HyperTerminal/etc). +- Issue the command `READ:SYS:CAT` (terminated with a line feed, `\n`) +- This will respond with a string like `STAT:SYS:CAT:DEV:::DEV:::...`. + * The IDs should look something like `DB1.H1` (meaning: daughter board 1, heater 1) + * The device type could be one of `TEMP` (temperature), `PRES` (pressure), `HTR` (heater) or `AUX` (auxiliary output). +- For each of the devices in the catalogue response, issue the command `READ:DEV:::NICK` (terminated with a line feed) +- The device will respond with a string that looks like `STAT:DEV:MB1.T1:TEMP:NICK:MB1_He3_Sorb` +- The last part of this response (`MB1_He3_Sorb` in this example) is the string that the IOC will need as a macro +- If it's not immediately obvious which channel name corresponds to which item in the table above, consult cryogenics for advice about how the channels have been named. + +## Regeneration + +### Physical process - background + +Sorption cooling stages can only dissipate a finite amount of energy before they must be "regenerated". This is contrast to dilution cooling, which can be run (essentially) continuously. + +A regeneration follows the following physical process: +- Regeneration is started either manually (by the user) or automatically by the IOC/LabVIEW driver if that option is selected +- The He3 sorption pump is warmed significantly (to about 30K). +- This heat causes the pump to release all of the Helium-3 stored in it. +- At the same time, the 1K pot is run, which causes the helium-3 to recondense when it comes into contact with the 1K pot +- Once all of the helium 3 has recondensed, the sorption pump is cooled back down. This causes it to start "pumping" and therefore cooling of the He3 Pot resumes. +- This process will take approximately 30-90 minutes depending on the starting temperature of the Heliox. + +The Heliox typically needs a regeneration after 24-48 hours of cooling. I think this depends on the sample and the external heat load. + +In effect, this means that the users lose temperature control while a regeneration is in progress. TODO: do the scientists know this or will they be surprised by this behaviour? Are regenerations typically done automatically or manually or does this depend on the scientist/experiment? + +### Detecting when a regeneration is required + +A regeneration is captured by a boolean with the following inputs: +- Mode = "Low Temp" +- And Heliox temperature > 0.4K +- And He3 Sorb heater in automatic mode +- And He3 Sorb heater percent heat < 0.2% +- And no comms errors within the last 120 seconds +- And auto-recondense enabled on VI front panel +- And one or both of: + * Either: + * Rate of change of temperature over the last 200 seconds > 0.0005K/min (calculated by line of best fit) + * AND variance of temperature over the last 200 seconds > 0.0005K + * AND (Heliox temperature - TSet) > 0.25K + * Or: + * (Heliox temperature - TSet) > 0.05K continuously for 600 seconds. This is the 'coarse check', where 600s is the 'coarse time'. + +This boolean must then stay true continuously for 120 seconds. If it does, then a regeneration is triggered in the VI. Under IBEX, the value of the boolean will be displayed on the OPI but no action will be taken. + +### Regeneration logic + +When a regeneration is triggered, the existing LabVIEW driver simply sends a temperature setpoint of zero. There is logic to do something much more complicated, but it is "commented out" in an `if False` statement. + +It appears to me as though the setpoint is never set back to the previous value in the LabVIEW code. There is a lot of code relating to this, but it is all disabled. + +Under IBEX we currently only allow manual regenerations. The same conditions as above are displayed on an indicator on the IBEX OPI, but no action is taken. This is because the scientists don't trust the current regeneration conditions and so they have not been hooked up. We would need to book lots of time with the Heliox to try and work out a better set of detection conditions if this logic were to be used in future. + +## LabVIEW driver oddities + +The following notes apply to the LabVIEW driver: +- The driver does not work with LabVIEW 2018. It works with versions from 2010 to 2014 - I am not sure in which intermediate version it stops working. The symptom is that it never even tries to send bytes to a serial port. I think it is crashing while trying to get a subVI by reference but I am not completely sure about this. + +## Troubleshooting + +### No comms to device but I'm sure the serial settings are correct + +The symptom is that the device acts as if it's not there - you will see serial writes going out but nothing coming back. This can happen if the device is left in local mode (for example, after a cryogen fill). + +I am not aware of a remote command to set the device back into remote mode, so you will need to walk up to the physical mercury controller. It has a touch screen with a small `l` in the bottom-left of the touch screen. If it is shown in blue, tap it once. It should turn orange, which indicates the device is now in remote. Check you now have comms. + +**NB** Only the main communications socket (25 pin d-type, labelled _RS232_) should be connected to a MOXA port, **not** any other sockets on the Mercury. + +### I can read the main temperature but not any channels + +Check channel assignments are correct - see table above and instructions for finding out channel names if unknown. + +### Devices are communicating but readback values are not correct + +Check the cables are connected correctly - you don't get super obvious errors if e.g. ITC and IPS cables are swapped. This issue may also be indicated by errors parsing responses from the device in the IOC log. + +An error could look like this if the IPS and ITC cables are swapped: + +`ITC503_01:MODE: Input "0A0C3H0M50P03" does not match format "%{A0|A1|A2|A3}"` + +### Regeneration starts but never finishes + +The Heliox detects whether a regeneration has "finished" by checking whether the temperature of the Helium-3 pot is smaller than a threshold. From testing, I believe the threshold is somewhere around 1.7K. The outer cryostat should be capable of achieving 1.6K or better. The Helium-3 pot won't go colder than the outer cryostat while regenerating, so if the outer cryostat is at a higher temperature than about 1.7K the regeneration will appear to hang and never finish. + +The outer cryostat could be warmer than expected for various reasons: +- Try gently increasing the needle valve percentage on the outer cryostat (e.g. from 6% to 8%) and check how the temperature responds. + * If it gets much colder, it may solve the issue. + * If it gets marginally colder or does not respond, or you have to adjust the needle valve beyond 10% or so, check with cryogenics group. There may be a leak or a cryogenic issue with the equipment meaning it is unable to reach the lowest temperatures. + * If it gets warmer, gently decrease the flow of He. The minimum attainable temperature is a minimum on a curve, and it is possible that reducing the flow of He can get the cryostat colder. + * If none of the above works or you are unsure, consult cryogenics section + +### Temperature control unstable + +The temperature control is known to be unstable at: +- Very close to base temperature, i.e. TSet < 300mK. This is because even the tiniest of heater changes cause a large change in temperature when this close to base temperature +- Near the "low temperature"/"high temperature" threshold, which is approximately 1.5-1.7K. This is a result of the Heliox changing it's mode of operation around this threshold. + +Example ramp (260mK-1.8K in steps of 20mK) demonstrating these instabilities (notice significant overshoot below 300mK and large instabilities from 1.5-1.7K): + +![](mercury_heliox_ramp_example.PNG) + +There is nothing the driver does in either SECI or IBEX which can affect the stability. Consult cryogenics to check outer cryostat performance, PID parameters, flow rates etc. + +### Regeneration starts when I set temperature setpoint = 0 + +This is a feature of the Heliox intentionally added by oxford instruments, when the device gets a setpoint of zero it interprets it as "regenerate and then cool" as opposed to just "cool". + +This can be avoided by setting a temperature in the range 0 < T < base temperature to cool to base temp. diff --git a/doc/specific_iocs/cryogenics/MercuryEnhancedCryo.drawio.png b/doc/specific_iocs/cryogenics/MercuryEnhancedCryo.drawio.png new file mode 100644 index 000000000..a01f12516 Binary files /dev/null and b/doc/specific_iocs/cryogenics/MercuryEnhancedCryo.drawio.png differ diff --git a/doc/specific_iocs/cryogenics/MercuryEnhancedCryo2.drawio.png b/doc/specific_iocs/cryogenics/MercuryEnhancedCryo2.drawio.png new file mode 100644 index 000000000..92500defc Binary files /dev/null and b/doc/specific_iocs/cryogenics/MercuryEnhancedCryo2.drawio.png differ diff --git a/doc/specific_iocs/cryogenics/MercuryEnhancedCryoDeviceScreen.png b/doc/specific_iocs/cryogenics/MercuryEnhancedCryoDeviceScreen.png new file mode 100644 index 000000000..8926f2e40 Binary files /dev/null and b/doc/specific_iocs/cryogenics/MercuryEnhancedCryoDeviceScreen.png differ diff --git a/doc/specific_iocs/cryogenics/MercuryiTC-Enhanced-Cryostat-Control.md b/doc/specific_iocs/cryogenics/MercuryiTC-Enhanced-Cryostat-Control.md new file mode 100644 index 000000000..cb3356baa --- /dev/null +++ b/doc/specific_iocs/cryogenics/MercuryiTC-Enhanced-Cryostat-Control.md @@ -0,0 +1,77 @@ +# Mercury iTC - Enhanced cryostat control + +## The Problem + +EMU is the only beamline to be fully controlled by new MercuryiTCs as opposed to ITC503s as of 03/11/2021. The MercuryiTCs are newer and will become replacements to the ITC503s for more beamlines as time goes on, particularly on WISH and RIKEN initially. The problem with the new MercuryiTCs is that they have two modes: + +- Pressure control mode + - Controls needle valve based on the pressure +- Automatic needle valve control mode + - Controls needle valve based on heater power, affecting the pressure but ignoring the pressure set by the user + +It is difficult to switch between the two modes and requires multiple Mercurys which cost thousands of pounds. For optimal control, scientists need the needle valve to be controlled based on the pressure (done automatically by the Mercury in Pressure Control Mode), but they also need the pressure to be controlled automatically based on the temperature setpoint (to be implemented in IBEX). + +## Design + +This design comes after feedback from scientists and cryogenics teams on the previous design (below). There were concerns about the stability of control with the use of a single lookup table as it had not been tested before. What had been tested before is the algorithm used on the orange cryostat. The second design uses elements from both the first design and the orange cryostat. We have removed the use of the lookup table in favour of calculating the new pressure setpoint from `(T - Tset) ^ 2` which is limited by two separate maximum pressures (one specifically for the given temperature) and one a more general "safe" maximum, and a minimum pressure. + +### Python test script + +It was decided that we should test the algorithm using a python test script which can be found in the `Settings/config/NDXEMU/Python/inst` under the name `cryo_control_test.py`. The script has two classes: `UserDefined` and `CryoControlAlgorithm`. `UserDefined` contains defaults for parameters in the algorithm that a user will be able to define. `CryoControlAlgorithm` contains all the algorithmic logic. To run a test call `CryoControlAlgorithm.run()`, you can change the user-defined values by passing them as parameters to this run function e.g. to change the cutoff point call `CryoControlAlgorithm.run(cutoff_point=7.0)`. Other parameters can be found in the script. + +### Flowchart + +![Flowchart design](MercuryEnhancedCryo2.drawio.png) + +Some amendments were made in a future meeting after the creation of this flowchart: +1. `temperature setpoint <= cutoff point` -> `temperature setpoint <= cutoff point && temperature <= cutoff point`. +2. Set delay should be 10s by default, rather than 200ms. +3. Pressure law changed: + 1. `new pressure setpoint = (temperature - temperature_setpoint)^2` becomes + 2. `new pressure setpoint = pressure setpoint minimum + temp scale * (temperature - temperature setpoint)^2` + +## Implementation + +Implementation is to be done by modifying the existing MERCURY_ITC IOC in IBEX ([MERCURY_ITC IOC](https://github.com/ISISComputingGroup/EPICS-ioc/tree/master/MERCURY_ITC), [MercuryiTC support module](https://github.com/ISISComputingGroup/EPICS-MercuryiTC)). This implementation will enable the mercury hardware to be always configured for Pressure Control Mode, whilst we add an automated pressure control behaviour to optimise the pressure for given temperature setpoints. This automated pressure control behaviour sets the pressure based on the temperature and the temperature setpoint. My recommendation would be to build the logic with a small state machine in snl, with new PVs where required + +### Switching the automated pressure control on and off + +The automated pressure control behaviour needs to be able to be turned on and off. The IOC should behave as if no changes had been made to the IOC when this behaviour is turned off. The behaviour should be turned on and off by a boolean checkbox in the OPI, which controls a PV whose value is persisted through autosave. + +The benefits of using autosave over a macro: + +- Makes the IOC more testable (not requiring a restart of the IOC in the IOCTestFramework to switch modes) +- Enables switching between modes without reloading config + +### Operation delay + +The user can choose a delay between setting setpoints to avoid overloading the device. This delay will be set in milliseconds with a default of 200 ms. It could be set to 0 ms to avoid any delay. The delay value should be an autosaved PV so the value is persisted. + +## Previous Design + +![Flowchart design](MercuryEnhancedCryo.drawio.png) + +### Temperature cut-off + +There will need to be two modes of operation, one at high temperatures which uses a lookup table and one at low temperatures which would set a constant pressure. We define high temperatures as any temperature above a user-defined cut-off point, and low temperatures as anything below it. This cut-off point should persist (using autosave) and default to 5 Kelvin. + +### Low-temperature operation + +Another problem with the MercuryiTCs is that when in automated needle valve control at a temperature of less than 5 Kelvin the needle valve is fully opened, which is not optimal for temperature control. When operating below the cut-off temperature our automated pressure control should set the pressure to a user-defined constant pressure value. This constant pressure value should persist (using autosave) and default to 5 mbar. + +### High-temperature operation + +When operating above the cut-off temperature the MERCURY_ITC IOC should use a lookup table to decide what to set the pressure setpoint to. There should be a reasonable default lookup table in the common configs area, but a user should be able to set their own lookup table stored in the instruments config area. The lookup table is a key-value pair. The key is the difference between the temperature and the temperature setpoint. The value is the pressure setpoint to set when the temperature - temperature setpoint is within the range of the values given key. The lookup table could be implemented using [ReadASCII](https://github.com/ISISComputingGroup/EPICS-ReadASCII). + +## Device Screen + +On branch at https://github.com/ISISComputingGroup/ibex_gui/tree/Ticket6729_enhanced_cryostat_control. + +![Device screen design](MercuryEnhancedCryoDeviceScreen.png) + +### Questions whilst designing + +- There seems to be a mix of terminology for the lookup table key. Here I have used the difference between the temperature and temperature setpoint because it was one of the terms used and seemed logical to me. The other suggestion was using the difference between the current heater power and the target heater power. Is using the difference between the temperature and temperature setpoint correct? Also, the IOC currently does not have the ability to read a current and target heater power it seems to just read a single value. The manual seems to suggest for a heater controller we can set and get a heater power but that is untested comms behaviour from our end. + - The heater power I have referred to here seems to be in Watts. There seems to be a few different heater values with different units. Is the one referred to previously a value in Watts, Volts or Percent? +- This design expects the user to set a single temperature channel and a single pressure channel for the behaviour to work with, does this match the requirements? If so are the single temperature cards and pressure cards that we want to use for this control fixed or do they vary? +- This design does not use parameters that the mercuryitc uses for auto needle valve control but relies on a lookup table being handwritten. Is this sufficient or do we need to come up with an algorithm to tune that table from some parameters? \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/MercuryiTC.md b/doc/specific_iocs/cryogenics/MercuryiTC.md new file mode 100644 index 000000000..73c8a3cb6 --- /dev/null +++ b/doc/specific_iocs/cryogenics/MercuryiTC.md @@ -0,0 +1,180 @@ +# Mercury iTC + +The Mercury iTC is a configurable temperature controller from oxford instruments ( [Manual](http://lmu.web.psi.ch/docu/manuals/bulk_manuals/OxfordInstruments/Dolly_Mark_II/Mercury_iTC_manual_Issue_05.pdf) ). + +It contains various slots into each slot a variety of devices can be placed. For the purposes of this document the motherboard is treated as a slot too, it contains a temperature sensor and a heater. + +Types of daughter boards are: + * temperature sensor + * Heater + * Auxiliary board (general purpose and stepper motors) + * Pressure sensor + * Cryogen + * GPIB board (communications) + +## Communications + +The device should be talked to via serial (RS232 / ISOBUS in OI terms) at a baud rate of 57600. This is variable on the front panel, but **only while the device is in local mode** (when the box in the bottom left of the home screen **_doesn't_** have an orange background). + +The configuration screens look like: + +![Comms general](comms1.jpg) +![Comms detailed](comms2.jpg) + +## Driver +Currently, the driver only measures and sets the following: + +- Temperature +- He Level +- Pressure + +and allows control of: + +- Pressure based on temperature (used initially for the little blue cryostat) + +There is a section on each below as well as how to set the communication settings. + +### Temperature + +{#mercury_itc_setup} +#### Setup + +To activate a card you must set the related IOC macro (note that this is different from in SECI where there was a separate VI to talk to each individual card e.g. Mercury_1_Temp_4 to talk to the 4th card as a temperature card). The following macros set the for the possible IOC slot: + +| Macro | IOC Name | +| ---- | -------- | +| MERCURY_01__TEMP_1 | %MYPVPREFIX%MERCURY_01:1 | +| MERCURY_01__TEMP_2 | %MYPVPREFIX%MERCURY_01:2 | +| MERCURY_01__TEMP_3 | %MYPVPREFIX%MERCURY_01:3 | +| MERCURY_01__TEMP_4 | %MYPVPREFIX%MERCURY_01:4 | +| MERCURY_01__LEVEL_1 | %MYPVPREFIX%MERCURY_01:LEVEL:1 | +| MERCURY_01__LEVEL_2 | %MYPVPREFIX%MERCURY_01:LEVEL:2 | +| MERCURY_01__PRESSURE_1 | %MYPVPREFIX%MERCURY_01:PRESSURE:1 | +| MERCURY_01__PRESSURE_2 | %MYPVPREFIX%MERCURY_01:PRESSURE:2 | +| etc | | + +To find out what values these macros should take, connect the mercury IOC and check the "System" tab on the opi, which will list the cards which are currently present. + +Example: If the list of Temperature cards displayed on the system tab is ["MB1.T1", "DB1.T1", "DB2.T1"] you should set the following macros: +- `TEMP_1`: `MB1.T1` +- `TEMP_2`: `DB1.T1` +- `TEMP_3`: `DB2.T1` +- `TEMP_4`: (empty string) + +#### Important PVs + +* `TEMP` Current temperature +* `TEMP:SP` Setpoint for temperature controller +* `TEMP:SP:RV` Read back of the set temperature point +* `NAME` Name associated with the card + +### He Level + +The helium level can be monitored by setting the macro `LEVEL_N` to point at the correct vi in a similar fashion to the temperature. + +### Pressure + +A pressure card can be monitored by setting the macro `PRESSURE_N` to point at the correct vi in a similar fashion to the temperature. + +### Software Pressure Control + +The little blue cryostat has a software pressure control mode to save Helium. The pressure set point, and in turn the needle value opening range, is set based on the set point and current temperature. There is a PV to set this to be on for the Mercury, in addition this will turn on Temperature PID, heater and pressure flow rate automatic settings and turn off the temperature flow rate automatic setting. The pressure will be set base on four regimes based on the temperature - setpoint: + +- Less than - 2x the temperature deadband: Pressure is set to `minimum pressure`. The idea is to let the cell warm up as quickly as possible. +- from -2x to -1x the deadband: Pressure is set to the value for the temperature from the table. The cell needs to warm up but more slowly. +- from -1x to 1x the deadband: Pressure is set to the value for the temperature from the table + a ramp is added from the offset to 0. Cell is the right temperature reduce the offset we used for extra cooling to be 0. +- above the deadband: Pressure is set to the value for the temperature from the table + offset + a value proportion to the square of the gain * distance above deadband the temperature is. Increase the pressure to cool the cell as quickly as possible reducing extra cool as we get near the deadband. + +Temperature pressures are then coerced to be between a minimum and maximum pressure. + +![Comms detailed](SPC.png) + +To enable the full auto control the FULL_AUTO_PRESSURE1` macros must be set. + +A full list of macros to set are: + +Macro | Default | Purpose +----- | ------- | ------- +SPC_PRESSURE_X | blank | The index of the pressure card to control for temperature card X +SPC_MIN_PRESSURE | 0 | Minimum pressure allowed +SPC_MAX_PRESSURE | - | Maximum pressure allowed +SPC_TEMP_DEADBAND | - | Deadband for the temperature and setpoint +SPC_OFFSET | - | Offset from which the ramp is reduced +SPC_OFFSET_DURATION | - | Time in minutes over which the offset is reduced +SPC_GAIN | - | Gain term for the extra pressure + +The pressures for the various temperature setpoints are read from `...Settings\config\common\other_devices\little_blue_cryostat.txt` + +### Example + +In this example, the front panel (home screen) on the Mercury looks like: + +![front panel showing 6 areas](front_panel.jpg) + +NB The device should be in remote mode this is indicated by the *i* (bottom left) having an orange background. +This mercury has 3 temperature sensors, Sample_Rod, VTI_DB6 and PT2_DB7, the VTI has a heater attached to the second temperature. + +The temperature controls are on the 1st and 2nd temperature and so on the first and second tab on the mercury device screen. NB the device screen also needs these macros set. + +For info here is a detailed temp loop: + +![detailed front board](detailed_temp_board.jpg) + +## OPI + +The OPI has macros that relate to the macros set at the IOC level. + +## Outdated Macros +There are some MercuryiTCs which are using outdated macros (in the format `VI_`...). See below for where these are located. +| **Instrument** | **Configuration/Component** | +| ------------- | ------------- | +| LET | LET_Mercury | +| OSIRIS | OSIRIS_Mercury | +| MAPS | mercury_itc | +| GEM | Mercury_cryostat | +| SANDALS | GEM_mercury | +| IRIS | triton | +| ZOOM | isisstat | +| LARMOR | ISISSTAT | + +If you use one of these MercuryiTCs could you please: +* add and configure the new macros if they have not yet been added +* configure the new macros if they exist but have not yet been configured + +The new macros and how to configure them can be found in [Setup](#mercury_itc_setup) section above: + + +## Troubleshooting + +### Mercury doesn't communicate + +1. One issue we have seen is that when the Mercury is transferred between instruments, or during some initial configuration, a user will change values like the Baud rate. It's good practice to check the devices settings (baud rate etc). +1. Ensure that it is set to remote mode. On the front panel the *i*, bottom left, should have an orange background. + +### Pressure card won't read heater voltage correctly + +This may be accompanied by an error in the ioc log: + +``` +2019/12/15 16:18:04.417 TE:NDW1799:MERCURY_01:PRESSURE:2:HEATER:VOLT devAsynFloat64 process read error lvDCOMDriver:readFloat64: status=0, function=13, name=HEATER:VOLT, value=0.000000, error=LabVIEW : parameter %p not found in the VI's connector pane. : IDispat +``` + +If this is the case, you need to get the latest version of `Mercury - Pressure.vi` from sourcesafe. Older versions had two parameters with the same name, which LvDCOM could not cope with correctly. + +### Cannot input IOC macros of the form `MB0.T1` + +You may need to rebuild ioc startups on the instrument - Release 7.1.0 initially shipped with a regex which was too restrictive and did not allow `.` as a character. + +To do this: +- Ensure it is ok to shutdown ibex temporarily (i.e. check with scientist and ensure script/dae is not running) +- Run `stop_ibex_server.bat` +- Run `make iocstartups` from `C:\instrument\apps\epics` (it is ok if this complains about some missing directories) +- Run `start_ibex_server.bat` + +### Mercury is not talking to any cards + +Ensure that IOC macros have been set which correctly correspond to the cards in the mercury itself. You can check which cards are currently in the mercury by looking at the "system" tab in the opi. The macros should be strings of a form similar to "DB1.T1" + +### Mercury is not reading a heater/needle valve correctly on a temperature/pressure card + +In the details screen of the OPI, check that the `associated heater channel` (for heaters) or `associated aux channel` (for needle valves) is not None. If it is None, check with the scientist/cryogenics whether a heater/needle valve should be configured for this channel. diff --git a/doc/specific_iocs/cryogenics/Oxford-Instruments-IPS.md b/doc/specific_iocs/cryogenics/Oxford-Instruments-IPS.md new file mode 100644 index 000000000..ac34ca82e --- /dev/null +++ b/doc/specific_iocs/cryogenics/Oxford-Instruments-IPS.md @@ -0,0 +1,112 @@ +# IPS + +The Oxford instruments IPS is a cryogenic magnet power supply. **This device can cause a magnet to [quench](https://en.wikipedia.org/wiki/Superconducting_magnet#Magnet_quench) if it sends commands while the magnet is in the wrong state! Please be careful when modifying this driver, especially if modifying the state machine.** In particular, the two most obvious states which must be avoided are: +1. PSU current != Magnet current and switch heater on +1. Ramping the Magnet current too fast + +When configuring the IOC, suggested blocks to create are: + +| Name | PV | Description | +| --- | --- | --- | +| Field | `FIELD:USER` | This is a composite field block that will choose to display either the persistent field or the power supply field, depending on the status of the switch heater | +| Magnet_status | `STS:SYSTEM:FAULT` | This is a status readback from the hardware. Should always be "Normal" (Enum: 0). **It is recommended to put run control on this being 0 (set limits -0.5 to 0.5), otherwise it can be difficult for the scientists to see that there is a problem.** | + +## Basic cryomagnet operation + +The circuit diagram of a cryogenic magnet with a switch heater looks something like the following: + +![Cryomagnet](cryomagnet_circuit_diagram.PNG) + +The basic idea of operation in persistent mode is as follows: +- If the switch heater is turned off, the magnet can be set to a particular current/field by the IPS +- Once the magnet is set to that current/field, the switch heater can be turned off. + * This closes the superconducting switch + * Because of how cryomagnets behave, you now get a "persistent" current which flows around the superconducting loop formed by the magnet and the switch. + * Persistent currents are an advantage because they allow much greater stability than a power supply + * When in persistent mode the IPS can be ramped down and the persistent current within the magnet will keep generating a field. +- Now that there is a persistent current flowing around the magnet/switch, the IPS can be ramped down + * This has the advantage that it saves power & cryogens + * Must set the IPS back to the same current as the magnet before turning the switch heater back on! + +**GOLDEN RULE OF OPERATING THE IPS: Only turn the switch heater on if PSU current == Magnet current** + +## State machine + +The IOC implements the following state machine in SNL: + +- Initial state + * The state that the state machine starts in + * The machine can get stuck in this state if the field PV is not processed +- At field + * This is the state where the magnet is at a particular field (either persistent or not), and is not ramping to anywhere. + * The activity of the IPS should be "hold" - this instructs the IPS to maintain it's current output at the present level. + +- Set PSU to match magnet + * The state machine always sets the IPS to explicitly match the current in the magnet, regardless of whether the current is already matched. This is only technically needed if the magnet was in persistent mode previously, but it doesn't do any harm to set it explicitly anyway. + * The activity of the IPS is "Ramp to Setpoint" + +- Wait for voltage to stabilize + * Once the PSU is at the required current, wait for the voltage to be stable. We define stable as 5 successive readings which are all within a *stability voltage* of each other. The stability voltage is set as a macro `STABILITY_VOLTAGE` which defaults to 0.1V. + +- Turn switch heater on and wait + * This state puts the activity back to "Hold" + * It also instructs the switch heater to turn on and then waits for the heater to warm up + +- Set PSU output + * At this stage in the state machine, we can begin the ramp to the target field + +- [PERSISTENT MODE ONLY] Switch off heater + * Once the magnet has reached the target field, it is safe to switch off the switch heater again + * We wait for the switch to cool back down again (and become superconducting) before moving onto the next state + +- [PERSISTENT MODE ONLY] Ramp down PSU + * The heater is now OFF (the switch is superconducting), so the PSU is free to ramp down without affecting the field produced by the magnet + * Set the IPS activity to "Ramp to Zero" and wait for the supplied current to drop to zero + * At the end of this state, the magnet is set to "persistent" mode. + +## IOC notes + +- The IOC does not enter it's state machine until a setpoint is set. + * If a ramp was in progress at the time the IOC was started, the ramp will continue, but no further action will occur (for example, the magnet won't be set to persistent mode and the heater status won't be changed). + * This is done so that there are no assumptions made about the state of the IPS which might be invalid + * To continue setting the field to the desired value after an IOC restart, simply resend the setpoint +- There are user facing records `FIELD:USER` and `FIELD:USER:SP` which should be used for blocks. This field readback will choose whether to display either the persistent field or the power supply field depending on whether the switch heater is on or not. The setpoint is an alias to `FIELD:SP`. + +## Settings + +There are various IPS units here is a list of the ones we know the settings for (Lab view settings are in `...LabVIEW Modules\Oxford Software\TESLTRON.INI`): + +Setting | Description | IN LabVIEW | 7T Magnet | 9T Chopper Magnet (LET) +------- | ----------- |---------------------------------------------|------------ | --------------- +`STABILITY_VOLTAGE` | 5 voltages are compared and they must differ by less than this to be declared stable | hard coded | 0.1 | 0.1 +`MAX_FIELD` | Maximum field | `PS0MaxField(2.2K)` and `PS0MaxField(4.2K)` | 7.0 [1] | 8.8 [2] +`MAX_SWEEP_RATE` | Maximum rate of change of the field in B/min | `PS0MaxSweep(4.2K)` and `PS0MaxSweep(2.2K)` | 0.399 | 0.3990 + +[1] - The limit may be lower on some beamlines due to the construction of their sample area. Below are known cases: + +| Beamline | Maximum field | +| ------- | ----------- | +| ZOOM | 5.5T | +| MERLIN | 2.0T | + +[2] - Limited by hardware. The IPS is configured to reject any setpoints above 8.8T. + +## Troubleshooting + +### Magnet won't go to field which has been set - "Activity" is "Clamped" + +Check on the "Advanced" tab of the OPI whether the IPS reports it's status as "clamped". If so, the state machine will be unable to proceed because being "clamped" means that the IPS will stay at it's current output indefinitely. "Clamped" mode is usually used by cryogenics when setting up the magnet, it is potentially undesirable to do this remotely without checking with them first. + +Note: You *may* be able to get out of a clamped state remotely with `caput %MYPVPREFIX%IPS_01:ACTIVITY:SP Hold`, but the old LabVIEW driver did not do this and it is not exposed as a user-facing option in the EPICS driver. **You should double check that this is ok with cryogenics before executing this command! It is potentially dangerous if the magnet has not been fully set up.** + +### Magnet won't go to field - sweep rate readback is zero + +Set the magnet to a non-zero sweep rate (you will need to be in manager mode to do this). Maximum permissible sweep rates are listed in the table above. If unsure, consult the instrument scientists or cryogenics. The IPS itself should limit the sweep rate to safe values in hardware. + +### Magnet won't respond to any setpoints + +Check remote mode is enabled. + +### Magnet system status reports something other than "normal" - e.g. "quenched" or "fault" + +This indicates a hardware fault. Inform instrument scientists that the magnet has a problem and has likely gone to zero field (even if the IPS still claims to be producing a non-zero field). Consult cryogenics for help. \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/Oxford-Instruments-Mercury-IPS.md b/doc/specific_iocs/cryogenics/Oxford-Instruments-Mercury-IPS.md new file mode 100644 index 000000000..30398863c --- /dev/null +++ b/doc/specific_iocs/cryogenics/Oxford-Instruments-Mercury-IPS.md @@ -0,0 +1,12 @@ +# Mercury IPS + +The Oxford Instruments Mercury IPS is a superconducting magnet power supply. It is the successor to the [IPS](Oxford-Instruments-IPS). Much of the information on the IPS wiki page also applies to the Mercury IPS. + +Note: although the Mercury IPS is the successor to the "old" IPS, cryogenics prefer to run the older IPS units as they are more reliable. + +## Hardware quirks + +The following faults can be seen when operating the magnet fully from the front panel, but it is likely that software will also run into the same conditions: + * The firmware will sometimes crash/freeze. To reset it, the whole power supply needs to be power-cycled. This is obviously undesirable for a magnet power supply, and cryogenics are chasing OI about this issue. It's not clear whether this issue is general to all Mercury IPS units or whether we have one faulty unit. + * The switch heater occasionally reports that it's ON when it's actually OFF + * The power supply reports a voltage of ~9000V which is incorrect (a sensible voltage for this power supply would be around ~8V while ramping) \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/SPC.png b/doc/specific_iocs/cryogenics/SPC.png new file mode 100644 index 000000000..b6d55caaa Binary files /dev/null and b/doc/specific_iocs/cryogenics/SPC.png differ diff --git a/doc/specific_iocs/cryogenics/Triton.md b/doc/specific_iocs/cryogenics/Triton.md new file mode 100644 index 000000000..863d017ad --- /dev/null +++ b/doc/specific_iocs/cryogenics/Triton.md @@ -0,0 +1,68 @@ +# Triton + +## Naming + +Technically, "Triton" refers to the gas handling apparatus. This is what IBEX and SECI talk to. When setting up a triton, scientists may refer to it as a "Kelvinox". This is the technical name for the dilution insert, which is controlled by the triton gas handling apparatus. + +To avoid confusion: all dilution fridges currently in use at ISIS are controlled by Triton gas handling systems, with one exception: the ICE fridge used on the muon beamlines. + +## Connection + +The Triton software runs on a dedicated PC that follows each fridge around. The IBEX control system makes a connection over TCP to issue commands to that software. + +Settings: +1. Hostname : For the definitive answer, type `hostname` at a command prompt on the Triton control PC. The list of Triton hostnames is available on the sharepoint. +1. TCP port : 33576 + +* If communication cannot be established and you are certain that the correct hostname has been entered, then try the IP address instead. This can be determined by typing `ipconfig` at a command prompt on the Triton control PC and looking for "IPv4 Address" in the output. +* Not all the Triton computers respond to a ping. +* Only one connection can be made at any time, additional connections are refused. + +There are various generations of Triton systems in use at ISIS. There are sometimes differences in the protocol between different versions of the software (see for example IBEX issue #3030). + +## Temperature control + +Temperature control in the triton systems is achieved by multiple cooling stages: +- Outer cryostat - usually a Eurotherm, ITC503 or Lakeshore. This cryostat will typically be able to cool the dilution fridge and sample to a few degrees Kelvin (1-5). +- Mixing chamber - this is where dilution cooling occurs, which brings the sample down towards base temperature (roughly 20-40mK). Dilution cooling only starts to work around 1K, hence the need for the outer cryostat. Dilution provides a constant amount of cooling power, so for temperature control the mixing chamber is fitted with a small heater which warms up the dilution to the desired temperature. +- There are a number of intermediate stages to bridge the gap between the outer cryostat and dilution. They are not always all present (depending on which fridge is used and experimental setup). In no particular order, there may be: J-T heat exchanger, 4K heat exchanger, sorb, still. The IOC monitors these temperatures if available from the Triton system, but they are not usually of interest to us (they are of interest to cryogenics). + +The "base" temperature refers to the lowest achievable temperature in a given setup. This will vary based on which fridge is used and the sample, but is typically in the region of 20-40mK. + +| Mode | Temperature range | Notes | +| --- | --- | --- | +| Base temperature | base (*) | In this mode the fridge is using dilution cooling without the heater on: the temperature drops towards base temperature. The heater is not used so PID settings and closed loop mode are irrelevant | +| Temperature control, in dilution | `base~1K` | In this mode the fridge is not using dilution cooling at all: the temperature is controlled by the outer cryostat. The heater is not used so PID settings and closed loop mode are irrelevant | + +(*) - The driver has a PID table which looks up appropriate PID settings and heater ranges from a file, based on the temperature setpoint readback. This can be set permanently via IOC macros (if entered manually, it will not be remembered on next IOC restart). + +**Note: the IOC does not bring a fridge into/out of dilution. This is a manual process that cryogenics group perform.** + +## Configuration + +The Triton IOC reads it's configuration (sensor <-> name mapping) from the triton control PC. However, the name <-> channel mapping is hard-coded in the triton IOC, which expects the following mapping: +- Stage 1 (PT1) : J-T heat exchanger +- Stage 2 (PT2) : 4K heat exchanger +- Sorb : Sorb +- Stil : Stil +- Mixing chamber : mixing chamber +- Cooling channel : **not read or used by IOC** + +This can be set using the UI in the diagram, you must leave the tab for the configurations to take effect + +![triton ui](triton_channel_settings.jpg) + +As of https://github.com/ISISComputingGroup/IBEX/issues/3993 this mapping should have been set on all of the fridges by cryogenics section. + +## Gotchas + +- Oxford software running on the remote PC can occasionally crash. If this happens, you need to VNC to the remote computer using the credentials on the passwords sharepoint page and manually restart the software. +- Some commands take a very long time to generate a reply (can be in the region of 10+ seconds before the reply comes through). This is particularly noticeable for setting values, especially setting closed loop mode on or off. +- [Older versions of Triton software only - see IBEX issue #3030] There are some indexing errors in the Oxford software. Therefore, if you want to read values relating to channel `T3`, you may need to query the device for some variables using `T3` as expected and for other variables using `T2`. +- Sample channel - if this comes back as `T0`, this indicates that it is running on a dedicated control channel, and is newer hardware. +- Beware of things coming back as strings when you expect numbers. For example many commands will respond with `NONE` or `NOT_FOUND` where you would usually expect a numeric response. +- Because the device responds to some commands very slowly, need to be quite careful about the overall command rate - ensure that the device can actually keep up with all the commands being sent. +- Switching "closed loop" on can change the setpoint temperature - I think it changes the setpoint to match the current temperature. This is usually not desirable. See https://github.com/ISISComputingGroup/IBEX/issues/4103 for more details. + * The workaround is to set the temperature setpoint, wait 500ms, change closed loop mode, wait 500ms, then set the temperature setpoint again. +- Sometimes the channel mapping will appear to be correct from the Oxford software, but will not report correctly from the remote interface. For example, the fridge may report it has no `SORB` channel, even if one is correctly configured. This is a bug in the oxford instruments software running on the triton control PC. To get out of this state, the triton software (on the OI pc) must be restarted. **Check with cryogenics before doing this!** diff --git a/doc/specific_iocs/cryogenics/comms1.jpg b/doc/specific_iocs/cryogenics/comms1.jpg new file mode 100644 index 000000000..7b39fc96c Binary files /dev/null and b/doc/specific_iocs/cryogenics/comms1.jpg differ diff --git a/doc/specific_iocs/cryogenics/comms2.jpg b/doc/specific_iocs/cryogenics/comms2.jpg new file mode 100644 index 000000000..2ea015459 Binary files /dev/null and b/doc/specific_iocs/cryogenics/comms2.jpg differ diff --git a/doc/specific_iocs/cryogenics/cryomagnet_circuit_diagram.PNG b/doc/specific_iocs/cryogenics/cryomagnet_circuit_diagram.PNG new file mode 100644 index 000000000..76cdadd62 Binary files /dev/null and b/doc/specific_iocs/cryogenics/cryomagnet_circuit_diagram.PNG differ diff --git a/doc/specific_iocs/cryogenics/cryosms/Cryogenic-SMS-PSU-design.md b/doc/specific_iocs/cryogenics/cryosms/Cryogenic-SMS-PSU-design.md new file mode 100644 index 000000000..c03169b1b --- /dev/null +++ b/doc/specific_iocs/cryogenics/cryosms/Cryogenic-SMS-PSU-design.md @@ -0,0 +1,96 @@ +# Cryosms PSU design + +These are used to power the HIFI Cryomag. There are four, one (higher max current) controls the main switched cryomagnet, the other three (lower max current) control the X, Y, and Z 'shim' coil magnets. They are capable of driving the main cryomagnet up to a field of ~4.9 Tesla. + +Magnet cool downs typically take several days, warm ups take a similar length of time. It usually sits at 3-4K, the temperature is monitored by a Keithley 2700. + +They are connected via USB to serial RS-232 directly to the NDXHIFI_CRYOMAG PC. + +All commands have a longhand and a shorthand version which can be found in the manual on our network share. The emulator has been made such that it can deal with the longhand or the shorthand, since it must be tested against the VI for compatibility, and the VI uses shorthand commands (vs the IOC which, in its current state in July 2019, uses longhand. This is likely to change). + +Commands can reply with a myriad of options. Some of these are time stamped, some are not. These are also listed in the manual on the network share. + + +## IOC + +An IOC exists for this PSU, named CRYOSMS. In order to prevent users from negligently sending commands to the PSU which could damage the cryomagnet (e.g. manually setting an incorrect ramp rate, sending values in wrong units), this IOC has a very non-standard communication system. Users are able to enter values into a large number of PVs as normal, however no use-facing setpoint PVs send commands to the device. Instead, writes triggered by users are sent to an asyn Port Driver, which handles the complex logic associated with ramping, handling quenches, etc. and feeds back values into a set of PVs with the HIDDEN access security group. Users are completely unable to interact or even view the contents of these PVs, the most they can do is confirm their existence. Finally, these hidden PVs send commands to the PSU through streamDevice, which is necessary due to the difficult syntax of certain responses. For reads from the device, the user-facing PVs interact directly with the PSU through streamDevice. + +## Operation + +The IOC will handle initialisation in asyn, where it will check various macros do determine whether certain modes should be enabled/disabled: + +All writes to the device will be disabled in any of these situations: + +- `T_TO_A` not supplied +- `MAX_CURR` not supplied +- `ALLOW_PERSIST` is on, but any of the subsequently required variables (`FAST_FILTER_VALUE, FILTER_VALUE, NPP, FAST_PERSISTENT_SETTLETIME, FAST_RATE`) are missing +- `USE_SWITCH` is on, but any of the subsequently required variables (`SWITCH_TEMP_PV, SWITHCH_HIGH, SWITCH_LOW, SWITCH_STABLE_NUMBER, HEATER_TOLERANCE, SWITCH_TIMEOUT, SWITCH_TEMP_TOLERANCE, HEATER_OUT`) are missing +- `USE_MAGNET_TEMP` is on, but any of the subsequently required variables (`MAGNET_TEMP_PV, MAX_MAGNET_TEMP, MIN_MAGNET_TEMP`) are missing +- `COMP_OFF_ACT` is on, but any of the subsequently required variables (`NO_OF_COMP, MIN_NO_OF_COMP, COMP_1_STAT_PV, COMP_2_STAT_PV`) are missing + +Additionally, the asyn driver will initialise various variables and send several commands to the PSU based on user provided macros: + +- The Tesla-Amps conversion rate will be set to T_TO_A +- The maximum allowed current will be set to MAX_CURR +- The maximum allowed voltage will be set to MAX_VOLT +- The heater output will be set to HEATER_OUT +- The ramp rate will be set to an appropriate value read from RAMP_FILE based on the current value of the field + +## Macros + +|Macro | Description | Defaults | +| - | - | - | +| MAX_CURR | Define the maximum current that is allowed | +| T_TO_A | The conversion value between Tesla and Amps for the system | +| MAX_VOLT | Set the maximum voltage for the system | +| WRITE_UNIT | Which unit to write to the PSU in | Amps +| DISPLAY_UNIT | Which unit will be used when displaying the value from the PSU | Gauss +| RAMP_FILE | Location of file containing ramp rates | +| ALLOW_PERSIST | Whether or not to allow setting of persistent values | No +| USE_SWITCH | Whether or not to monitor and set switches on and off | No +| SWITCH_TEMP_PV | PV address of switch temperature | +| SWITCH_HIGH | The value at which the switch will be considered warm | 3.7 +| SWITCH_LOW | The value at which the switch will be considered cold | 3.65 +| `SWITCH_STABLE_NUMBER | The number of readings past a threshold needed before the switch can be considered warm/cold | 10 +| HEATER_TOLERANCE | The tolerance between the magnet and lead currents before considering them as close enough to allow the leads to warm | 0.2 +| SWITCH_TIMEOUT | The time to allow for the switch to warm or cold after turning the heater on/off before considering there to be an error situation | 300 +| SWITCH_TEMP_TOLERANCE | needs clarification | +| HEATER_OUT | The heater output to be used | +| USE_MAGNET_TEMP | Whether to act if the Magnet Temperature is out of range | No +| MAGNET_TEMP_PV | The PV address of the magnet temperature | +| MAX_MAGNET_TEMP | Maximum allowed temperature of magnet | 5.5 +| MIN_MAGNET_TEMP | Temperature below which there is potentially an error reading temperature from the magnet | 1 +| COMP_OFF_ACT | Whether to act if magnet temperature is out of range | No +| NO_OF_COMP | Number of compressors in the system | 2 +| COMP_1_STAT_PV | The PV address for the status of the first compressor | +| COMP_2_STAT_PV | The PV address for the status of the second compressor | +| FAST_RATE | The ramp rate to use for the fast ramps | 0.5 +| FAST_PERSISTENT_SETTLETIME | The number of seconds to settle after a ramp fast to persist | 5 +| PERSISTENT_SETTLETIME | The number of seconds to settle after a ramp to persist | 60 +| FILTER_VALUE | The value to use to filter the target reached calculation | 0.1 +| FAST_FILTER_VALUE | The value to use to filter the target reached calculation when ramping fast to a target | 1 +| NPP | The npp value when calculating whether or not the target has been reached | + +It initialises and waits in a Ready state, depending on things like switch status, temperatures, magnet mode, settle times, etc. Ready means that it is ready to drive its field (up or down). It is dangerous to ramp the magnet too fast, so the IOC uses 'ramp tables' which contain field strength-ramp rate pairs. i.e. the magnet can safely ramp up to the field strength at the ramp rate associated with it. Any higher and you risk quenching. + +To prevent a quench, the IOC will, upon setting a setpoint, calculate the steps required to safely ramp to the new setpoint. It will then send pairs of values, one ramp rate and one ramp set point, which will start the PSU ramp. The ramp set point is always sent as a new mid setpoint (unless it is doing a ramp to 0). + +If all runs smoothly, the magnet will get up to the field you specify. + +## Queued State Machine + +The IOC makes use of a queued state machine, based upon the meta state machine library from boost. Possible states, events (things that prompt state changes) and actions (code to perform on given state transitions) are defined in this state machine, alongside a table of possible transitions and actions to perform on them. Inside the driver, a double-ended queue is used to store future transitions to be executed (e.g. when the device starts ramping it can queue the next event to be reaching the target). Within this queue individual events are stored inside variant objects from the boost variant library in order to preserve typing (boost msm fires events based on the typing of the object being passed to it). A separate thread is run by the driver which continually attempts to process the forwardmost item in the queue by applying a visitor to it which sends the event to the QSM and deletes it from the queue. + +### Pausing + +The PSU can pause in the middle of a ramp. This stops ramp generator activity in the PSU and immediately responds with an acknowledge message. Subsequent ramp status queries will inform the IOC that the PSU is paused and holding at a certain current/field strength. + +### Ramping Through 0 + +The field can switch polarity, depending on the direction of the current driving it. The PSU keeps track of this using a 'direction' which is either +, -, or 0. 0 is only used for Low Field Option mode, see the manual's Appendix N for more. + +When the polarity is switched, i.e. when driving the field from 3T to -3T, the PSU will first ramp to 0, then switch polarity (this shorts the terminals and causes a voltage excursion for 5ms), then carry as it would normally and select appropriate ramp rates as it drives the magnet up to field. + +## Quenching + +The magnet can quench if it is ramped too aggressively, or if its temperature drifts too high. If the PSU detects a quench, it report that it thinks the magnet has quenched. The IOC will interpret this and halt all queued actions. It will stay in this state until the IOC is restarted. \ No newline at end of file diff --git a/doc/specific_iocs/cryogenics/detailed_temp_board.jpg b/doc/specific_iocs/cryogenics/detailed_temp_board.jpg new file mode 100644 index 000000000..c31537bf4 Binary files /dev/null and b/doc/specific_iocs/cryogenics/detailed_temp_board.jpg differ diff --git a/doc/specific_iocs/cryogenics/front_panel.jpg b/doc/specific_iocs/cryogenics/front_panel.jpg new file mode 100644 index 000000000..97e827692 Binary files /dev/null and b/doc/specific_iocs/cryogenics/front_panel.jpg differ diff --git a/images/heliox.png b/doc/specific_iocs/cryogenics/heliox.png similarity index 100% rename from images/heliox.png rename to doc/specific_iocs/cryogenics/heliox.png diff --git a/doc/specific_iocs/cryogenics/hlm/HLM-Web-Server-deployment.md b/doc/specific_iocs/cryogenics/hlm/HLM-Web-Server-deployment.md new file mode 100644 index 000000000..d77e7630d --- /dev/null +++ b/doc/specific_iocs/cryogenics/hlm/HLM-Web-Server-deployment.md @@ -0,0 +1,106 @@ +# HLM Webserver deployment + +Deployment notes for [HLM View](https://github.com/ISISNeutronMuon/HLM_View), running with Apache + mod_wsgi on Windows Server 2012 R2. + + +#### Web App +To run with the Django development server using `python manage.py runserver 0.0.0.0:8000`: + +``` +1. Installed python-3.9.4-amd64 +2. Installed Git-2.31.1-64-bit +3. (Optional) Installed npp.7.9.5.Installer (Notepad++) +4. Cloned the HLM View repo in C:/ +5. Created virtual environment: C:\HLM_View\venv and activated it (Scripts/activate.bat) +6. Installed requirements: cd C:\HLM_View and pip install -r requirements.txt + +7. Manually copy over static files that are not version controlled + (from https://github.com/ISISNeutronMuon/HLM_View/files/6404513/static.zip) + - secret_key.txt in .project\ + - jquery-3.5.1.min.js in HLM_View\static\js + - plugins\ in HLM_View\static + - bootstrap\ in HLM_View\static + +8. Update the db connection at DATABASES in settings.py +9. Use an online Django secret key generator and create a `secret_key.txt` file with the key inside in the HLM_View\project folder. +10. Run python manage.py runserver (in root with the venv activated)- works with debug=True, + Note: If set to false Django won't provide static files anymore. + This step is just for testing as Django dev server should never be used in production. +``` + +#### Grafana Dashboard +To set up the Grafana dashboard: +``` +1. Install grafana-7.5.5.windows-amd64 +2. Set up Grafana admin credentials (Access details) +3. Import dashboard with .json file from HLM_View\Grafana +4. Configure the Grafana Data Source (MySQL DB) +5. In Grafana/conf, copy sample.ini , rename to custom.ini +6. Then set to allow_embedding and anonymous access (for iframes) +7. Restart Grafana service to apply new config +8. Updated iframe links in the HLM View templates (details.html, building.html) +from localhost:3000 to new address +``` +[HLM_View\Grafana](https://github.com/ISISNeutronMuon/HLM_View/blob/master/Grafana/Object%20Measurements.json) + +#### Web Server +To run with a production web server using Apache + mod_wsgi: +``` +Used resources: + https://github.com/Johnnyboycurtis/webproject#apache-and-mod_wsgi + https://github.com/GrahamDumpleton/mod_wsgi/blob/develop/win32/README.rst + https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/modwsgi/ + https://montesariel.com/blog/post-3 + https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/ + +1. Added new hosts in project/settings.py ALLOWED_HOSTS, set DEBUG to False, check the Django deployment checklist + +2. Install VC_redist.x64.exe +3. Download httpd-2.4.47-win64-VS16.zip (ApacheLounge), extract in C:\Apache24 +4. Run "C:\Apache24\bin\httpd.exe" then go to localhost 80 to confirm it works + +5. pip install mod_wsgi, error: MVS C++ 14.0 required +6. Download vs_buildtools__1321439799.1619704707, and install universal + C++ packages +7. Successfully installed mod-wsgi-4.7.1 +8. CMD: run mod_wsgi-express module-config (this will create mod_wsgi.cp39-win_amd64.pyd +in HLM_View/venv/Lib/site-packages/mod_wsgi/server) + +9. Edit "C:\Apache24\conf\httpd.conf": + Set ServerName to localhost:80 + + At the end of http.conf, added: + + # Django Project + LoadFile "C:/Instrument/Apps/Python/Python39/python39.dll" + LoadModule wsgi_module "C:/HLM_View/venv/Lib/site-packages/mod_wsgi/server/mod_wsgi.cp39-win_amd64.pyd" + WSGIPythonHome "C:/HLM_View/venv;C:/Instrument/Apps/Python/Python39/" + WSGIScriptAlias / "C:/HLM_View/project/wsgi.py" + WSGIPythonPath "C:/HLM_View/venv/Lib/site-packages/;C:/HLM_View/" + + + + Require all granted + + + + Alias /static "C:/HLM_View/static/" + + Require all granted + + +10. Run "C:\Apache24\bin\httpd.exe" and check that http:/// works. +Note: Apache logs can be found in "C:\Apache24\logs" + +11. Fix a Python bug that causes an error 500 every time a query is made to the database. + Edit the __init__.py file in project-root\venv\Lib\site-packages\asgiref by adding the following: + # PATCH that fixes a Python Bug: + import sys + import asyncio + + if sys.platform == "win32" and sys.version_info >= (3, 8, 0): + asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) + + +12. Install Apache httpd.exe as a service: C:\Apache24\bin>httpd.exe -k install -n "HLM View Web Server" +``` + diff --git a/images/mercury_heliox_ramp_example.PNG b/doc/specific_iocs/cryogenics/mercury_heliox_ramp_example.PNG similarity index 100% rename from images/mercury_heliox_ramp_example.PNG rename to doc/specific_iocs/cryogenics/mercury_heliox_ramp_example.PNG diff --git a/doc/specific_iocs/cryogenics/triton_channel_settings.jpg b/doc/specific_iocs/cryogenics/triton_channel_settings.jpg new file mode 100644 index 000000000..e0b5f3c68 Binary files /dev/null and b/doc/specific_iocs/cryogenics/triton_channel_settings.jpg differ diff --git a/doc/specific_iocs/dae/DAE-Live-View.md b/doc/specific_iocs/dae/DAE-Live-View.md new file mode 100644 index 000000000..90e72c264 --- /dev/null +++ b/doc/specific_iocs/dae/DAE-Live-View.md @@ -0,0 +1,53 @@ +# Live View + +## normal integral 2D detector live view + +The DAE, when in [event mode](#dae_event_histogram_modes), can serve up an [AreaDetector](https://github.com/areaDetector/ADCore) image as a live view of the neutron data. This can be viewed using an ImageJ plugin or as an Intensity plot in CSS. + +Using the live view device screen you should be able to add a view with the DET macro as 1 or 2 + +You need to run the DAE in event mode, this will need an event mode wiring table and also defining a second time regime in the time channels section. + +Live view PVs have a root at `%MYPVPREFIX%DAE:AD1` and `%MYPVPREFIX%DAE:AD2` for the two detector case + +You first need to tell it the spectra to use for the live view e.g. +``` +caput %MYPVPREFIX%DAE:AD1:INTG:SPEC:START:SP 11 +caput %MYPVPREFIX%DAE:AD1:INTG:SPEC:SIZEX:SP 80 +caput %MYPVPREFIX%DAE:AD1:INTG:SPEC:SIZEY:SP 80 +``` +will tell it to start from spectrum 11 and use consecutive spectra in an 80x80 grid. Setting `SPEC:SIZEX` sets the underlying area detector `SizeX` and `MaxSizeX`. Though `SizeX` can be smaller, we do binning and size changes etc using a region of interest plugin later so they should always be the same here. + +Live view is disabled by default, it is enabled with +``` +caput %MYPVPREFIX%DAE:AD1:INTG:ENABLE:SP 1 +``` +depending on the number of spectra, MAX_ARRAY_BYTES may need adjusting + +## Live view in histogram mode + +It is possible to use liveview in [histogram mode](#dae_event_histogram_modes), but this is slower and puts more load on the DAE unless a separate integrating detector card is used (such as on LOQ or SANS2D). Use the parameters to define spectrum numbers etc. as above and then enable this way with: +``` +caput %MYPVPREFIX%DAE:AD1:INTG:SPEC:MODE:SP 1 +``` + +## 1D detector with TOF axis giving 2D view + +In this case Y is the spectrum number and X is the time of flight bin. + +* `SIZEY` is the number of spectra in the 1D detector +* `SIZEX` is (number_of_time_channels + 1) + - `number_of_time_channels` is shown on the DAE Information tab as `Time Channels` +* `SPEC:START:SP` is number_of_dae_spectra_to_skip_to_get_to_1D_detector_spectra + +Typically `number_of_dae_spectra_to_skip_to_get_to_1D_detector_spectra` will be the number of monitor spectra if they occur first in the spectra order, but on a multiple detector instrument it could be larger. + +You then need to put the live view into "TOFChannel" rather than "TOFSummed" mode either on the OPI or with: +``` +caput %MYPVPREFIX%DAE:AD1:INTG:DATAMODE:SP 1 +``` +`number_of_time_channels` is displayed on the DAE Run Information tab as "time channels". Note that this value will change if they ever change their time channel boundaries. + +## error on profile picture about not enough data for `dataWidth*height` + +This is caused by the array pv being viewed not being large enough to contain all the data, this is determined by `NELEMENTS` in `liveview.cmd` in `ioc/isisdae`. It is set by the macro `LIVEVIEW_NELEMENTS` it needs to be at least `sizeX` x `sizeY`. In some cases `EPICS_CA_MAX_ARRAY_BYTES` could need increasing, but we have set this quite large now (and EPICS7 did away with it), so it would be gateway/pcaspy/CSS where it might have an effect. \ No newline at end of file diff --git a/doc/specific_iocs/dae/DAE-Normal-Log.md b/doc/specific_iocs/dae/DAE-Normal-Log.md new file mode 100644 index 000000000..32620affa --- /dev/null +++ b/doc/specific_iocs/dae/DAE-Normal-Log.md @@ -0,0 +1,303 @@ +# `isisicp` Standard startup log + +``` +2017-04-20T10:47:54 ################################################################################ + +2017-04-20T10:47:54 ISIS ICP Starting from c:\labview modules\dae\service\x64\Release at 2017-04-20T10:47:54 + +2017-04-20T10:47:54 SVN revision 842 (2010/06/11 11:37:02, Modified) + +2017-04-20T10:49:42 SELOGGER with ISISDB schema rev 37 using SQLITE 3.6.22 + +2017-04-20T10:49:42 There is 81 percent of memory in use. +2017-04-20T10:49:42 There are 4193848 total Kbytes of physical memory. +2017-04-20T10:49:42 There are 795580 free Kbytes of physical memory. +2017-04-20T10:49:42 There are 8590636 total Kbytes of paging file. +2017-04-20T10:49:42 There are 2411068 free Kbytes of paging file. +2017-04-20T10:49:42 There are 8589934464 total Kbytes of virtual memory. +2017-04-20T10:49:42 There are 8589814888 free Kbytes of virtual memory. +2017-04-20T10:49:42 There are 0 free Kbytes of extended memory +2017-04-20T10:49:42 Setting DAE type to NEUTRON on NDXHRPD +2017-04-20T10:49:42 XML key "UseFullInstNameForFiles" not found +2017-04-20T10:49:42 Run digits = 5 +2017-04-20T10:49:42 Opening connection to VISA resource ISISDAE0 + +2017-04-20T10:49:42 Calling NIVisa::reinit() +2017-04-20T10:49:42 ScanBus skipped - set ISISVME_SCANBUS environment variable to enable +2017-04-20T10:49:42 Endian workaround DISABLED, blocks transfers DISABLED +2017-04-20T10:49:42 This is a VXI DAE +2017-04-20T10:49:42 This Card Supports Fast (Fermi) Chopper Vetos +--- DAE2 card with address/position/number = 0 --- +Firmware register: 0x41030403 +Hardware type: ENVIRONMENT CARD (Configured from detector card) +Firmware sub version = 3 +Firmware minor version = 4 +Firmware hardware version = 3 + +This is an ENVIRONMENT card +RC register value = 0x110008 +Status: Stopped +Frame Sync: External +Delayed start: disabled +Frame sync delay: 29996 +FIFO Veto is ENABLED (vetoed 0) frames +SMP (chopper) Veto is ENABLED (vetoed 0) frames +Fermi Chopper Veto is DISABLED (counted 0) frames +TS2 Pulse Veto is DISABLED (counted 0) frames +ISIS 50 Hz Veto is DISABLED (counted 0) frames +External Veto 0 is ENABLED (vetoed 0) frames +External Veto 1 is DISABLED (counted 0) frames +External Veto 2 is ENABLED (vetoed 0) frames +External Veto 3 is DISABLED (counted 0) frames +GOOD/RAW frames: 0/0 +GOOD/RAW uamph: 0/0 +Period Type: Software + +2017-04-20T10:49:42 Highest DAE2 spectrum: 2 +Number of position: 2 +MPOS: 0 to 0 +MODN: 0 to 1 + +2017-04-20T10:49:42 Recreated DAE1 specmap with DAE1 spec range 1 to 2 + +2017-04-20T10:49:43 Unable to determine card memory size; assuming 128Mb +2017-04-20T10:49:43 This Card Supports HARDWARE periods +This Card Supports VETO logging +This Card supports delayed frame sync +--- DAE2 card with address/position/number = 1 --- +Firmware register: 0x82060105 +Hardware type: DETECTOR CARD (Neutron, 16 DIM) +Firmware sub version = 5 +Firmware minor version = 1 +Firmware hardware version = 6 + +This is a DECTECTOR card with 128 MB memory +Highest DAE2 spectrum used = 2 +14661 time channels +SOFTWARE periods: number of periods = 1 +Total counts register = 0 +Veto for Dim0 is ENABLED, CLEAR +Veto for Dim1 is ENABLED, CLEAR +Veto for Dim2 is ENABLED, CLEAR +Veto for Dim3 is ENABLED, CLEAR +Veto for Dim4 is ENABLED, CLEAR +Veto for Dim5 is ENABLED, CLEAR +Veto for Dim6 is ENABLED, CLEAR +Veto for Dim7 is ENABLED, CLEAR +Veto for Dim8 is ENABLED, CLEAR +Veto for Dim9 is ENABLED, CLEAR +Veto for Dim10 is ENABLED, CLEAR +Veto for Dim11 is ENABLED, CLEAR +Veto for Dim12 is ENABLED, CLEAR +Veto for Dim13 is ENABLED, CLEAR +Veto for Dim14 is ENABLED, CLEAR +Veto for Dim15 is ENABLED, CLEAR +Delayed Frame synch = 0 +POSLUT checksum = 817e75953d48998d20aebc226ce44c8 + +2017-04-20T10:49:43 Highest DAE2 spectrum: 191 +Number of position: 976 +MPOS: 0 to 376 +MODN: 5 to 11 + +2017-04-20T10:49:43 Recreated DAE1 specmap with DAE1 spec range 1 to 191 + +2017-04-20T10:49:44 Unable to determine card memory size; assuming 128Mb +2017-04-20T10:49:44 This Card Supports HARDWARE periods +This Card Supports VETO logging +This Card supports delayed frame sync +--- DAE2 card with address/position/number = 2 --- +Firmware register: 0x82060105 +Hardware type: DETECTOR CARD (Neutron, 16 DIM) +Firmware sub version = 5 +Firmware minor version = 1 +Firmware hardware version = 6 + +This is a DECTECTOR card with 128 MB memory +Highest DAE2 spectrum used = 191 +14661 time channels +SOFTWARE periods: number of periods = 1 +Total counts register = 0 +Veto for Dim0 is ENABLED, CLEAR +Veto for Dim1 is ENABLED, CLEAR +Veto for Dim2 is ENABLED, CLEAR +Veto for Dim3 is ENABLED, CLEAR +Veto for Dim4 is ENABLED, CLEAR +Veto for Dim5 is ENABLED, CLEAR +Veto for Dim6 is ENABLED, CLEAR +Veto for Dim7 is ENABLED, CLEAR +Veto for Dim8 is ENABLED, CLEAR +Veto for Dim9 is ENABLED, CLEAR +Veto for Dim10 is ENABLED, CLEAR +Veto for Dim11 is ENABLED, CLEAR +Veto for Dim12 is ENABLED, CLEAR +Veto for Dim13 is ENABLED, CLEAR +Veto for Dim14 is ENABLED, CLEAR +Veto for Dim15 is ENABLED, CLEAR +Delayed Frame synch = 0 +POSLUT checksum = 6a385651302654f3d7429d1656ce8ed0 + +2017-04-20T10:49:44 Highest DAE2 spectrum: 256 +Number of position: 1536 +MPOS: 0 to 255 +MODN: 0 to 5 + +2017-04-20T10:49:44 Recreated DAE1 specmap with DAE1 spec range 1 to 256 + +2017-04-20T10:49:45 Unable to determine card memory size; assuming 128Mb +2017-04-20T10:49:45 This Card Supports HARDWARE periods +This Card Supports VETO logging +This Card supports delayed frame sync +--- DAE2 card with address/position/number = 3 --- +Firmware register: 0x82060105 +Hardware type: DETECTOR CARD (Neutron, 16 DIM) +Firmware sub version = 5 +Firmware minor version = 1 +Firmware hardware version = 6 + +This is a DECTECTOR card with 128 MB memory +Highest DAE2 spectrum used = 256 +14661 time channels +HARDWARE periods: number of DAQ periods = 1 +Current DAQ period number = 0 +Current DAQ period size (words) = 0 +Total counts register = 0 +Veto for Dim0 is ENABLED, CLEAR +Veto for Dim1 is ENABLED, CLEAR +Veto for Dim2 is ENABLED, CLEAR +Veto for Dim3 is ENABLED, CLEAR +Veto for Dim4 is ENABLED, CLEAR +Veto for Dim5 is ENABLED, CLEAR +Veto for Dim6 is ENABLED, CLEAR +Veto for Dim7 is ENABLED, CLEAR +Veto for Dim8 is ENABLED, CLEAR +Veto for Dim9 is ENABLED, CLEAR +Veto for Dim10 is ENABLED, CLEAR +Veto for Dim11 is ENABLED, CLEAR +Veto for Dim12 is ENABLED, CLEAR +Veto for Dim13 is ENABLED, CLEAR +Veto for Dim14 is ENABLED, CLEAR +Veto for Dim15 is ENABLED, CLEAR +Delayed Frame synch = 0 +POSLUT checksum = 8bc9ea7e4e6221c9f8844dcc39d5fc + +2017-04-20T10:49:45 Highest DAE2 spectrum: 24 +Number of position: 72 +MPOS: 0 to 15 +MODN: 0 to 7 + +2017-04-20T10:49:45 Recreated DAE1 specmap with DAE1 spec range 1 to 24 + +2017-04-20T10:49:45 Unable to determine card memory size; assuming 128Mb +2017-04-20T10:49:45 This Card Supports HARDWARE periods +This Card Supports VETO logging +This Card supports delayed frame sync +--- DAE2 card with address/position/number = 4 --- +Firmware register: 0x82060105 +Hardware type: DETECTOR CARD (Neutron, 16 DIM) +Firmware sub version = 5 +Firmware minor version = 1 +Firmware hardware version = 6 + +This is a DECTECTOR card with 128 MB memory +Highest DAE2 spectrum used = 24 +14661 time channels +HARDWARE periods: number of DAQ periods = 1 +Current DAQ period number = 0 +Current DAQ period size (words) = 0 +Total counts register = 0 +Veto for Dim0 is ENABLED, CLEAR +Veto for Dim1 is ENABLED, CLEAR +Veto for Dim2 is ENABLED, CLEAR +Veto for Dim3 is ENABLED, CLEAR +Veto for Dim4 is ENABLED, CLEAR +Veto for Dim5 is ENABLED, CLEAR +Veto for Dim6 is ENABLED, CLEAR +Veto for Dim7 is ENABLED, CLEAR +Veto for Dim8 is ENABLED, CLEAR +Veto for Dim9 is ENABLED, CLEAR +Veto for Dim10 is ENABLED, CLEAR +Veto for Dim11 is ENABLED, CLEAR +Veto for Dim12 is ENABLED, CLEAR +Veto for Dim13 is ENABLED, CLEAR +Veto for Dim14 is ENABLED, CLEAR +Veto for Dim15 is ENABLED, CLEAR +Delayed Frame synch = 0 +POSLUT checksum = 43557b38a98c1d6b91c2f490c3b9421d + +2017-04-20T10:49:45 Card 0 is present on dae 0 (ISISDAE0)(Environment card) +Card 1 is present on dae 0 (ISISDAE0)(Neutron Detector card) +Card 2 is present on dae 0 (ISISDAE0)(Neutron Detector card) +Card 3 is present on dae 0 (ISISDAE0)(Neutron Detector card) +Card 4 is present on dae 0 (ISISDAE0)(Neutron Detector card) + +2017-04-20T10:49:45 XML key "DAEDevice1" not found +2017-04-20T10:49:45 CRPT "c:\data\current.run" size 65414 KB +2017-04-20T10:49:45 CRPT version 2 ($Revision: 827 $) +2017-04-20T10:49:45 CRPT data "c:\data\data.run" size 781250 KB +2017-04-20T10:49:46 XML key "AlertEmail" not found +2017-04-20T10:49:46 XML key "UAmpScale" not found +2017-04-20T10:49:46 XML key "CompressionLevel" not found +2017-04-20T10:49:46 XML key "CompressionBlockSize" not found +2017-04-20T10:49:46 Setting up DAE +2017-04-20T10:49:46 Highest dae, detector card, crate number used = 0, 4, 4 +2017-04-20T10:49:46 Number of time regimes = 1 +2017-04-20T10:49:46 DAE memory used = 19 Mb +2017-04-20T10:49:46 Number of periods (daq, total) = (1, 1) +2017-04-20T10:49:46 Resetting DAE - ignore veto counter values as not yet cleared +2017-04-20T10:49:46 SMP vetoed frames = 0 +Externally vetoed frames = 0 +FIFO vetoed frames = 0 +TS2 pulse vetoed frames = 0 +ISIS not at 50Hz vetoed frames = 0 +Fermi chopper vetoed frames = 0 +TOTAL VETOED FRAMES from all sources = 0 + +2017-04-20T10:49:46 RELOAD TABLES ONLY requested - DAE memory not zeroed +2017-04-20T10:49:46 Programming Time channels +2017-04-20T10:49:46 Global Frame sync delay = 29996 us +2017-04-20T10:49:46 Electronics delay = 7 clock pulses +2017-04-20T10:49:46 Setting 14661 time channels from 3.781250 to 100003.781250 us +2017-04-20T10:49:46 Setting FS delay on DC 1 to 0 us +2017-04-20T10:49:46 Setting FS delay on DC 2 to 0 us +2017-04-20T10:49:46 Setting FS delay on DC 3 to 0 us +2017-04-20T10:49:46 Setting FS delay on DC 4 to 0 us +2017-04-20T10:49:46 Programming POSLUT +2017-04-20T10:49:46 Card: 1 DAE2 Highest: 2 DAE1 low: 64 DAE1 high: 65 NPOS: 2 +MPOS: 0 to 0 +MODN: 0 to 1 + +2017-04-20T10:49:46 +2017-04-20T10:49:46 There is a gap of total size 3 in the DAE1 spectra on the card +Card: 2 DAE2 Highest: 191 DAE1 low: 0 DAE1 high: 193 NPOS: 1280 +MPOS: 0 to 383 +MODN: 4 to 11 + +2017-04-20T10:49:46 +2017-04-20T10:49:46 Card: 3 DAE2 Highest: 256 DAE1 low: 66 DAE1 high: 321 NPOS: 1536 +MPOS: 0 to 255 +MODN: 0 to 5 + +2017-04-20T10:49:46 +2017-04-20T10:49:46 Card: 4 DAE2 Highest: 24 DAE1 low: 322 DAE1 high: 345 NPOS: 72 +MPOS: 0 to 15 +MODN: 0 to 7 + +2017-04-20T10:49:46 +2017-04-20T10:49:46 Software periods enabled +2017-04-20T10:49:46 (setting period card, if present, to 1 hardware period of 15000 frames) +2017-04-20T10:49:46 No period card present +2017-04-20T10:49:46 Setting vetos +2017-04-20T10:49:46 Enabling FIFO veto +2017-04-20T10:49:46 Enabling SMP veto +2017-04-20T10:49:46 Enabling EXTERNAL veto 0 +2017-04-20T10:49:46 Disabling EXTERNAL veto 1 +2017-04-20T10:49:46 Enabling EXTERNAL veto 2 +2017-04-20T10:49:46 Disabling EXTERNAL veto 3 +2017-04-20T10:49:46 Disabling Fermi Chopper veto +2017-04-20T10:49:46 Disabling TS2 Pulse veto +2017-04-20T10:49:46 Disabling ISIS 50Hz veto +2017-04-20T10:49:46 *** +*** ISIS ICP STARTUP COMPLETE 2017-04-20T10:49:46 +*** +``` diff --git a/doc/specific_iocs/dae/DAE-PVs.md b/doc/specific_iocs/dae/DAE-PVs.md new file mode 100644 index 000000000..e8358ec0c --- /dev/null +++ b/doc/specific_iocs/dae/DAE-PVs.md @@ -0,0 +1,16 @@ +# DAE Spectra + +The DAE IOC serves spectra information (data collected in the Data Acquisition Electronics hardware) via the following PV format +``` +$(P)DAE:SPEC:$(PERIOD):$(SPEC):X +$(P)DAE:SPEC:$(PERIOD):$(SPEC):Y +$(P)DAE:SPEC:$(PERIOD):$(SPEC):YC +$(P)DAE:SPEC:$(PERIOD):$(SPEC):C +``` +Where SPEC is the spectrum number, PERIOD the period, X is the time-of-flight axis values, Y is the counts / microsecond (i.e. divided by histogram time bin width), YC is just counts in each bin (i.e. not divided by time), and C is total counts for that spectrum (summed over all time-of-flight bins). So X, Y and YC are arrays and C is a single number. X represents time bin _centres_, if there are `N` histogram time bins there will be `N` values in `Y`, `YC` and `X`. The size of the waveform array can be determined from the `.NORD` PV field e.g. `X.NORD` (note: the NeXus raw data file stores bin _boundaries_ rather than centres, so the time_of_flight axis (equivalent to X) will be size `N+1` rather than `N` in the data file. Mantid will handle this transparently for you when it loads the file). + +A spectrum number exists for every detector (pixel) and monitor. In addition the scientist can associate a "monitor number" (starting at 1 and increasing sequentially) with each spectrum number that is associated with a monitor. The monitor data can then be accessed via additional PVs that have the same general format as above but used the prefix `$(P)DAE:MON:$(PERIOD):$(MON_NUM)`. So for example if monitor number 1 is spectrum number 1234 then the counts per unit time can be accessed as either `$(P)DAE:SPEC:$(PERIOD):1234:Y` or `$(P)DAE:MON:$(PERIOD):1:Y`. There are no corresponding `:DET:` names for detectors, they are always accessed as `:SPEC:`. The `:MON:` mechanism is a convenient alias for monitors, but provides no additional information to using `:SPEC:`. The underlying spectrum number for a monitor can be obtained via `$(P)DAE:MON:$(PERIOD):1:S`. Some instruments can change their detector setup between experiments and depending on how they have assigned spectrum numbers this may result in a different `:SPEC:` being needed for each monitor than previously. So using `:MON:` syntax to access monitors is usually safer. + +For most instruments `$(PERIOD)` will always be `1`, however some instruments scan their devices and detector/monitor data at each scan point can be accessed via using different values of `$(PERIOD)`. SANS, reflectometry and muon instruments regularly use multiple periods, most other instruments only occasionally if at all. If the instrument is currently running in period mode then the PV `$(P)DAE:NUMPERIODS` will have a value greater than `1` and `$(PERIOD)` can then have values from `1` to `NUMPERIODS` representing the `NUMPERIODS` scan points. + +Monitors and detectors are stored separately in the NeXus raw data (.nxs) file and when this is read into Mantid a separate workspace is created for each. If periods have been used, then a "workspace group" is created. Workspace indexes always start at 0, the actual spectrum/monitor number will be stored in one of the workspace axes. As monitor numbers always start at 1 and increase numerically, index 0 of the monitor workspace will always be monitor number 1 etc. The same does not apply to the detector workspace i.e. index 0 there is not guaranteed to be spectrum 1 as if spectrum 1 is a monitor it will not be present in that workspace. diff --git a/doc/specific_iocs/dae/DAE-Pre-and-Post-commands.md b/doc/specific_iocs/dae/DAE-Pre-and-Post-commands.md new file mode 100644 index 000000000..4cfb7d3cf --- /dev/null +++ b/doc/specific_iocs/dae/DAE-Pre-and-Post-commands.md @@ -0,0 +1,15 @@ +# DAE Pre & Post commands + +It is possible to get additional actions to be carried out by DAE IOC when commands such as begin/end/pause/resume/abort are executed. These actions are typically used to do things like open/close an intermediate (fast) shutter. A `PRE` action is executed before the command and a `POST` after it has run. Whether the IOC waits for a PRE/POST to complete before the next action is controlled by whether `PP` or `CA` is passed as part of configuration, `CA` will wait for a completion callback and `PP` will just initiate processing and then carry on to start the next operation. The PV specified for PRE/POST will have the number 0 written to it for a PRE action and 1 for a POST action, often you will specify the `.PROC` field of a PV so just processing is initiated, but you could specify e.g. the `.VAL`, or another field such as `.A` of a calcout record, and use the 0/1 value to determine the action to take. + +The PV links to use during PRE/POST are set by macros, either in the configuration or in `globals.txt`. They have names like `PRE_BEGIN_1` and `POST_BEGIN_1`, to specify then in globals.txt you would add something like +``` +ISISDAE_01__PRE_BEGIN_1=SOME:PV:TO:ACCESS.PROC CA +``` +Note the `CA` added at the end to request a channel access put, which means the DAE IOC will wait for processing of the PV to finish before executing the main command (begin in this case). As mentioned above `PP` would request processing but not wait, but `PP` is only valid for PVs in the same IOC and it is unlikely the PV of interest will be inside the DAE IOC. For PVs not in the DAE IOC, a channel access put will be done anyway and whether the record processes will be determined by properties of the field written to. I've not determined yet whether adding `CA` to a link that refers to a PV already outside the DAE IOC make any difference to Wait behaviour as it will already be a channel access link, but if you wish this behaviour it is safer to add `CA`. + +## genie python interface + +By default IOC based PRE/POST commands are executed, however in genie python you can request that they are not by passing `prepost=False` to commands like `begin` e.g. `g.begin(prepost=False)`. This shortcuts some of the logic, normally it would write to the `$(P)DAE:BEGINRUN` PV which does a fanout to run: (PRE, main command, POST) with the main command executed via a PV with `_` appended e.g. `$(P)DAE:BEGINRUN_`. Thus if you write to `$(P)DAE:BEGINRUN_` PV instead of to `$(P)DAE:BEGINRUN` you will avoid running PRE and POST but still start the run. As well as begin, end/pause/resume/abort all follow this same scheme. + +As well as server side/DAE IOC based PRE/POST, genie python can have client side pre-post functions defined e.g. via`set_begin_precmd()`. Execution of these functions is not skipped with `prepost=False`, instead the `prepost` argument is passed to these functions to allow them to decide what to do. \ No newline at end of file diff --git a/doc/specific_iocs/dae/DAE-Trouble-Shooting-corrupted-max-database.PNG b/doc/specific_iocs/dae/DAE-Trouble-Shooting-corrupted-max-database.PNG new file mode 100644 index 000000000..550b6e33f Binary files /dev/null and b/doc/specific_iocs/dae/DAE-Trouble-Shooting-corrupted-max-database.PNG differ diff --git a/doc/specific_iocs/dae/DAE-Trouble-Shooting.md b/doc/specific_iocs/dae/DAE-Trouble-Shooting.md new file mode 100644 index 000000000..bd3045fe2 --- /dev/null +++ b/doc/specific_iocs/dae/DAE-Trouble-Shooting.md @@ -0,0 +1,503 @@ +# DAE Troubleshooting + +**Note:** If your DAE is not starting up correctly and you are struggling to see what the error is while, it might be worth running the IOC via `/ioc/ISISDAE/.../runIOC.bat` as this will give you additional messages compared to running from `ProcServ`. + +### Restarting the DAE in IBEX + +This IOC does not appear in the normal IOC restart list in the IBEX client, first open an EPICS terminal + +`...\EPICS\epicsterm.bat` + +then run + +`console -M localhost ISISDAE_01` + +and when connected press `Ctrl-x` once, you should see some restart messages from the IOC. Quit the EPICS term. + +### invalid tcb start + +It is likely that you are in a muon configuration for the DAE but using a neutron tcb file or vice versa. + +Either change the [tcb file](#dae_configuration) you are using or do the following steps to change the DAE type: + +1. Change the DAE type of your icp_config.xml (in EPICS/ICP_Binaries) to the correct value (1 for DAE2 neutron, 2 for DAE2 muon, 3 for DAE3 neutron, 4 for DAE3 muon). +2. In the same directory edit the isisicp.properties file to use either neutron or muon for `isisicp.simulation.detcards.type` +3. Restart the ISISDAE using console in an EPICS terminal and end the isisicp task in the task manager (the ISISDAE should autorestart it) + +### DAE switches from processing to Unknown and never goes into SetUp / Run can not be ended + +See [here](#webdashboard_troubleshooting_instrument_page_not_working) + +*DAE2* + +This issue has been observed on LARMOR and TOSCA, accompanied by the following error messages continuously being logged to `C:\data\log\icp-.log`: + +``` +2017-04-13 20:52:14 NIVISA: Error "Could not perform operation because of I/O error." (code 0xbfff003e) returned from "viMoveIn32" while transferring 1001 items at VME address 0x18000fa4 +2017-04-13 20:52:15 NIVISA: Error "Could not perform operation because of I/O error." (code 0xbfff003e) returned from "viMoveIn32" while transferring 1001 items at VME address 0x18000fa4 +2017-04-13 20:52:16 NIVISA: Error "Could not perform operation because of I/O error." (code 0xbfff003e) returned from "viMoveIn32" while transferring 1001 items at VME address 0x18000fa4 +2017-04-13 20:52:17 (0) NIVisa: [Warning] (NIVisa::retryBlock) retryBlock: 1001 items from address 0x18000fa4 +2017-04-13 20:52:17 (0) NIVisa: [Information] (NIVisa::reinit) Calling NIVisa::reinit() +2017-04-13 20:52:17 Calling NIVisa::reinit() +2017-04-13 20:52:17 Endian workaround DISABLED, blocks transfers DISABLED +2017-04-13 20:52:17 This is a VXI DAE +``` +This was resolved by power cycling the DAE followed by stopping the visa server and running `resman`. Instructions how to reset the DAE can be found in [these slides](https://stfc365.sharepoint.com/:p:/r/sites/ISISExperimentControls/_layouts/15/Doc.aspx?sourcedoc=%7BFB34A9F9-48C9-4590-A6B0-FDAE2EEA38B0%7D&file=ISISICP%20and%20DAE.pptx&action=edit&mobileredirect=true&DefaultItemOpen=1). + +*DAE3* + +On a DAE3 machine a vendor network library is used rather than NI Visa and the equivalent sorts of errors will have `Qx` or `Quixtream` prefixes. Access from the ISISICP is via the network, so there is no intermediate service/server to restart. Usually the ISISICP will retry failed connections, but check with electronics if there are repeated failures. You can try restarting ISISICP in case the vendor library needs a reload itself. An example of the error message is: + +``` +Qxtrm_channel::RDMARead failed rdma2 address 0x40010 nbytes 4(Quixtream: The timeout period on this channel expired before the transfer commenced. Channel status: Transfer failed. Data packet not received before timeout. ) +``` + +In general if you see an error like this or starting with `NIVISA` you should restart the DAE, then [contact electronics](https://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Contact%20details%20for%20other%20groups.docx). + +### No log files are produced in `c:\data` even though blocks are set to log. +The reason may be because the isisicp program that writes the datafile hasn't been configured to read values from MySQL for EPICS - the old SECI program used a different route. This is a one off change needed for SECI -> IBEX migration. In `C:\LabVIEW Modules\dae\isisicp.properties` set `isisicp.epicsdb.use = true` to enable this. You will need to restart the `isisicp` process for this to take effect. To do this, just end the `isisicp` process in task manager. + +### DAE doesn't seem to be connected/I want to run without a DAE connected +The DAE can be set to run in simulation mode, this must be unset before data will be collected. To set the mode run `g.set_dae_simulation_mode(True)` or `g.set_dae_simulation_mode(False)` to unset. + +To change the simulation mode manually, in `icp_config.xml` change the simulate property to 1 (or 0 if turning off simulation mode). `icp_config.xml` can be found in either the "LabVIEW modules" or "ICP Binaries" directory. Stop the DAE IOC from the console, then kill the ISISICP process. Finally, restart the DAE IOC from the console. + +### Log file for LabVIEW modules DAE + +Both SECI and IBEX use the same underlying `isisicp` program (located in `labview modules\dae`) that writes to `C:\Data\Export only\logs\icp\log\icp-log`. There is an [example DAE log in this wiki](DAE-Normal-Log). This is a good place to locate DAE specific issues, not all of the details in this log appear in the ISISDAE IOC log. + +### Error pop up: `*** ICP failed to start - your DAE may be switched OFF or is missing cards ***` +The DAE unit may be switched off. This is particularly likely during shut down. Change the DAE into simulation mode as described above. + +If the DAE is on then it is likely that it has been power cycled. +Two possible ways to fix it are: + +1) Restarting the DAE rack server + +1) Via NI MAX, using the following steps: + 1) Connect to the DAE rack server via Remote Desktop from the NDX machine (the machine should be listed in the Remote Desktop dialog - if not use address 192.168.1.2) + 1) Run NI MAX on the DAE rack server + 1) Under Software->NI-VISA X.X.X, select VISA Server and click "Stop server now" (if you cannot see this button you may be on the wrong tab on the bottom bar) + 1) Under "Devices and Interfaces", select "VXI System" and click "Run the VXI Resources Manager". + 1) Finally, repeat b) but click "Start server now" + +If on NI MAX the "Run the VXI Resources Manager" button and VISA server buttons are greyed out/not available this is likely a sign that the VISA server cannot talk to the DAE. + +For DAE3 see `DAE3 does not start` below + +### Blocks not being added to Nexus file +This should not occur but has when a database was missing our extra column in the archive. If the sample table in the archive is missing a sample_id, run the following. Note that it can take a while on a database with a large number of rows in that table. +``` "c:\programs files\wherever...\Mysql.exe" -u root -p --execute="ALTER TABLE sample ADD COLUMN sample_id BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT COMMENT 'Sample id'" archive``` + +### Not enough CRPT memory + +CRPT (Current Run Parameter Table) memory is a large in-memory structure used to store information about the run, including histogrammed data. Data is read from the DAE into CRPT memory and then written to file, in event mode CRPT memory is where events are histogrammed on the fly during collecting to provide real-time spectra. If you get a CRPT size error, it means the product of (number of periods) * (number of spectra) * (number of time channels) is too big. If you are in histogram mode you either need to reduce one of these variables or get the CRPT size increased (icp_config.xml) but remember this is real memory that the ICP will claim at startup. If you are in event mode and get a CRPT error, it may mean you have misconfigured the time regime you plan to use for the on-the-fly rebinning e.g. you are trying to rebin events at event mode resolution not at a coarser resolution. The event mode / histogram mode choice and which time regime to use is governed by the [wiring tables](#dae_configuration). + +### End of run script not working or data not being copied to the archive + +There is a known bug where starting a run at the same time as the previous run is being saved can be cause the nexus file not to be marked read-only and so not copied to the archive. A `NEXUS ERROR: ERROR:` message will appear in the log. See https://github.com/ISISComputingGroup/IBEX/issues/4977 + +To fix this and other errors see [experimental runs troubleshooting](Experimental-Runs) + +### No frames/beam current registered by the DAE + +Try switching the timing source to "Internal test clock" (found in experiment setup tab of DAE) and starting a run. If frames are received in this state, it means that the DAE isn't receiving timing pulses from the central source. If that's the case, it needs attention from the electronics group (e.g. Simon Moorby). Note, this may occur on more than one beam line so keep an ear open for any other reports. + +Don't forget to switch the timing source back when you're done! + +Other things to to check in this state are: + +- [ ] Visit the beamline - (possibly with electronics is suspecting a hardware problem). + Software usually doesn't just stop normally when other things are working - right? :smile: +- [ ] Most importantly, ask the scientists if anything happened around the time of the problem, in a recent case they mentioned someone had moved a cable on an ADC (although this was not the problem!). +- [ ] Look at the lights on the ADC or detector input module cards on the DAE. If no lights flickering, there is no data coming in and this is a good indicator that the HT might be off (a few lights might mean shutter closed or beam off). +- [ ] data/transfer lights on a DAEII, flickering & transfer lights inactive not a good sign. Could be the link to the PC if transfer lights are not showing activity. +- [ ] If frame/raw counts are not showing up, a good diagnostic is to put the DAE into "Internal Test Clock". If this works and frames appear, it is likely that there may be a problem with a Time of Flight signal (this often affects more than one beamline. + +### My total counts are low +Make sure that the timing is appropriate (e.g. a DAE Timing Source of `ISIS (first TS1)` will only count the first pulse not all 4). If it is left on "internal test clock" you will receive no uamps at all. + +### Count rate low (across multiple instruments) + +This can be due to an analogue ("PPP") signal being incorrect, this is an analogue signal that the DAE uses internally to calculate beam current. + +If you see this, DSG group may be the best contacts, they may then pass the issue onto the relevant (accelerator?) teams to investigate more. + +Exact copy-pasted note from the relevant team the last time this happened: + +> [...] the Buffer Unit from the `Diags` Room that sends EIM6 to the HET Racks where it becomes the PPP Signal is loading the new electronics, causing this loss of signal + +### Simulation mode DAE complains about missing cards + +From an issue in Ticket https://github.com/ISISComputingGroup/IBEX/issues/3099 - example traceback: + +``` +[2018-04-09 15:26:49] sevr=major setDCEventMode: Unknown detector card 3 +[2018-04-09 15:26:49] setDCCardMode: Unknown detector card 3 +[2018-04-09 15:26:49] setDCEventMode: Unknown detector card 4 +[2018-04-09 15:26:49] setDCCardMode: Unknown detector card 4 +[2018-04-09 15:26:49] Cannot find card for crate 3 +[2018-04-09 15:26:49] Unknown detector card 3 +[2018-04-09 15:26:49] Cannot find card for crate 4 +[2018-04-09 15:26:49] Unknown detector card 4 +[2018-04-09 15:26:49] Attempt to use missing detector card/crate 3 +[2018-04-09 15:26:49] Attempt to use missing detector card/crate 4 +[2018-04-09 15:26:49] : Exception occurred. +``` + +The issue here is that the default simulated DAE has 2 detector cards in it, but the real DAE has more cards. To fix edit `isisicp.properties` in LabVIEW modules to create more cards. Note this is not an ibex issue - it will also affect DAE simulation mode under SECI. The number of cards on each crate is given by the maximum missing card for the crate (see log), more crates can be added as well as cards. An example from wish with 3 crates, 10, 10 and 12 card per crate is: + +``` +isisicp.simulation.detcards.crate0.number = 10 +isisicp.simulation.detcards.crate1.number = 10 +isisicp.simulation.detcards.crate2.number = 12 +``` + +If you have defined `isisisp.datadae.use = true` in `isisicp.properties` then you need to make sure the detector card referred to in data_dae.xml is created by above. If this is a pure setup/test machine rather than a real instrument, you may just want to set `isisisp.datadae.use = false` + +### Real DAE complains about missing cards (but was previously working) +If you see messages like +``` +setDCEventMode: Unknown detector card 1 +setDCCardMode: Unknown detector card 1 +Attempt to use missing detector card/crate 1 +Unknown detector card +``` +when trying to BEGIN on a real DAE, then there are two likely causes: +- you are loading a wiring table that is specifying cards that do no exist, you need to correct the wiring table +- The ICP has not detected all the cards you believe are present in the DAE, hence they appear to be "missing" or "unknown" + +If the wiring table is correct, try a restart of the ISISICP - the DAE is only scanned at program startup, it might be the DAE hardware was not feeling very responsive first time around. If this doesn't help, then it may be the detector card has failed, or it could be the hardware is in a strange state and needs a reset. Electronics group have programs that can do this. + +If the system is running DAE3, then there is another possible cause. check the log for a line like +``` +Unable to create Quixtream on process20: Quixtream Error: Failed to bind the socket to the local port. +``` +The quickstream driver expects certain ports in the UDP dynamic range to be available - this error indicates something is not. It is not very helpful about which specific port (it uses UDP ports from `0xFE00` (65024) upwards) so a reboot may be the only option. + +### Real DAE complains about missing cards/crates (new card/crate hardware has just need added) + +If a new DAE3 card or DAE2 crate has been added, it may not be picked up automatically. The ISISICP scans "crates", where for DAE2 this was a full 11 slot VME crate, but for DAE3 each "crate" is actually a physical DAE3 card. So adding a new DAE2 detector card to an existing crate does not require any additional changes, but adding an additional DAE3 card requires a change to `icp_config.xml` in `c:\labview modules\dae`. If it contained: +``` + DAEDevice0 ISISDAE0 +``` +this tells it to scan for crate 0. Add an additional line: +``` + DAEDevice1 ISISDAE1 +``` +to also scan for crate 1. For DAE2 the name like `ISISDAE0` refers to an alias for a VISA resource name, for DAE3 just follow the same naming convention but no VISA alias needs creating. The number of `DAEDevice*` lines present should match: +- For DAE2, the number of VME crates +- For DAE3, the number of DAE3 cards + +### DAE exception messages + +If you get an error in you IOC log like: + +``` +[2018-10-26 17:33:37] sevr=major Win32StructuredException code 0xc0000005 pExpCode 0xc0000005 pExpAddress 0000000000000000 +[2018-10-26 17:33:37] 2018/10/26 17:33:36.741 IN:DEMO:DAE:AD1:INTG:TMIN:SP devAsynFloat64 pPvt->result.status=3, process error isisdaeDriver:writeFloat64: status=0, function=184, name=INTG_TMIN, value=0.000000, error=Win32StructuredException code 0xc0000005 pExpCode 0xc0000005 pExpAddr +``` + +One cause would be the IOC is trying to call a function in the ISISICP that it can't find. If the ISISICP has been updated, but /RegServer has not been run, then new functions added there will not be visible. See [here](#first_time_install_configure_dae). + +### DAE3 does not start + +DAE3 is new ethernet based acquisition electronics on ZOOM and MARI, it used `ISISICP` and looks like DAE2 for most purposes. If everything remains in processing, it may be that the `arp` network entries did not get created - these should be done as a system time boot task. Do `arp -a` and see if there is an entry for 192.168.1.101 etc. If not, run `set_dae3_arp.bat` in `c:\labview modules\dae` as as administrator + +Note that DAE3 does not ping, so the only way to know if it is there is by running `qxtalk` or the `isisicp` (via ibex or seci) + +If IBEX/SECI has either been in simulation mode or not running for a long time previously, then some of the tcp ports used by dae3 may have been grabbed by the operating system as described in `Real DAE complains about missing cards (but was previously working)` above on this page. You will see errors like +``` +Quixtream Error: Failed to bind the socket to the local port." +``` +in the ICP log and some client applications may complain about `NULL Pointer` errors. The only solution is to reboot the NDX computer. + +### Error code 112 + +If there is an error code 112 reported in the log it means that the disk (data volume) is full and it can not start the `isisicp` program. + +### DAE Type mismatch error + +If you get an error from ISISICP `*** ISISICP STARTUP FAILED (DAE type mistmatch)***` it means you are running `isisicp` program with the wrong sort of dae ie 2 when you have 3. You need to source the correct version of the code for your type of DAE. + +### isisicp.exe keeps allocating 4GB of memory and then releasing it + +It may be https://github.com/ISISComputingGroup/IBEX/issues/3701 - you just need to change the archive array table type to MEDIUMBLOB. Due to a bug in the C++/MySQL connector, each time the database is read, a LONGBLOB's worth of memory (4GB) is allocated and then released. By changing to MEDIUMBLOB (16MB), a much smaller amount is used. + +In `C:\Instrument\Apps\MySQL\bin>`, run `mysql.exe -u root -p` and enter the MySQL root password when prompted. + +Then run these commands to modify the DB in place: + +``` +USE archive; +ALTER TABLE sample MODIFY COLUMN array_val MEDIUMBLOB; +``` + +**Note:** that command rewrites the whole `sample` table, so if the table is big it may take a while. You may wish to consider truncating the `sample` table before doing this (though that may not be possible if it contains useful data) + +### Instrument stuck in `WAITING` state + +We have observed on a couple of occasions that the DAE got stuck in `WAITING` despite no blocks being outside of runcontrol limits. The cause is yet unclear but in the meantime try: +- force a resync of the system with e.g. for OFFSPEC `caput IN:OFFSPEC:CS:RC:SYNC:SP 1` +- restart the `RUNCTRL_01` IOC + +### ISISDAE reports `time regimes 1 and 2 are incompatible` + +[Time regimes](#dae_configuration) are incompatible when their starts differ by a non-integer number of microseconds, but sometimes rounding errors may lead to this happening in other circumstances. This check is actually no longer required and has been removed in ISISICP SVN revisions 2010 and above. + +### ISISICP writes a corrupted journal file + +The symptom is that `C:\data\journal_.xml` will not be valid xml, it will be truncated at some point. We believe this happens when there are too many blocks set to log into the journal in a particular configuration. + +After switching back to a configuration with fewer blocks, the journal file can be (carefully!) manually edited to remove the corrupt entry. Once this is done, runs should go back into the journal as normal (however, runs done while in the configuration with too many blocks will be lost from the journal). + +Freddie may also have a patched version of the isisicp that fixes this issue. + +### My blocks aren't being written to a run title properly + +See the documentation in the [user manual](https://github.com/ISISComputingGroup/ibex_user_manual/wiki/Add-blocks-to-run-title) + +### Exception in ICP log + +If the ICP is showing an error in the form of `NeXusEventCallback: [Warning] (NeXusEventCallback::allFrameCallback) Invalid DAE time value` this indicates a hardware problem. Newer versions of the ISISICP will estimate and then correct the time information based on good data either side of the corrupt data, if these messages are appearing frequently you should get in touch with the electronics group. + +### A `measurement.nxs` file is being written to the `C:\data` area + +This file is written by the ISISICP on some instruments if they have defined a non-zero "measurement ID". It is intended to be used for correlating runs. The presence of the file itself is nothing to worry about. + +### Gap in time in journal file for run start and end (so may think No Data has been written) + +If the instrument is in a WAITING state for the entire run, then the end_time as written in the nexus file/journal will be the same as the start time. The ICP interprets end time as end of neutron data collection, so if this never starts it remains the same as the start time and run duration is 0. All sample environment data will be collected OK in a WAITING state. + +### Simulated DAE does into VETOING after a PAUSE/RESUME + +The symptom is that, when you do a begin the instrument will go into running and look happy, but after you do a pause/resume it will permanently be in vetoing (until the run is ended and the next run is started). + +This is a bug in the ISISICP as it does not simulate hardware period properly. + +Resolution: +- Ensure period mode is not "hardware" (use software) +- Kill and restart `isisicp.exe` (need to do this as the simulated isisicp can't change period modes correctly without restarting) + +### Control program unable to read from DAE3 + +If there are lots of read timeouts, but writes work, then this could be a firewall issue. Try disabling the firewall on the DAE private (192.168.*) network (this is the network that is not the "domain" network on an instrument, often it is called "public"). There should be firewall rules to allow programs access, but something may have gone wrong with them + +### Beam current in Dashboard oscillating + +The beam current in the dashboard is not read from the accelerator, but is calculated from the DAE which records the total amount of proton charge received so far. The software reads the DAE charge value, then reads again after a certain amount of time, and then the difference between these plus the time gap allow an effective beam current to be calculated. By effective I mean it will include running at a slow chopper speed as well as vetos, so with a 25Hz chopper on TS1 you will expect to see roughly half the TS1 accelerator delivered value. + +If the DAE beam current is oscillating but the accelerator is constant, then this can be due to issues with the time or charge component of the calculation. If it is oscillating rapidly, it may be due to some reads taking longer and meaning the "time" value used in the calculation does not correspond to the charges used. Look for DAE read timeouts, or sometimes if a lot of spectra are being displayed in the GUI this can slow down DAE proton reads. If there are a lot of DAE timeouts in the log, contact electronics group. + +If the DAE beam current is incorrect for a period of time rather than rapid oscillations, this is more likely to be due to either vetos or the DAE syncing to the wrong accelerator pulse. HRPD run at 10Hz and were seeing a DAE beam current going to zero for a period of time and then returning to normal. TS1 runs at 50Hz, but 1 pulse in 5 (10Hz) goes to TS2 so if a 10Hz chopper syncs to the missing pulse you will not see any data. It looked like on HRPD this syncing was shifting and occasionally latching onto the empty pulse for a period of time (several minutes). This syncing could be a chopper or DAE issue, probably start with DAE first . + +### Add Single Run Entry into the Journal + +It is possible to add a single run into the journal database if it was not included for some reason simply run: + +``` +"c:\labview modules\dae\JournalParser.exe" <9 DIGIT run number> cycle_ "c:\data\export only" +``` +e.g. `"c:\labview modules\dae\JournalParser.exe" LOQ 00108032 cycle_19_3 "c:\data\export only" NDXLOQ` + +### CRPT is not initialised - please set experiment parameters + +Ensure you have loaded the correct Time Channels and Data Acquisition tables in the Experiment setup tab of the DAE perspective. Reloading them causes initialisation. If you do not have any tables do the following: +1. Copy the detector, wiring, spectra and TCB tables from the [system tests](https://github.com/ISISComputingGroup/system_tests) +1. In the GUI go to DAE -> Experiment Setup -> Time Channels, select `Use TCB file` and select one of the copied TCB files +1. In DAE -> Experiment Setup -> Data Acquisition, select the wiring, detector and spectra tables + +See [here](#dae_configuration) to learn more about these files. + +{#dae_troubleshooting_cannot_see_nexus_files} +### User Says they Can Not see their Nexus Data files on external machine + +After a run the data files should be copied to an external archive so that the users can do more processing. If the user can not get these then there can be multiple causes and it depends exactly where they are looking. The first thing to check is that the machine is copying the files correctly in the post +end run script. The log for this is stored at `C:\Data\log\post_command_.log`. If there is a problem rectify the problem, possible causes are: + +- mapped drive (`d:`) is not connected properly: often double clicking on the drive is enough to get it remapped +- something else: fix and document here + +After fixing the issues the files will be copied after the next run is finished. So start and end a run (with a test like title), you may also want to put it in simulation mode if the DAE is switched off. Finally check the files have appeared in `\\Instrument\data\cycle_`. + +### DAE Server restart doesn't work message when clicking start "Cannot start server on port XXX" + +If you need to restart the NI visa server on the host machine and clicking start on the server says can not start server, it may be that the via driver has started in the admin account instead of the user account. Log onto the host machine and quit the NI VXI Resource manager application. This should allow the user accounts server to start. + +### Exception in ICP log + +If the ICP is showing an error in the form of `Simulation error (read32)` this indicates that you have a DAE3 ICP installed. Only a DAE2 ICP has a simulation mode. You should reinstall the ICP from `\\isis\inst$\Kits$\CompGroup\ICP\ISISICP\DAE2`. + +### No environment cards in simulation mode + +In simulation mode the ISISICP should automatically give you 1 environment card. If you see the following error: + +``` +Error - Found 0 environment/period cards (must have only 1) and 0 detector cards +``` + +It can be a sign that the ISISICP is trying to restore from a corrupt save file (which could have been caused by running out of disk space, for example). Delete `current.run`, `data.run`, and the mysqlite db files from `c:\data` and try again (after having made enough disk space). + +### cannot lock named mutex (abandoned): ISISICP_PROCESS_LOCK + +If you see the following error: + +``` +(0) Application: [Error] System exception: cannot lock named mutex (abandoned): ISISICP_PROCESS_LOCK +``` + +This is a sign that there may be multiple ISISICP processes running concurrently. Check for any isisicp processes running as the wrong user (e.g. admin). + +It can also be a sign that the ISISICP is trying to restore from a corrupt save file (which could have been caused by running out of disk space, for example). Delete `current.run`, `data.run`, and the mysqlite db files from `c:\data` and try again (after having made enough disk space). + +### Saved counts are very low + +In event mode the ISISICP will wait to get a consistent snapshot from all detector cards. If one of the detector cards has a hardware fault, it can cause the saved counts to appear to be very low. The solution in this case is to ask electronics to look into the fault/replace the faulty detector card. See [ticket 5835](https://github.com/ISISComputingGroup/IBEX/issues/5835) for further details of this issue. + +### DAE Internal test clock is at wrong frequency + +Originally DAE2 had a fixed 50Hz internal test clock, newer DAE2 cards and DAE3 have a register at address 0x244 that can be set to the frame length in microseconds (so 20000 for TS1, 100000 for TS2). The DAE makes a guess as power on, but may not get this right. The ISISICP will be modified to set this, but in the meanwhile you may need to use the old "dae tester.vi" to set the value + +### errors +An error message like: +``` +QXTRM: [Information] QxtrmInterface::readWithRetry Retrying Null pointer +``` +means the control program cannot connect to the DAE. It may be the static ARP addresses have disappeared, or the DAE is switched off, or the DAE has locked up. + +### Histograms Show as Disconnected when DAE otherwise appears to be working + +Sometimes, the four histograms on the DAE display do not show a plot and stay disconnected. Data is however correctly being read from the DAE and saved to file without problems. This may occur if the DAE hardware has previously been disconnected or switched off and the ICP has recovered its connection to the DAE apparently successfully. + +The DAE histogram display comes via a different route and does not recover the same way as the data acquisition does. The solutions is to restart the DAE IOC (instructions above). + +### DAE tab showing pink borders +This seems to be a problem with CSS. To fix, restart the GUI. + +### set a different run number + +In `labview modules\dae\service\x64\release` you can run e.g. +``` +instctrl.exe SETVAL RUN_NUMBER 441 +``` +to set a new run number + +## DAE IOC cannot start ISISICP + +When using EPICS ISISDAE if you see errors like +``` +[2022-01-07 11:11:37] (Re)Making local connection to ISISICP +[2022-01-07 11:11:37] CoCreateInstance() failed with error -2147024809, retrying with CLSCTX_ALL +[2022-01-07 11:11:37] updateRunStatus exception: CoCreateInstance (ISISICP) : The parameter is incorrect. +``` +This likely means that the isisicp did not write the registry entries. This has only been seen on windows server machines. +To solve, import appropriate isisicp reg file from the isisicp install location (`isisicp_icp_binaries.reg` or `isisicp_lv_modules.reg`) using `regedit` + +If you get +``` +[2022-01-07 11:22:06] (Re)Making local connection to ISISICP +[2022-01-07 11:22:06] CoCreateInstance() failed with error -2147024891, retrying with CLSCTX_ALL +[2022-01-07 11:22:06] (Re)Making local connection to ISISICPsevr=major CoCreateInstance (ISISICP) : Access is denied. +``` +This is a dcom access permission, most likely seen in jenkins as they run under a domain rather than local account. +Again, only seen on windows server. To resolve: + +* add account trying to run isisicp to "distributed com users" local windows group +* run `dcomcnfg` command and make sure "distributed com users" has appropriate default permissions (it should already have appropriate limit permissions, but check) + +If you get the error "server execution failure" (seen on MUONFE) and also the system log has errors/crashes mentioning `visa64.dll` then you need to copy all the DLL files in `c:\labview modules\dae\service\x64\visa` into `c:\labview modules\dae\service\x64\release`. This probably indicates an incompatibility between the system installed visa and what isisicp is expecting. Only seen on MUONFE which runs in simulation mode, so not worried about which visa version being used. If seen on a DAE2 instrument may need more looking at as these machines use visa client-server to the NDH, and the visa versions either end need to match to some level. + +### DAE not showing period changes in NeXus data file when running in event mode + +Due to a historical problem with electronic noise causing random period changes, many instruments have a setting enable to set the period number to 1 in event mode data events regardless of what the DAE claims it is. To make the DAE vale be used you will need to edit `isisicp.properties` in `c:\labeview modules\dae` and add/update the line to be `isisicp.badheaderworkaround = false` then restart the `isisicp` process + +### cycle number not updating + +The isis cycle is contained in a local file `setcycle.cmd` on the NDX that is read by the `end_of_run.cmd` which will also update this file to the most recent copy from the mapped o: drive on the NDX. If the cycle number is not updating then: +- sometimes the first run in a cycle gets given the wrong number in the data file, but is put in the correct cycle directory. This is a "feature" as the cycle number is updated as part of copying the file, hence the file will already have been written using the previous number. +- if all files are in the wrong cycle, either the central copy has not been updated, or the `o:` drive on the instrument has become disconnected so it is not being updated to the latest version + +to reconnect the o: drive, log onto the NDX and just open it in windows explorer, that should reconnect using cached credentials. Then check that `o:\setcycle.cmd` has the correct cycle number. + +### run stuck in ending when using period card + +Check icp log file. If it says "waiting for period card" and beam is off, then the issue is that a default end waits to complete the current period cycles, which if beam is off is not going to happen (no trigger signal for frames). Solution is to type `g.end(immediate=True)` in a python console. + +If the instrument is not using hardware periods via a period card, then the `c:\data` disk area may be full. If it is running in event mode it will get stuck in ending and be be unable to write a data file if it has already filled the disk with raw events. Check nagios and/or the computer. See below. + +## run stuck in ending or data files not being created + +If data files are not appearing on the archive or in the instrument journal viewer, it may be the end process has not completed. After an END is issued remaining data is read from the DAE and then a separate thread in the ISISICP process is spawned to complete the end process. To avoid too many of these threads running at once, they all share an internal semaphore. If somethings gets stuck in the ending process, then runs will not complete. You can see this by there being lots of `current.run*`, `data.run*`, and `events*.tmp` files in `c:\data` on the NDX instrument. Normally you should only see files for the current run number and maybe the previous run number if the END is still happening in the background. If you see many more than this it would indicate a problem. + +To resolve the problem you need to restart the ISISICP in failed run recovery mode. +- make sure NDX instrument machine in SETUP +- edit `c:\labview modules\dae\isisicp.properties` and add/amend a line to say `isisicp.failedends.rerun = true` +- kill the `isisicp.exe` process +- the process should now restart and re-save the missing runs +- re-Edit `isisicp.properties` and comment out the `isisicp.failedends.rerun` line so it will not automatically do any reruns on next restart. This is just in case a later run failed to end and crashed the program due to corrupt data, if it tried to re-end this on restart you might get into an infinite isisicp re-end crashing loop. + +## corrupted MAX database + +for DAE2 systems the VME connection should be visible in the NI measurement and automation explorer, if when you try and view it you get + +![Capture](DAE-Trouble-Shooting-corrupted-max-database.PNG) + +you probably need to follow https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000000P8awSAC&l=en-GB + +## ISISICP program crashed and cannot restart, `c:\data` area full + +If you need to recover a system that has filled up its `c:\data` area due to a long event mode run with e.g. noisy detectors then you can use the following. This assumes the scientists do not need the data, you can move the files off instrument but recovery is hard and may not be possible. + +* log onto NDX computer +* Run `stop_ibex_server` or `kill seci` as appropriate +* open `c:\data` in windows explorer and sort files by name +* look for a very large `eventsYYYYY.tmp` file, make a note of the `YYYYY` number and then select and `shift+delete` this file (you do not want to move it to recycle bin - make sure the prompt says "permanently delete this file" and not "delete this file") +* Also now shift+delete `current.run`, `current.runYYYYY`, `data.run`, `data.runYYYYY` +* open `c:\data\events` and shift+delete the folder `run_YYYYY` +* start ibex or seci again + +Hopefully there is either only one `eventsYYYYY.tmp` file, or the most recent one (largest YYYYY number) is also the largest in size. If there are several files making a decision may need a bit more thought. It is possible that a very large run was done earlier which nearly filled up the disk, then a new one was started that pushed it over the limit while the previous one was ending in the background. Check with the scientists if it is ok to delete this earlier bigger run. You should always delete all YYYYY files corresponding to the run that was in progress when the system filled up, but you may also need to handle an earlier YYYYY number set of files to free up enough disk space. + +For info, the `eventsYYYYY.tmp` file is the NeXus HDF5 file being written as the run goes along, and the `run_YYYYY` folder is the raw events read from the electronics. The `.tmp` is renamed to `.nxs` during an END is nothing has gone wrong, otherwise raw events can be replayed from `run_YYYYY` to create a NeXus file from scratch + +If the run number comes back incorrectly (like as `000001`) then it means `c:\data\recovery.run` has been corrupted. There are backup weekday copies in c:\data e.g. `recovery.run_Thu` so you can pick one to copy to `recovery.run` however note that this may include an old run number so after restarting you may need to reset the run number (see on this page) + +## SQLite database is very large + +The `selog.sq3` file in `C:/Data` is used by ISISICP to store log data prior to a run end, it is then purged. It will normally only get very big if a run has not been ended for a long time, which either happens over a shutdown if logging has been left running, or on systems that do not often end runs such as RIKENFE, MUONFE, DETMON, HIFI-CRYOMAG etc. The solution is to begin and end a run, though on HIFI-CRYOMAG the scientist will normally do this themselves once a cycle to keep log data together in the same data file, so consult with them if it needs doing. Ending a run may take a while and will happen in the background, so don't stop ibex as soon as you get back to SETUP as it will likely not have finished saving. If you need to stop ibex, wait until the data file for the run you have saved appears in the archive. + +## Instrument state keeps 'Processing' but always goes back to 'Unknown' + +Check to see if you have any errors similar to the following: + +``` +[2016-11-07 16:04:49] CoCreateInstanceEx (ISISICP) : Class not registered +``` + +If so, you haven't registered your `isisicp.exe` program with the registry. Follow the steps to [Configure DAE for simulation mode on developer's computer](#first_time_install_configure_dae) + +If you have done this it may be that the isisicp.exe program is too old. Older versions do not contain a function which is needed by IBEX. Check the file `svn_revision.txt` in `c:\labview modules\dae` - it needs to be 1633 or higher. If it needs updating, ask a SECI specialist to update the program. + +## Multiple VXI devices + +If you have multiple DAE2 VME crates then you will legitimately have multiple VXI devices in NI MAX. However there are cases when you can get two devices showing in MAX but have only one device in reality. This is usually because of either: +- The DAE USB cable has been moved into a different USB port on the PC +- the DAE NI USB card has been changed + +in these cases when the "scan for new hardware" happens on the PC, it thinks it has found a new device. Usually you will have a disconnected VXI0 and a new VXI1 device. To remedy the situation +- stop visa server in NI max +- delete VXI0 +- rename VXI1 to VXI0 (change the system identifier number in the menu for the device from 1 to 0) +- start visa server + +**Full rebuild, if above doesn't work**: +- stop visa server in NI max +- delete all VXI* device +- view refresh +- as admin run `compmgmt.msc` +- computer management -> device manager -> right click scan for hardware changes on NDH computer name and possibly USB sub tree too +- close and reopen NIMAX +- if VXI0 now appears, jump to "add VME device" step below +- if it is still not showing reboot NDH computer, after reboot visa server may be running again so stop it +- right click on Frame0 of VXI0 and "add VME device" then choose DAE2 from VME profile +- start visa server diff --git a/doc/specific_iocs/dae/Dae-Spectra-Updating-Problem.md b/doc/specific_iocs/dae/Dae-Spectra-Updating-Problem.md new file mode 100644 index 000000000..053c0fef4 --- /dev/null +++ b/doc/specific_iocs/dae/Dae-Spectra-Updating-Problem.md @@ -0,0 +1,9 @@ +# Dae Spectra Updating Problem + +There has been a long running problem with spectra stopping updating after a certain amount of time +[issue 2750](https://github.com/ISISComputingGroup/IBEX/issues/2750) this has not been reproducible on developer test machines, the only clue came recently when the related detector diagnostics display stopped and reported a message that looked a bit like thread termination. The proposed solution has been to add additional exception handling on the assumption that a windows structured exception rather than a C++ std::exception is the cause. To handle these non-C++ exceptions, the following has been done: + +1. Compiling with `/EHa` which allows catch(...) to handle Structured Exceptions +2. Registering a structured exception handler to convert Structured Exceptions into C++ exceptions + +In theory (2) is all that is needed, but the handler needs to be installed on a per thread basis so just to be extra safe (1) has been implemented too \ No newline at end of file diff --git a/doc/specific_iocs/dae/Datastreaming.md b/doc/specific_iocs/dae/Datastreaming.md new file mode 100644 index 000000000..6bf70b8cd --- /dev/null +++ b/doc/specific_iocs/dae/Datastreaming.md @@ -0,0 +1,52 @@ +# Datastreaming + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 +:hidden: + +datastreaming/* +``` + +The datastreaming system is being built as part of in-kind work to ESS. It will be the system that the ESS uses to take data and write it to file - basically their equivalent to the [ICP](/specific_iocs/DAE-and-the-ICP). The system may also replace the ICP at ISIS in the future. + +In general the system works by passing both neutron and SE data into [Kafka](https://kafka.apache.org/) and having clients that either view data live (like Mantid) or write the data to file, additional information can be found [here](http://accelconf.web.cern.ch/AccelConf/icalepcs2017/papers/tupha029.pdf) and [here](https://iopscience.iop.org/article/10.1088/1742-6596/1021/1/012013). + +The datastreaming layout proposed looks something like this, not including the Mantid steps or anything before event data is collected: + +![](ESSDSLayout.png) + +## Datastreaming at ISIS + +Part of our in-kind contribution to datastreaming is to test the system in production at ISIS. Currently it is being tested in the following way, with explanations of each component below: + +![](ISISDSLayout.png) + +## The Kafka Clusters +There are two Kafka clusters, production (`livedata.isis.cclrc.ac.uk:9092`) and development (`tenten.isis.cclrc.ac.uk:9092` or `sakura.isis.cclrc.ac.uk:9092` or `hinata.isis.cclrc.ac.uk:9092`). The development cluster is set up to auto-create topics and so when new developer machines are run up all the required topics will be created. However, the production server does not auto-create topics this means that when a new real instrument comes online corresponding topics must be created on this cluster, which is done as part of the install script. Credentials for both clusters can be found in the keeper shared folder. + +### Grafana dashboard +A Grafana dashboard for the production cluster can be found at `madara.isis.cclrc.ac.uk:3000`. This shows the topic data rate and other useful information. Admin credentials can also be found in the sharepoint. + +### Deployment +Deployment involves the use of Ansible playbooks, the playbooks and instructions for using these can be found [here.](https://github.com/ISISComputingGroup/ansible-kafka-centos) + +## Neutron Data +The ICP on any instrument that is running in full event mode and with a DAE3 is streaming neutron events into Kafka. + +## SE Data +See [Forwarding Sample Environment](datastreaming/Datastreaming---Sample-Environment) + +## Filewriting + +See [File writing](datastreaming/Datastreaming---File-writing) + +## System Tests +Currently system tests are being run to confirm that the start/stop run and event data messages are being sent into Kafka and that a Nexus file is being written with these events. The Kafka cluster and filewriter are being run in docker containers for these tests and so must be run on a Windows 10 machine. To run these tests you will need to install [docker for windows and add yourself as a docker-user](https://docs.docker.com/docker-for-windows/install/#install-docker-desktop-on-windows). + +## The future of streaming at ISIS + +After the in-kind work finishes and during the handover, there are some proposed changes that affect the layout and integration of data streaming at ISIS. This diagram is subject to change, but shows a brief overview of what the future system might look like: + +![](FUTUREISISDSLayout.png) \ No newline at end of file diff --git a/images/ESSDSLayout.png b/doc/specific_iocs/dae/ESSDSLayout.png similarity index 100% rename from images/ESSDSLayout.png rename to doc/specific_iocs/dae/ESSDSLayout.png diff --git a/images/ESSDSLayout.xml b/doc/specific_iocs/dae/ESSDSLayout.xml similarity index 100% rename from images/ESSDSLayout.xml rename to doc/specific_iocs/dae/ESSDSLayout.xml diff --git a/doc/specific_iocs/dae/Experimental-Runs.md b/doc/specific_iocs/dae/Experimental-Runs.md new file mode 100644 index 000000000..07194dbc7 --- /dev/null +++ b/doc/specific_iocs/dae/Experimental-Runs.md @@ -0,0 +1,54 @@ +# Experimental Run Troubleshooting + +Something Has Gone Wrong with my Experimental Run + +## Script Failed to Finish and Take All Measurements + +This is from a problem see on IRIS. + +1. Get the script and configuration. + In this case the important bits are: + ``` + def set_sample_temp(temp, high, low, tlccr_offset): + g.cset(Sample=temp, highlimit=high, lowlimit=low, runcontrol=True) + g.cset(TLCCR=temp+tlccr_offset, runcontrol=False) + + def next_sample(temp, uamps): + g.change(title="Fast Frozen Liver Cells transplant + Cryoprotectant + algae "+str(temp)+"K PG002") + g.begin() + g.waitfor(uamps=uamps) + g.end() + + for i in range(47): + temp += 5 + set_sample_temp(temp, temp+2, temp-2,-5) + next_sample(temp, 20) + ``` +1. Find the PVs and plot them in log plotter + 1. Open IBEX GUI. Switch to the instrument in question + 1. If the config is loaded right click on PV and display block history (otherwise do this and edit) + 1. Look for the time in question (zoom out horizontally button in bar and use the hand to drag to the place) + 1. Add other plots which might be interesting (right click add PV) + 1. Block as `IN::CS:SB:` + 1. Other PVs `IN:::PV.VAL` (these must be marked in the DB file with `info(archive, "VAL")`) + 1. Good blocks to choose might be SP and SP:RBVs in this case plotting the SP and SP:RBV showed that the SP had not be set in the machine. So run control hadn't been able to progress and it had hung. +![log plotter image](LogPlotter.png) + +## Experiment stuck in `Waiting` state after beginning a run + +This issue was encountered on Iris during the transition between Seci and Ibex. SECI was in a waiting state prior to shutdown, then IBEX was left in this waiting state. Usually SECI run control is transient i.e. set by Open GENIE and cleared on a SECI restart, so just restarting SECI would usually clear it. The is now a new PV that you can write to from IBEX to force a resync of run control + + caput %MYPVPREFIX%CS:RC:SYNC:SP 1 + +## Experimental Files not being Archived and so not Appearing in the Journal + +At the end of a run log file should be transferred from `C:\data` to the archive. For this to happen the files should be marked read only, then when the run ends it is copied thanks to a script in `\Documents\Configurations\COMMON\end_of_run.cmd`. +Double checking that the files are read only is a good first step when items are missing from the archive. +If this has failed: + +* Make sure the network location d:\data\ is accessible. This can need connecting to manually after a restart, i.e. open the folder in explorer. + +To archive files end a run. +Should you have runs ending and read only files, then check on the archive in `INST\instrument\logs\cycle_nn_n`, the log files there will provide some extra information. If the file `exception_copy.log` doesn't exist, or hasn't been updated for a long time, then the script above isn't running the code to clean up old files, or it has hung. + +* Problems with the end of run script can be diagnosed by looking at the `post_command_.log'` file in the ICP logs area on the instrument. The ICP log files (in the same area) may also be instructive to check for other issues - for example, if the files are not being set read-only. \ No newline at end of file diff --git a/images/FUTUREISISDSLayout.png b/doc/specific_iocs/dae/FUTUREISISDSLayout.png similarity index 100% rename from images/FUTUREISISDSLayout.png rename to doc/specific_iocs/dae/FUTUREISISDSLayout.png diff --git a/images/FUTUREISISDSLayout.xml b/doc/specific_iocs/dae/FUTUREISISDSLayout.xml similarity index 100% rename from images/FUTUREISISDSLayout.xml rename to doc/specific_iocs/dae/FUTUREISISDSLayout.xml diff --git a/images/ISISDSLayout.png b/doc/specific_iocs/dae/ISISDSLayout.png similarity index 100% rename from images/ISISDSLayout.png rename to doc/specific_iocs/dae/ISISDSLayout.png diff --git a/images/ISISDSLayout.xml b/doc/specific_iocs/dae/ISISDSLayout.xml similarity index 100% rename from images/ISISDSLayout.xml rename to doc/specific_iocs/dae/ISISDSLayout.xml diff --git a/doc/specific_iocs/dae/ISISICP---enabling-incremental-event-file-creation.md b/doc/specific_iocs/dae/ISISICP---enabling-incremental-event-file-creation.md new file mode 100644 index 000000000..3087022c9 --- /dev/null +++ b/doc/specific_iocs/dae/ISISICP---enabling-incremental-event-file-creation.md @@ -0,0 +1,12 @@ +# ISISICP - enabling incremental event file creation + +The ISISICP can write an event mode NeXus data file incrementally during the run rather than replaying all events at run end, this means it will take up disk space more gradually rather than grabbing an extra large amount at the end. It will ultimately use the same amount of disk space, but it is easier to judge when to end a run. Also the data file will appear on the archive earlier. The program still needs to save the events to disk as well as write the data file incrementally as an HDF5 file can become corrupt if the program restarts. + +To enable incremental mode edit `isisicp.properties` in `C:\labview modules\dae` and add/uncomment lines saying +``` +isisicp.incrementaleventnexus = true +isisicp.kafkastream = true +``` +The changes will not take effect until you restart the ISISICP program - make sure you are in SETUP and any runs have finished saving, then kill the `ISISICP` process using task manager (you can restart seci/ibex if you prefer, but that is not necessary) + +After this is enabled, you can also run Mantid on any computer and use the Kafka live listener to view and process events live during the run. diff --git a/doc/specific_iocs/dae/LogPlotter.png b/doc/specific_iocs/dae/LogPlotter.png new file mode 100644 index 000000000..05b45b09f Binary files /dev/null and b/doc/specific_iocs/dae/LogPlotter.png differ diff --git a/doc/specific_iocs/dae/Shrinking-ISISICP-Memory-Usage-on-Simulated-DAE-machines.md b/doc/specific_iocs/dae/Shrinking-ISISICP-Memory-Usage-on-Simulated-DAE-machines.md new file mode 100644 index 000000000..43dc50b49 --- /dev/null +++ b/doc/specific_iocs/dae/Shrinking-ISISICP-Memory-Usage-on-Simulated-DAE-machines.md @@ -0,0 +1,14 @@ +# Reducing `ISISICP` memory use in simulation mode + +If you want a minimal ICP with no real DAE support then: + +In `c:/labview modules/dae/isisicp.properties` add the line +``` +isisicp.simulation.detcards.shared = false +``` +In `c:/labview modules/dae/icp_config.xml` change the CRPTSize line to a smaller value e.g. +``` + CRPTSize 10000 +``` + +Load some very simple wiring tables defining a handful of detector and specify only a few time channels (e.g. SURF monitor only tables, muon tables, ibex RPCTT tables). Number_of_spectra * number_of_time_channels * number_of_period must be less than CRPTSize above \ No newline at end of file diff --git a/doc/specific_iocs/dae/datastreaming/Datastreaming---File-writing.md b/doc/specific_iocs/dae/datastreaming/Datastreaming---File-writing.md new file mode 100644 index 000000000..67440045b --- /dev/null +++ b/doc/specific_iocs/dae/datastreaming/Datastreaming---File-writing.md @@ -0,0 +1,101 @@ +# File writing + +The [filewriter](https://github.com/ess-dmsc/kafka-to-nexus) is responsible for taking the neutron and SE data out of Kafka and writing it to a nexus file. When the ICP ends a run it sends a config message to the filewriter, via Kafka, to tell it to start writing to file. + +## ISIS-filewriter +https://github.com/ISISComputingGroup/isis-filewriter has been created for an easy setup of the filewriter using docker-compose. it is hardcoded currently and requires the `file_writer_config.ini` file to be changed to point at the `_runInfo` topics manually. To begin with we ran it just pointing at `ZOOM_runInfo`, and it successfully wrote files containing event data. +Steps to run the docker-compose script can be found in the `README` of the project. + +The isis-filewriter repository also contains some utility scripts for Kafka operations, as well as some test `nexus_structure` files which were used to write files using the filewriter at ISIS. `add_data_to_runinfo` was used to write the config for all of the streams on MERLIN and successfully wrote events, histograms and some sample environment data. + + +*** + +## Log of changes and updates to the filewriter deployment and configuration messages +### Notes for trying to get the filewriter working on windows server 2016: +#### trying to run filewriter natively: +- hdf5 Conan library does not seem to build under windows, however it's falling over in the Conan step +- ess takes ownership of the library +- did not get any further than this as the Conan step failed, the rest of the libraries built +- not sure what is falling over but the hdf5 library can probably be fixed + +#### trying to run a docker instance of the filewriter +- DATASTREAM is potentially a VM and recursive hyper-v may not work - confirmed +- Docker desktop does not run on build 14393 which is what it's on +- I don't think this will work as we need hyper v for a windows build +- will continue trying to install docker but so far no luck +- windows containers are a bit weird and we may just end up with the same problems as #1 + +following [this link](https://blog.couchbase.com/setup-docker-windows-server-2016/) + +1. Enabled containers and restarted +1. Installed Docker - weird error but seemed to install: +``` +Start-Service : Failed to start service 'Docker Engine (docker)'. +At C:\Users\ibexbuilder\update-containerhost.ps1:393 char:5 ++ Start-Service -Name $global:DockerServiceName ++ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + CategoryInfo : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], + ServiceCommandException + + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand +``` + +Docker service then does not start and gives this error in logs: +`fatal: Error starting daemon: Error initializing network controller: Error creating default network: HNS failed with error : The object already exists.` + +A quick google leads to [this](https://github.com/moby/moby/issues/34018#issuecomment-313790817) + +After deleting `hns.data` from `C:\ProgramData\Microsoft\Windows\HNS` and restarting HNS it still does not work and gives the same error. + +##### Verdict + +The best option here would be to try and get it running natively, as DATASTREAM is a Virtual Machine itself and Docker appears to not work. As well as this, we can then run it with `nssm` which is how we run the forwarder as well, which makes for consistent service management. We could also probably use the log rotation that the forwarder is using which is build into NSSM. + +#### Update - 07/10/20 +The filewriter is now running in a docker-compose script on NDHSPARE62, this is with Docker desktop rather than the enterprise edition and is not using the LCOW framework. We should think about a more permanent solution, however Docker clearly works on server 2019 and not 2016. NDADATASTREAM is running 2016 so may make sense to update that if we want the filewriter running on it as well. + + +#### combine-runinfo (not used) +https://github.com/ISISComputingGroup/combine-runinfo has also been created to workaround the filewriter only being able to point at one configuration topic, so we can use the filewriter for all instruments. combine-runinfo's purpose is to run a [Kafka Stream Processor](https://kafka.apache.org/10/documentation/streams/developer-guide/processor-api.html) to forward all new configuration changes into the `ALL_runInfo` topic to be used with a single instance of the filewriter. + +This project is written in Kotlin and then compiled with Gradle to create a runnable `.jar` file. This is flexible, and we could re-write it in Java if it's used permanently and maintaining another language is an issue. +#### Update - 08/10/2020 +`combine-runinfo` didn't work with the messages from all topics, running the `.jar` gave these errors: +``` +to topic ALL_runInfo due to org.apache.kafka.common.errors.RecordTooLargeException: The message is 3146528 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration. + at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.recordSendError(RecordCollectorImpl.java:138) + at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.access$500(RecordCollectorImpl.java:50) + at org.apache.kafka.streams.processor.internals.RecordCollectorImpl$1.onCompletion(RecordCollectorImpl.java:201) + at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:930) + at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856) + at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.send(RecordCollectorImpl.java:167) + at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.send(RecordCollectorImpl.java:102) + at org.apache.kafka.streams.processor.internals.SinkNode.process(SinkNode.java:89) + at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:201) + at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:180) + at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:133) + at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:87) + at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:366) + ... 5 more +Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The message is 3146528 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration. +``` +The `combine-runinfo` was updated to use bytes rather than strings, however this did not solve the message size issue. +After this it was decided that as we were going to use a python script to modify the runinfo messages anyway to contain sample environment data and so on we may as well just forward the modified run info messages directly into `ALL_runInfo` instead. + +#### Adding ISIS data to the filewriter configuration +To add static data to the filewriter configuration without directly modifying the ICP's output to the `runInfo` topics a script will be used. Things like instrument name and other fields that do not change between instruments can be added here but there are a few gaps that will need to be streamed: +- Stuff in root of file - things like inst name that can be derived from topic are ok, things that cannot be, like experiment identifier, DAE modes etc +- Events in `detector1_events` - currently not being forwarded +- Sample environment is tricky - we need to know what blocks to put in the file template, it's not as simple as just going "anything with the PV prefix of IN:ZOOM" although we could add to the script to look at the forwarder status and check in the currently forwarded PVs +- Fields derived from detector events such as `total_counts` + +The general structure of the file can be written as this will likely not differ between instruments (at least not much) so this will be added in by the script that forwards to `ALL_runInfo` + +NB. I couldn't use the NeXus-Constructor for this as it no longer takes a NeXus file as an input, the version on master doesn't allow top-level fields or arbitrary groups, and there aren't many things in the ZOOM file for example that are in `/raw_data_1/instrument` which is where the NeXus constructor puts components by default. Because of events also being stored in the entry (`raw_data_1`), the NeXus-Constructor crashes when trying to output to a JSON file as it tries to write the events out which cannot be worked around unless you modify the source code to ignore that particular group. Even with this done the constructor is still quite unresponsive because of the amount of data in the in-memory NeXus file. + +#### Update - 23/10/2020 (end of cycle) + +We managed to get the filewriter to work and output _a_ file containing one of each type of stream (`hs00`, `f142`, `ev42`) on MERLIN, which proved that the data streaming stack could be used at ISIS on a very basic level. For some reason when trying to output all events, histograms and sample environment the filewriter refused to write any sample environment data. at first we thought this was because the forwarder had crashed, which it had, however even with it up and running again the filewriter did not write any `f142` streams. Besides this it did manage to write histograms from 9 monitors at once as well as events from MERLIN. these can be seen in the files on NDHSPARE62. +Metadata such as start time and other ISIS-specific static data was not added to the file but this could be added easily in the future by the ICP. + + diff --git a/doc/specific_iocs/dae/datastreaming/Datastreaming---Sample-Environment.md b/doc/specific_iocs/dae/datastreaming/Datastreaming---Sample-Environment.md new file mode 100644 index 000000000..87a12351b --- /dev/null +++ b/doc/specific_iocs/dae/datastreaming/Datastreaming---Sample-Environment.md @@ -0,0 +1,18 @@ +# Sample environment forwarding + +All IBEX instruments are currently forwarding their sample environment PVs into Kafka. This is done in two parts: + +### BlockserverToKafka +This is a Python process that runs on each NDX (see code [here](https://github.com/ISISComputingGroup/EPICS-inst_servers/tree/master/BlockServerToKafka)) it monitors the blockserver config PVs and any time the config changes it pushes a new configuration to the forwarder, via a Kafka topic `forwarder_config`. This is a process written and managed by IBEX developers. + +The instrument name for the BlockServerToKafka service is `BSKAFKA`. + +### Forwarder +This is a Python program responsible for taking the EPICS data and pushing into Kafka. ISIS currently has two instances of the forwarder running (one for the production and one for development). They are both running as services (Developer Forwarder and Production Forwarder) under `nssm` on NDADATASTREAM, which can be accessed via the `ibexbuilder` account. The configuration files and logs for these forwarders are located in `C:\Forwarder\dev_forwarder` and `C:\Forwarder\prod_forwarder`. The actual source lives in `C:\forwarder\fw_py`, updating it is a case of running `git pull` and re-installing the requirements. + +Source for the forwarder is available [here](https://github.com/ess-dmsc/forwarder) + +On NDADATASTREAM the forwarder is run as a service with `nssm` - this is responsible for things like log file rotation and configuring which config files are used with the forwarder. To edit these services run `nssm edit ProdForwarder` or `nssm edit DevForwarder` which will open a GUI for doing so. +To start/stop/restart the services use `nssm [start/stop/restart] [service name]` + +_NB: The forwarder was previously written in C++ but has now migrated to Python instead._ \ No newline at end of file diff --git a/doc/specific_iocs/dae/datastreaming/Datastreaming-How-To.md b/doc/specific_iocs/dae/datastreaming/Datastreaming-How-To.md new file mode 100644 index 000000000..4ca260538 --- /dev/null +++ b/doc/specific_iocs/dae/datastreaming/Datastreaming-How-To.md @@ -0,0 +1,18 @@ +# Data streaming how-to guide + +This is a guide for basic operations using either the development or production Kafka clusters we use for data streaming at ISIS. + +Note that there are many ways to do the following, what is written here is the way commonly done at ISIS on our development and production clusters. Something like `kafka-tool` is a nice GUI that will list topics, brokers, etc and create or delete topics. You may have more luck running things like `kafkacat`, `kafkacow` or any of the official Kafka scripts under the [Windows subsystem for linux](https://docs.microsoft.com/en-gb/windows/wsl/install-win10) + +## Topic operations + +Pushing to one topic does not necessarily mean that the other topics in the cluster receive the data and replicate it, so use with caution. If you need to create a topic that is replicated through all of the topics you should probably follow [this guide](https://coralogix.com/blog/create-kafka-topics-in-3-easy-steps/) by `ssh` on the actual server machines themselves. + +### Create a new topic +There is a script in the [isis-filewriter](https://github.com/ISISComputingGroup/isis-filewriter/tree/master/scripts) repository which will create a script for you. It takes a broker, topic name, and number of partitions (usually 1 partition is fine for a basic topic, more for concurrent streams) + +### List topics +To list topics on a broker you need to use the metadata API. GUIs such as offset-explorer can do this quite easily, or you can use [Kafkacat](https://github.com/edenhill/kafkacat) or [Kafkacow](https://github.com/ess-dmsc/kafkacow) + +### Viewing or "consuming" data from a topic +Like above, the best way of doing this programmatically is by using the API in your given language. [Kafkacow](https://github.com/ess-dmsc/kafkacow) does this and de-serialises from the relevant flatbuffers schema the data has been pushed in such as `ev42` for event data. diff --git a/doc/specific_iocs/daq/MuSR-Rotation-control.md b/doc/specific_iocs/daq/MuSR-Rotation-control.md new file mode 100644 index 000000000..53e6b114b --- /dev/null +++ b/doc/specific_iocs/daq/MuSR-Rotation-control.md @@ -0,0 +1,13 @@ +# MuSR Rotation control + +MuSR can rotate it's detectors and magnets about the sample position. The two valid orientations are either longitudinal (aligned with the beam) or transverse (rotated 90 degrees relative to the beam). + +Changing the orientation is a manual procedure, however the orientation state of MuSR is measured using an analog voltage. This analog voltage is fed into the same DAQMX box which is used for the zero field system, using a fourth (`SPARE`) channel. This channel is exposed via the `ZFMAGFLD` ioc. + +The logic and DB records for MuSR's rotation are in `custom_records.db` in MUSR's settings area (these are loaded by the `INSTETC` ioc). The ComponentSwitcher blockserver component monitors the rotation state of MuSR and may change the blockserver configurations if the state changes. + +Condition
(Channel 4 voltage) | Action | +---------------------------------------|------------------------------------------------------------------------------| +voltage >= +4V | If orientation is not transverse, set orientation to transverse, and update the config files accordingly.
If orientation is already transverse, keep orientation set to transverse | +-4V < voltage < +4V | Do not change the current orientation | +voltage <= -4V | If orientation is not longitudinal set orientation to longitudinal and update the config files accordingly.
If orientation is already longitudinal, keep orientation set to longitudinal | \ No newline at end of file diff --git a/doc/specific_iocs/daq/Muon-Separator-Power-Supply.md b/doc/specific_iocs/daq/Muon-Separator-Power-Supply.md new file mode 100644 index 000000000..96146189a --- /dev/null +++ b/doc/specific_iocs/daq/Muon-Separator-Power-Supply.md @@ -0,0 +1,52 @@ +# Muon Separator Power Supply + +The Muon Separator Power Supply is controlled by a National Instruments DAQ-Mx, since IBEX cannot communicate directly with the Separator PSU. The configuration for the DAQ is done in `EPICS\ioc\master\SEPRTR\iocBoot\iocSEPRTR-IOC-01\st-daq.cmd` There is further documentation on how the separator works [here](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/Muon%20FE/Supporting%20the%20Muon%20Separator.pptx). + + +The DAQ uses 4 AI ports and 4 AO ports used as such: + +| AI Port| Use | --- | AO Port | Use | +| --- | --- | --- | --- | --- | +| 0 | Read Kicker Voltage | --- | 0 | Not used | +| 1 | Read Kicker Current | --- | 1 | Not used | +| 2 | Read Separator Voltage | --- | 2 | Set Separator Voltage | +| 3 | Read Separator Current | --- | 3 | Not used | + +Each port sends or receives a voltage in the range 0-10V. The actual separator PSU operates between 0-200kV and 0-2.5mA, so some scaling is needed + +The calibration (scaling) relationship is: +- Separator Voltage: 0-10V (DAQ) = 0-200kV (PSU) +- Separator Current: 0-10V (DAQ) = 0-2.5mA (PSU) + +## Separator stability + +The calculation of the stability of the muon power supplies is calculated in two stages. First, the number of data points which lie out of the current and voltage stability boundaries is calculated from a sample set of data from the cDAQ. The record which calculates this is updated with every update from the voltage from the device, both voltage and current aren't used to trigger this calculation as this could double count some data. + +This data accumulates over one second until a timekeeper PV activates and sends the total number of data points which lie outside the stability boundary recorded in that second to a circular buffer. This buffer is kept in an aSub record, its default length is 600s (10 minutes). + +To recover the amount of time which was spent outside of the stable boundary, multiply the number of samples collected by the time each sample represents, (1/sample frequency). This value is exposed in a PV for a nagios alarm to monitor. + +The stability boundaries are defined as: +1. Voltage stability: `V_lower_limit < V_measured < V_upper_limit` +1. Current stability: `I_measured <= (I_stable + I_limit)` + +**N.B**: When the stability limits are set so that the signal is always unstable, then the unstable time will sometimes not equal the buffer size due to data loss. This will be a large problem if the frequency is larger than the number of elements. + +## Moving average Filtering +After installing the separator software on MUONFE, we found interference on the voltage input. To counter this, a filtering scheme was developed, described in the python notebooks here: https://github.com/ISISComputingGroup/separator-signal-analysis + +This takes a moving average of two adjacent points, although a different stride length (number of indices between +the two points) of 20 was also considered. The stride length is currently a constant (1) in the c++ source, and the python test libraries + +An aSub record to perform this is now referenced in the separator's voltage db file. The data flow for voltage now looks like: + +1. Raw data `$(P)DAQ:VOLT:_RAW` +1. Calibration `$(P)_APPLYVOLTCALIB` + - Calibrated data `$(P)CALIBRATE:VOLT` +1. Filtering `$(P)_APPLYVOLTFILTER` + - Filtered Data `$(P)FILTERED:VOLT` + +To remove the filtering from the data flow, change the INAA field of `_STABILITYCHECK` in the separator stability db file to point at `$(P)CALIBRATE:VOLT`. + +### Data loss at large stride lengths +As a consequence of performing a moving average, the shape of voltage input data is reduced by the stride length. This means that there will be fewer data points than the (unfiltered) current data, so the assumption of the separator stability calculation that there are the same number of points for both current and voltage is no longer valid. It was decided that a stride length of 1 does not 'stretch out' the voltage trace enough to break the assumption. A longer stride length (e.g. 20) would be a significant source of error and need to be addressed. \ No newline at end of file diff --git a/doc/specific_iocs/fluxgates_magnetometers/Bartington.md b/doc/specific_iocs/fluxgates_magnetometers/Bartington.md new file mode 100644 index 000000000..3637d75e1 --- /dev/null +++ b/doc/specific_iocs/fluxgates_magnetometers/Bartington.md @@ -0,0 +1,13 @@ +# Bartington + +There is a system in use by the North Side Muon beamlines from [Bartington](https://www.bartington.com) that will be used to measure the magnetic field in the sample position. These will be read in the same way as the existing magnetic sensors on those beamlines, which is via a NI DAQ 9215 card with a simple calibration to translate the Voltage output from the fluxgate into a field reading in each of three directions: X, Y, and Z. + +The PSU for the system is a [model PSU 1](https://www.bartington.com/psu1/) from Bartington. + * an Operator's manual is here: `\\isis\shares\Manuals\Bartington\PSU1_OM2443.pdf` + +The Probe has the identifier MAG-13MC70 (see [MAG-13 series](https://www.bartington.com/mag-13/)) + * an Operator's manual is here: `\\isis\shares\Manuals\Bartington\Mag-13_OM3143.pdf` + +Digital manuals have yet to be located. + +We are able to borrow the hardware to verify the Voltage to Gauss/Tesla relationship and to test the IOC functionality. diff --git a/doc/specific_iocs/fluxgates_magnetometers/Group3-Hall-Probe.md b/doc/specific_iocs/fluxgates_magnetometers/Group3-Hall-Probe.md new file mode 100644 index 000000000..18ae6c718 --- /dev/null +++ b/doc/specific_iocs/fluxgates_magnetometers/Group3-Hall-Probe.md @@ -0,0 +1,25 @@ +# Group3 Hall Probe + +| - | - | +| - | - | +| IOC | `G3HALLPR` | +| OPI | `Group3 Hall Probe` | +| Manual | `\\isis\shares\ISIS_Experiment_Controls\Manuals\Group3__Hall_Probe` | + +## Description + +This is a hall probe IOC, which is used as part of HIFI's zero-field system. The HIFI zero-field uses two devices (and correspondingly, will use two `G3HALLPR` IOCs), which are on separate serial ports. + +Each IOC can talk to 3 probes, which can be named via the `NAME0`..`NAME2` macros (e.g. `X` or `-Y`). + +A multiplier can be set for each field via the `SCALE0`..`SCALE2` macros (usually this will be either `1` or `-1`). + +The scan rate for both field and temperature are independently specified as IOC macros. + +## State machine + +An SNL program is in use to automatically change the measurement range of each probe. + +The thresholds at which this SNL program moves up/down a range are specified in PVs of the form `$(P)$(SENSORID):STATEMACHINE:R0:UP` - which in this case would be the threshold, in Gauss, at which to move up from `r0` (Range 0, 0.3 Tesla) to `r1` (Range 1, 0.6 Tesla). These thresholds may be modified via PV sets in the configuration, if needed. + +To allow the device to settle in a new range, there is also a minimum delay after changing states before the next state change - this is in PVs of the form `$(P)$(SENSORID):STATEMACHINE:STATE_CHANGE_DELAY`, which again can be modified by PV sets in the configuration if desired. By default it is 2 seconds. \ No newline at end of file diff --git a/doc/specific_iocs/fluxgates_magnetometers/Metrolab-PT2025-Teslameter-IOC.md b/doc/specific_iocs/fluxgates_magnetometers/Metrolab-PT2025-Teslameter-IOC.md new file mode 100644 index 000000000..64cc261a4 --- /dev/null +++ b/doc/specific_iocs/fluxgates_magnetometers/Metrolab-PT2025-Teslameter-IOC.md @@ -0,0 +1,5 @@ +# Metrolab PT2025 Teslameter + +Magnetometer for Metrolab PT2025 Teslameter. + +There are lots of setting that can be made from the front panel. The IOC assumes it is in the mode where the current reading is continuously written to the port. diff --git a/doc/specific_iocs/fluxgates_magnetometers/Zero-Field-Magnetometer-IOC.md b/doc/specific_iocs/fluxgates_magnetometers/Zero-Field-Magnetometer-IOC.md new file mode 100644 index 000000000..f4a83d384 --- /dev/null +++ b/doc/specific_iocs/fluxgates_magnetometers/Zero-Field-Magnetometer-IOC.md @@ -0,0 +1,49 @@ +# Zero-field Magnetometer (NI cDAQ) + +## Purpose + +The magnetometer reads magnetic field strengths from the beamline. This IOC takes the magnetometer signal, applies corrections and performs a check to make sure that the magnetometer has not been overloaded. + +This IOC is designed to be used in conjunction with a sequencer IOC, which uses the field values to calculate corrections needed to maintain a magnetic field strength. When a new field value is required, the sequencer will process the `$(P)TAKEDATA` PV of this IOC. When the new field values are available, this IOC processes `$(PVPREFIX):ZFCNTRL_01:INPUTS_UPDATED.PROC`, which it expects to allow the sequencer IOC to continue its calculations. This PV can be changed by setting the `$(SQNCR)` macro. + +## Corrections applied to the input fields + +The maths which defines these operations is described in finer detail [here](../magnets/Zero-field-controller). + +1. Magnetometer range scaling + - The three field components are multiplied by a scaling factor to transform the input on to the range of the magnetometer. The scaling factor is set by the macro `$(RANGE)`, and is held in the PV `$(P)RANGE`. + +2. Applying offsets + - Each component has a unique offset which is subtracted from the scaled data. These offsets are set as the macros `$(OFFSETX), $(OFFSETY), $(OFFSETZ)`. These can also be set by the PVs `$(P):X:OFFSET` and from the OPI. + +3. Multiply with sensor matrix + - To obtain the corrected field strengths, the scaled and offset values need to be multiplied by a 'field sensor' matrix. + - Macros define the sensor matrix: `$(MATRIX_X_Y)`, where X and Y are the row and column indices. + - PVs hold the elements of the sensor matrix: `$(P)SENSORMATRIX:XY`. These PVs can also be changed on the OPI. + - The multiplication itself is done as an aSub record. This allows us to use the libraries available to us through GSL. + - Set up using GSL vector and matrix structures, performing the matrix multiplication is a relatively straightforward call to a vector/matrix multiplication BLAS routine in GSL. + - The documentation for GSL data structures used is [here](https://www.gnu.org/software/gsl/doc/html/vectors.html). + - The GSL documentation for BLAS routines is lacking (see [documentation](https://www.gnu.org/software/gsl/doc/html/blas.html)). To get more in-depth information about the arguments used I would look up the routine in the [BLAS documentation](http://www.netlib.org/blas/). + +4. Output + - The corrected field strengths are written directly to the PVs `$(P)X:CORRECTEDFIELD` from the aSub record. + - The aSub record also calculates the magnitude of the corrected field and places it in `$(P)FIELDSTRENGTH`. + +## Overload + +High field values will overload the magnetometer, which is detected in this IOC. The overload condition for the EMU magnetometer is: + +`max(measured_field) > magnetometer_range * 4.5` + +Where: + - max(measured_field) is the largest component of the 3 scaled (but not offset or matrix multiplied) input field strengths. + - magnetometer_range is the scaling factor used in the magnetometer range scaling + - 4.5 is a constant taken from the EMU VI. This value may be different for other magnetometers. + +## Process order + +A requirement of this IOC is to maintain as consistent a time as possible between measuring the data and performing calculations using it. To achieve this, no records in this IOC scan. Instead, each record forward links to the next record to be processed in order. This is outlined below: + +`TAKEDATA` is processed at regular intervals by the sequencer which is running in the `ZFCNTRL_01` ioc. + +![](magnetometer_daq.png) \ No newline at end of file diff --git a/images/magnetometer_daq.png b/doc/specific_iocs/fluxgates_magnetometers/magnetometer_daq.png similarity index 100% rename from images/magnetometer_daq.png rename to doc/specific_iocs/fluxgates_magnetometers/magnetometer_daq.png diff --git a/doc/specific_iocs/gas_liquid_handling/BioLogic-MPS-70-4-(stopped-flow).md b/doc/specific_iocs/gas_liquid_handling/BioLogic-MPS-70-4-(stopped-flow).md new file mode 100644 index 000000000..3d8346230 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/BioLogic-MPS-70-4-(stopped-flow).md @@ -0,0 +1,11 @@ +# BioLogic MPS 70-4 + +This is not a device we support as a group (there is no obvious documentation on how it can be controlled remotely, other than with the supplied software), but is set up independently using it's own proprietary software. It was last seen on SANS2D. + +The following are some notes for context should a support call be received: +* The device outputs a trigger signal via a cable hard wired to the DAE +* The device was last used connected to a laptop in the sample area with a remote desktop connection to this from the cabin over wired ethernet. The Laptop ran the manufacturer software +* The last time we saw it, it was connected by USB. a previous device used serial to the laptop. There is some suggestion the USB connection may time out if idle for too long. +* In any case the device itself is not connected to the ISIS network or the moxa and is not accessed through IBEX. + +See [here](https://github.com/ISISComputingGroup/IBEX/issues/8528) for some further details. \ No newline at end of file diff --git a/doc/specific_iocs/gas_liquid_handling/DMA4500m-Density-Meter.md b/doc/specific_iocs/gas_liquid_handling/DMA4500m-Density-Meter.md new file mode 100644 index 000000000..ebeff14ce --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/DMA4500m-Density-Meter.md @@ -0,0 +1,71 @@ +# DMA4500m Density Meter + +The device's IOC was originally developed at ESS, then updated to match ISIS requirements and conventions. The original code is available [here]( https://bitbucket.org/europeanspallationsource/m-epics-dma4500m/src/master/). + +It is a densitometer which currently measures density and temperature. + +## Documentation +Vendor documentation is available at `\\ISIS\shares\ISIS_Experiment_Controls\Manuals\AntonPaar__DMA4500`. + +## Connection Details + +| RS-232C Specifications | | +|---------------|------------------| +| Baud rate | 9600 Baud | +| Stop bits | 1 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | None | + +Notes: + - Commands and returned values are terminated with CR + - Only one command should be sent per second + +## Quirks +#### Raw data and temperature setpoint +According to the device manual, the `getrawdata` command returns the "current raw data values". Judging from the device output, the response is in this format: +``` +{density};{temperature};{temperature setpoint};{sample id} +``` +where: +- **density** and **temperature** are "instant" measurements of the current sample density and temperature. The seem to oscillate slightly if the command is run multiple times in a row, so the error is likely quite high. +- **temperature setpoint** is the current temperature setpoint. After a `settemperature` command is sent to the device, this setpoint immediately updates to the value specified in `settemperature` and the "instant" temperature measurement can be observed to ramp up/down towards the setpoint +- **sample id** is a value that is increased by 1 every time a measurement is run (at least when started by the `start` command, possibly when started through the device GUI as well) + +## Simulation +The IOC logic is fairly complex and uses features not supported by RECSIM, so RECSIM has not been implemented for this device. + +## Automeasure +When automeasure is enabled, the IOC periodically starts a measurement without user intervention. 3 PVs are involved in this: +- `AUTOMEASURE:FREQ`: the interval between measurements (in seconds) +- `AUTOMEASURE:ENABLED`: a binary record that's 1 when automeasure is enabled, 0 otherwise (default) +- `AUTOMEASURE`: the `calcout` PV responsible for starting measurements + +`AUTOMEASURE` runs every second, using `AUTOMEASURE:FREQ` and `AUTOMEASURE:ENABLED` as its inputs. When automeasure is enabled, the value of this PV is set to the interval specified in `AUTOMEASURE:FREQ`. Every second, the PV's value is decreased until it reaches 0; when this happens, a new measurement is started and the value is reset to `AUTOMEASURE:FREQ`. The countdown is suspended while a measurement is running (through an SDIS field) so we're only counting the interval between two measurements. In pseudocode: + +``` +def process_pv(): + if not automeasure_enabled: + count = max(automeasure_freq, 1) + else: + if count = 0: + count = automeasure_freq + else: + count-= 1 + return count +``` + +Note that a measurement is started whenever the value of `AUTOMEASURE` is 0, so if `AUTOMEASURE:ENABLED` is 0 but `AUTOMEASURE:FREQ` is also 0 we have to manually set the count to a non-zero value to avoid starting a measurement. This has the side effect that there will always be at least a 1 second delay between enabling automeasure and the start of the first measurement. + +### Delays when running on the real device +I have observed that when the automeasure code is run on the real device the interval between measurements can be inconsistent. This seems to be due to a random delay when starting measurements: the IOC tells the device to start a measurement, but the device takes a few seconds to actually start one. This can lead to longer intervals than the user may have expected. + +## Measurement Mode +The data sent back by the device is parsed according to the current measurement mode, which is set by `$(MEASUREMENT_MODE)` macro. Currently, the only supported mode is `DENSITY_ONLY` (for measuring density, temperature and measurement validity). + +From the device's point of view, the measurement results are returned by the `getdata` command. The quantities to be measured and their position in the results list are determined by the active "method". Methods are defined by the user using the device's GUI, and they can include a large number of different quantities in any order specified by the user. The device does offer a `getdatahead` command to get the name and position of the quantities returned by `getdata`, but there is no easy way of parsing the output of this command to automatically make sense of the results list. For this reason, we need to manually add a measurement mode for every method used at ISIS, and users must make sure to set the `$(MEASUREMENT_MODE)` macro in IBEX to match the method they're using. + +## IOC diagram: +![Diagram of DMA4500M IOC](dma4500m_ioc_diagram.png) + + diff --git a/doc/specific_iocs/gas_liquid_handling/Edwards-Turbo-Instrument-Controller.md b/doc/specific_iocs/gas_liquid_handling/Edwards-Turbo-Instrument-Controller.md new file mode 100644 index 000000000..00042e2c4 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/Edwards-Turbo-Instrument-Controller.md @@ -0,0 +1,5 @@ +# Edwards TIC + +The Edwards turbo instrument controller (TIC) is a relatively straightforward turbo pump controller. In addition to the turbo pump controls, it also has three measurement gauges which can be read back into the OPI. Each gauge can be disabled via macros to prevent disconnected errors appearing. + +This driver was given to us by the Daresbury lab. This was added to the IBEX codebase with a few changes to adhere to IBEX coding standards. \ No newline at end of file diff --git a/doc/specific_iocs/gas_liquid_handling/JASCO-PU--4180-HPLC-Pump.md b/doc/specific_iocs/gas_liquid_handling/JASCO-PU--4180-HPLC-Pump.md new file mode 100644 index 000000000..968fdbcef --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/JASCO-PU--4180-HPLC-Pump.md @@ -0,0 +1,87 @@ +# JASCO HPLC Pump + +## Documentation + +Documentation is available for the pump at `\\ISIS\shares\ISIS_Experiment_Controls\Manuals\Jasco__PU-4180 HPLC Pump` + +## Connection Details + +| RS-232C Specifications | | +|---------------|------------------| +| Baud rate | 4800 Baud | +| Stop bits | 2 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | Xon/Xoff control | + +Notes: + - Communication command is terminated with CR. + - Communication return is terminated with CR and LF. + - When connecting the pump to the MOXA, be sure to use a different port from the Knauer pump. This will allow the Knauer to use its own port as the one for the Jasco will be locked by its IOC (which runs continuously). + +## Simulation + +The driver has both a state machine and fairly complex logic in the IOC, therefore RECSIM for the device is not particularly supported. + +## Pump Control + +The device has the ability to control a number of parameters for a pump: + +- Flowrate (in mL/min): `FLOWRATE:SP` +- Pressure Maximum and Minimum (in kg/cm^2): `PRSSURE:MAX:SP`, `PRESSURE:MIN:SP` +-- Setting these values will result in an error output and stop the pump if exceeded during a executed program. +- Composition components (A, B, and, C in %): `COMP:{}:SP`. + +The user can then start the pump by using the `START:SP` record. The pump can be stopped using the `STOP:SP` + +The user can also select a timed run for either a set time or volume. The user must first set a time `TIME:RUN:SP` (in seconds) or volume `TIME:VOL:SP` (in mL). Then a pump can begin by using the `TIMED:SP` record. + +The calculations for the time for a set volume (done in `TIME:VOL:CALCRUN`) and the calculation for the pumped volume for a set time (done in `TIME:RUN:CALCVOL`) use `FLOWRATE:SP:RBV`, not `FLOWRATE:SP`. The RBV value is different from the setpoint if the setpoint has never been set, in which case using the setpoint would produce an invalid time or volume. + +Note that the Jasco IOC only allows one of these two values to be modifiable at a time (which then sets the other) - which method is used needs to be explicitly set via `TIME:CALC:SP`. This is to avoid race conditions i.e. if a user sets both `TIME:VOL:SP`and `TIME:RUN:SP` at the same time (for example via caLab or scripting) they could not be sure which one wins. + +NB: The devices display screen does not show current pump status information when in operation. + +## Additional commands not in the manual + +The following status command has been confirmed by JASCO and is used in the IOC driver, although it is not documented in the manual: +``` +command : { status load p } +A status value is a bitwise OR. + +Bit0=0: Pump off +Bit0=1: Pump on +Bit1=1: Program hold +(Bit2-3: Reserved) +Bit4-5=1: Program stop +Bit4-5=2: Initial run +Bit4-5=3: Program run + +Data value is returned as follows: + 0: Pump off + 1: Pump on (Program stop) +33: Initial run (Initial run, Pump on) +49: Program run (Program run, Pump on) +51: Program hold (Program run, Pump on, Program hold) +``` +## Troubleshooting + +There are three known issues with the pump that have been confirmed by JASCO. They have detailed the potential issues and the work around that is associated with them. The design of the pumps driver should mitigate them, but this information should help in the event these are encountered during support: + +### Pump stops responding + +This issue involves the pump being hard locked, and completely unresponsive. No data will be returned from the device, and the front panel will be unresponsive. The only way to resolve this is to power cycle the device. This is caused by the `flowrate set` command being executed at the same moment as a program step (such as setting component values). A script is available in the IOC directory called `exercise_jasco4180.py` that can be used to test if the device crashes under use. The driver has been designed to be resilient against this crash and should always successfully finish. This can be used to ensure new driver changes do not introduce device breaking bugs. + +### Pump freezes with 'Program busy' + +This issue involves the pump freezing and not interpreting commands briefly, and can result in commands being skipped. This is caused by the `fileno set` (Set a program file number) being executed while the run status is in `Initial` or `Run`. When the program file is opened other commands not related to creating a time program will not be accepted. You must ensure the program is closed after setting the time program. + +### Composition control is locked + +This issue involves the composition being locked into a manual mode, where no compositions will be accepted remotely, and the single channel (100% A, B, C, or, D) can only be switched using the front panel. This occurs when a time program has not been written correctly. For example, if the maximum number of lines in a program (64) has been exceeded or not closed. This should be resolved by first ensuring the file is closed, using `1 fileno set`, and then performing a re-run program using initial conditions `8 pump set`. Note that you might want to ensure you have a flow rate of 0 to ensure no liquid is pumped while troubleshooting. + +### Target runtime calculation logic + +Work was completed on the JASCO to update its calculation logic for pump runtime in [#4720](https://github.com/ISISComputingGroup/IBEX/issues/4720). This flow diagram illustrates this: + +![Runtime calculation logic](jasco_runtime_logic_flow.png) diff --git a/doc/specific_iocs/gas_liquid_handling/Knauer-1050-HPLC-Pump.md b/doc/specific_iocs/gas_liquid_handling/Knauer-1050-HPLC-Pump.md new file mode 100644 index 000000000..5ef2dd3dd --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/Knauer-1050-HPLC-Pump.md @@ -0,0 +1,56 @@ +# Knauer 1050 HPLC Pump + +## Documentation + +Documentation is available for the pump at `\\ISIS\Shares\ISIS_Experiment_Controls\Manuals\knauer 1050 HPLC` + +## Connection Details + +| RS-232C Specifications | | +|---------------|------------------| +| Baud rate | 115200; 38400, 19200, 9600 Baud | +| Stop bits | 1 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | Hardware | + +Notes: + - Communication command is terminated with CR. + - Communication return is terminated with CR. + + +## Specifications + +### Pump heads + +The 1050 was designed to provide exceptionally precise and reliable solvent delivery for a wide range of HPLC applications. The pump can be fitted with either a 10 or 50 ml/min pump heads that will be RFID recognised for auto set-up. Depending on the head used there are maximum flow rates, pressures and accuracies: + +| | 10 ml/min | 50 ml/min | +|------------------------|-------------------------------------------|-----------------------------------------| +| Flow rate range | 0.001 to 9.000 | 0.01 to 50.00 ml/min | +| Max. delivery pressure | 400 bar (40 MPa, 5 800 psi) < 10 ml/min | 200 bar (20 MPa, 2 900 psi) < 25 ml/min | +| | | 150 bar (15 MPa, 2 200 psi) < 50 ml/min | +| Flow rate accuracy | deviation ≤ 1 % | deviation ≤ 1 % | +| Flow rate precision | RSD ≤ 0.1 % | RSD ≤ 0.3 % | + +### Remote/Local modes + +The pump starts in `LOCAL` mode, and when in this state you are able to access the devices front panel and make changes as well as remotely poll the device status. The device mode can be toggled into `REMOTE` mode using the `MODE:SP` record, and this will result in the device accepting remote instructions but it will lock the devices front panel. + +### Gradient Controls + +The device can pump with a composition of 4 components, A, B, C and, D. These gradients can be set using the `COMP:X:SP` (replacing X with the desired component). However the sum of these gradients must be equal to 100%. If not correctly set then you will be unable to start the pump until this is resolved. + +### Pressure Limits + +The high and low pressure limits can be set using the `PRESSURE:MIN:SP` and `PRESSURE:MAX:SP` records. If a limit is reached during a pump then the pump will stop. + +### Flow Rate/Min + +The devices desired flow rate per minute can be set using the `FLOWRATE:SP` record. + +### Pump Control + +The user can set the pump to run for either a set time/volume, or a continuous pump. These can be set using `STOP:SP`, `START:SP`, and, `TIMED:SP`. + +To specify a run time the `TIME:SP` record can be used, and to set a desired volume the `VOL:SP` record can be used. diff --git a/doc/specific_iocs/gas_liquid_handling/Knauer-k-6.md b/doc/specific_iocs/gas_liquid_handling/Knauer-k-6.md new file mode 100644 index 000000000..b2c55f59d --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/Knauer-k-6.md @@ -0,0 +1,30 @@ +# Knauer K-6 + +## Documentation + +Documentation is available for the pump at `\\ISIS\Shares\ISIS_Experiment_Controls\Manuals\Knauer K-6 Valve Drive` + +The Knauer Electric Valve Drives K-6 is designed for automatic control of the +6-port/3-channel-injection valve, 6-port/2-channel-switching valve, 7-port/1- +channel-switching valve by K-6 drive + +## Valve head + +The 7-port/1-channel-switching valve can be switched to 6 positions. Each switch +command will result in a 60° turn, so that the 6th switch command returns to the +first position. + +## Specifications + +| RS-232C Specifications | | +|---------------|------------------| +| Baud rate | 9600 Baud | +| Stop bits | 1 bit | +| Start bits | 1 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | XON/XOFF | + +Notes: + - Communication command is terminated with CR. + - Communication return is terminated with CR. diff --git a/doc/specific_iocs/gas_liquid_handling/MKS_PR4000B.md b/doc/specific_iocs/gas_liquid_handling/MKS_PR4000B.md new file mode 100644 index 000000000..ec2206b39 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/MKS_PR4000B.md @@ -0,0 +1,29 @@ +# MKS PR4000B + +## Communications + +Serial settings (baud rate, parity) can be changed on the front panel of the device. Ensure the IOC matches the physical device. Device communicates using a straightforward ASCII protocol with carriage-return terminators (`\r`). + +## Channels + +Each PR4000B unit can talk to two sensors independently. The controller can acquire a large amount of diagnostic data alongside the "primary" readings for each channel. This diagnostic data polls more slowly than the main readings, as they probably change less frequently. + +## IOC + +The IOC is forked from a Diamond repository, however it has been heavily modified to fit ISIS standards and working practices. In reality it should be treated as "our own" driver rather than as a "vendor" driver. + +It has a couple of simple aSub records for converting the string representation of a unit to it's numeric representation. These are to get around a limitation in EPICS that `mbbi` records can only have 16 states (20 were needed). These aSub records could be removed in future if future versions of EPICS support MBBI records with more states (at the time of writing, this was not a feature of EPICS 7). + +## Serial settings + +The two MKS PR4000B units on NIMROD's gas panel use different serial settings. Settings are as per the table/photo below + +| - | Unit 1 (left) - MKSPR4KB_01 | Unit 2 (right) - MKSPR4KB_02 | +| --- | --- | --- | +| Baud | 9600 | 9600 | +| Bits | 7 | 7 | +| Parity | even | odd | +| Stop | 2 | 2 | +| Null modem required | No | No | + +![](nimrod_gas_panel.png) \ No newline at end of file diff --git a/doc/specific_iocs/gas_liquid_handling/NIMA-Trough.md b/doc/specific_iocs/gas_liquid_handling/NIMA-Trough.md new file mode 100644 index 000000000..eedc560d7 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/NIMA-Trough.md @@ -0,0 +1,35 @@ +# NIMA Trough + +The NIMA Trough is used to fabricate thin films of material. + +## Specifications + +The implementation for IBEX uses an lvDCOM solution. The IOC connects to a VI interface to process set points and read back values. When the IOC is started, it attempts to boot the LabVIEW VI. In general use, the VI will be open with its graphing view along side the IBEX client. + +## Running the IOC + +The NIMA trough IOC is not compatible with versions of the VI that are older than 2019-02-15. Ensure you update and VI drivers you have locally from source safe. Updated drivers should be available on DEMO. + +## Simulation + +When not connected to a physical device the NIMA Trough VI has an example data file (graph plot) and will respond to record processing. This allows you to observe some of the state and field behaviour. For example, you can process opening the barriers, and the VI will indicate the barriers are opening. + +## Physical Setup + +The Nima trough is a large metal container in which is a tray with a motor and sensor connected. This is connected to a control box (beige about 30cm x 15cm x 5cm). The control box has two LEDs on it labelled V1 and V2 they should both be lit for correct operation. The LED are + +- V1: unknown (maybe power or connection to the motors) +- V2: connection to the serial port. If this is unlit there is a problem with the serial communication either at the software level or hardware level. During recent diagnosis the VI appeared to claim it could stop the barrier (but not start the barrier) and there were lots of OKs in the message box but the cable was broken so it was not communicating; i.e. be wary that the VI is claiming it can communicate. + +## Troubleshooting + +### Unable to start a run + +The NIMA Trough GUI can perform 3 runs, and each is individually saved to a file. Once these 3 files have been written too, you are unable to begin further runs. A pop-up on the NIMA Trough warns you that files must be deleted in order to continue recording runs. This is shown below with the 3 runs tab highlighted in red: + +From the IOC perspective though you will simply be unable to begin a run, and there is no warning that the number of runs has been reached, but the pop-up on the NIMA Trough interface should be clear. If you are unable to begin runs from the IOC it's worth checking the NIMA Trough GUI to ensure there are free files available to write to. + +### Units for Area + +By default on the NIMA Trough the area is in cm^2, however this can be changed. Since we use an lvDCOM wrapper around an interface to the main NIMA Trough GUI, we are unable to access the unit information. The EGU of the area record is cm^2 by default, so if there are issues with it having unexpected values it's worth checking to see if the NIMA Trough GUI has had it's units altered. We were informed that it is expected that cm^2 will be used exclusively. + diff --git a/doc/specific_iocs/gas_liquid_handling/WPI-Aladdin-Syringe-Pump.md b/doc/specific_iocs/gas_liquid_handling/WPI-Aladdin-Syringe-Pump.md new file mode 100644 index 000000000..986e2a279 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/WPI-Aladdin-Syringe-Pump.md @@ -0,0 +1,94 @@ +# WPI Aladdin Syringe Pump + +## Documentation + +Documentation is available for the pump at `\\isis\shares\ISIS_Experiment_Controls\Manuals\WPI__Aladdin_1000_syringe_pump` + +## Connection Details + +| RS-232C Specifications | | +|---------------|------------------| +| Baud rate | 19200, 9600, 2400, 1200, or 300 Baud | +| Start bits| 1 bit | +| Stop bits | 1 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | None | + +Command Syntax: + - Communication command is terminated with . + - Communication return starts with and is terminated with . + +Command float formatting: + +The device requires strict formatting on its floats. The output value must fit 5 characters and be within the range `9999. to .9999`. + +## Specifications + +### Networked Pumps + +Multiple pumps can be daisy chained together and controlled via a single IOC. Each pump has an address which can be set using the `ID:SP` record. By default (and in a single pump configuration) this is `00`. IOC now supports talking to up to 4 pumps at the same time without having to modify the `ID:SP`. You can set the macros for ID1 through ID4 in the IOC config and then reference the usual PVs with the :1: through :4: prefixes. E.g. ALDN1000_01:1:... + +A given pump can have it's own unique address defined. So two pumps might not have the same address. Typically pumps are labelled with a sticker `00`, `01` etc. that will indicate this. If you ever need to change this the pump has an method that will allow you to define this (refer to the manuals section on the `ADR` method). + +### Volume Units + +The device has it's own logic for setting the units of volume to be pumped `VOLUME:UNITS` based on the diameter of the syringe installed in the pump `DIAMETER`. In testing it was found that: Diameter <= 14.00mm - uL, Diameter > 14.00mm = mL. A small warning about this has been placed on the OPI to inform users. + +### Rate Units + +The units for the rate can be selected but will only be set when setting the `RATE:SP`. You can resend the same set point to set newly selected rate units. + +## Connection notes + +Use supplied modem telephone cable into pump port labelled **“computer port”**, then use adapter supplied to connect to a female moxa cable and into moxa box. (There is another port on the pump labelled _Network_ - this is not for the main network, but used if you wish to join pumps together in a "pump network") + +Note most of these units have been helpfully labelled by the soft matter group, so this should be indicated like so: + + + + +For one pump it must be given address 0, see below. Additional pumps can be daisy chained using the “network” modem port on the first pump to connect to the “computer” port on a second. The additional pumps must each have a unique address. + +The manual says to power pump on after rs232 cable is attached + +The device has a configurable baud rate. See setup key below + +The device supports two rs232 modes: basic and safe. It looks like LabVIEW uses “basic” and attempts to turn off safe mode. A pump in basic mode can be controlled via an emulator like hyperterm. + +A triangle in top left of LED display is the “rs232 indicator” – indicates valid reception of a command at some point since power on. It looked like this triangle remained after cable was disconnected for example. + +### Notes from the “setup key” section of manual +Hold diameter key to enter setup mode, you will initially get first configuration parameter “PF:” which is power failure mode. It will then cycle through each setup parameter after about 2 seconds. + +To change value of a parameter press an arrow key under value on LED display, to store new value wait two seconds or press any non-arrow key + +The Last two setup parameters are: +* `Ad` - the pump address, should be 00 for just one pump +* `nnnn` – the baud rate, we use 9600 so make sure this number is displayed there + +If r232 error is displayed on LED panel, indicates a timeout in “safe communication mode”. We should not be in this mode anyway. +Before the pump can operate, it needs to know syringe inside diameter – set this from front panel of device + +### Terminal emulator test +If the pump is in basic rs232 mode, connect and send command +VER +followed by carriage return, should get a reply with the firmware version details + +### enabling basic mode if in safe mode + +According to manual you need to send the string `0x2 0x8 SAF0 0x55 0x43 0x3` where `0x` represents the hex value of the character to send. It may be this assumes pump address 0, if you want to set a non-zero addressed pump then you may need to prefix SAF0 with the pump address and then calculate the new checksum (which is what 0x55 0x43 is). See manual. + +### connection cables + +The manual mentions two connector cables, a “GN-PC” variety to connect to a PC and “GN-NET” variety to connect pumps. A separate rs232 adapter is also mentioned. Check it is a GN-PC telephone cable and not a GN-NET telephone cable to connect to this rs232 adapter, I’m not sure if this is identified ion the cable but i think the GN-PC is dark grey and GN-NET is light grey + +It should look like this: + + + +Where the black cable is a Male Moxa cable and the grey cable is plugged into the "PC" port on the pump. + +### Sockets on device + +On the rear of the pump itself are two identical sockets, one labelled "To Computer" and the other "To Network", both under a heading "RS232" (or words to that effect). The cable which is ultimately connected to the MOXA needs to be plugged into the "To Computer" socket. The "To Network" socket is for daisy-chaining pumps together. If for some reason the IOC is receiving replies identical to those being sent, then it could be that the cable is in the wrong socket. diff --git a/doc/specific_iocs/gas_liquid_handling/WPI-SP2XX-Syringe-Pump.md b/doc/specific_iocs/gas_liquid_handling/WPI-SP2XX-Syringe-Pump.md new file mode 100644 index 000000000..c97f16e8b --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/WPI-SP2XX-Syringe-Pump.md @@ -0,0 +1,8 @@ +# WPI SP2XX Syringe Pump + +The WPI SP2XX series syringe pump simply infuses and withdraws liquid from the target. + +## Connecting the pump + +The device requires a specific cable to be connected to a moxa or PC. The cable is grey and has RJ11 to DB9 (female) connections. You should be able to then connect this to a moxa DB9 (male) to RJ45 to communicate with it. The devices front panel will automatically switch to a "Remote" dialog, confirming it is being controlled by the driver. + diff --git a/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_1.png b/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_1.png new file mode 100644 index 000000000..1a83182ca Binary files /dev/null and b/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_1.png differ diff --git a/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_2.png b/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_2.png new file mode 100644 index 000000000..4f90448f1 Binary files /dev/null and b/doc/specific_iocs/gas_liquid_handling/WPI_aladdin_2.png differ diff --git a/doc/specific_iocs/gas_liquid_handling/Watson-Marlow-323-Peristaltic-Pump.md b/doc/specific_iocs/gas_liquid_handling/Watson-Marlow-323-Peristaltic-Pump.md new file mode 100644 index 000000000..8c133d322 --- /dev/null +++ b/doc/specific_iocs/gas_liquid_handling/Watson-Marlow-323-Peristaltic-Pump.md @@ -0,0 +1,30 @@ +# Watson-Marlow 323 Peristaltic Pump + +## Documentation + +Documentation is available for the pump at `\\ISIS\shares\ISIS_Experiment_Controls\Manuals\Watson-Marlow__323` + +## Connection Details + +| RS-232 Settings | | +|---------------|------------------| +| Baud rate | 9600 Baud | +| Stop bits | 2 bit | +| Parity | None | +| Data length | 8 bit | +| Flow control | None | + +Notes: + - Communication command is terminated with CR. + - Communication return is terminated with CR. + +## Pump Control + +The device has the ability to control a number of parameters for a pump: + +- Rotation speed (in rpm with minimum of 3 and maximum of 400): `SPEED` +- Direction (Clockwise or Anti-Clockwise): `DIRECTION` +- Status (Running or Stopped): `STATUS` + + +The user can also start the pump by using the `START:SP` record. The pump can be stopped using the `STOP:SP` record. diff --git a/images/dma4500m_ioc_diagram.png b/doc/specific_iocs/gas_liquid_handling/dma4500m_ioc_diagram.png similarity index 100% rename from images/dma4500m_ioc_diagram.png rename to doc/specific_iocs/gas_liquid_handling/dma4500m_ioc_diagram.png diff --git a/images/jasco_runtime_logic_flow.png b/doc/specific_iocs/gas_liquid_handling/jasco_runtime_logic_flow.png similarity index 100% rename from images/jasco_runtime_logic_flow.png rename to doc/specific_iocs/gas_liquid_handling/jasco_runtime_logic_flow.png diff --git a/images/nimrod_gas_panel.png b/doc/specific_iocs/gas_liquid_handling/nimrod_gas_panel.png similarity index 100% rename from images/nimrod_gas_panel.png rename to doc/specific_iocs/gas_liquid_handling/nimrod_gas_panel.png diff --git a/doc/specific_iocs/loading_rigs/Instron-stress-rig---MiniTower.md b/doc/specific_iocs/loading_rigs/Instron-stress-rig---MiniTower.md new file mode 100644 index 000000000..7ee18f3e6 --- /dev/null +++ b/doc/specific_iocs/loading_rigs/Instron-stress-rig---MiniTower.md @@ -0,0 +1,94 @@ +# Instron stress rig (MiniTower) + +```{note} +This page relates to a newer-style ethernet-connected stress rig, which we talk to via a manufacturer DLL. For documentation about the GPIB-connected rig (i.e. the 50kN rig on ENGIN-X) see [Instron-stress-rig (GPIB)](Instron-stress-rig) +``` + +## Physical setup + +The stress rig has several components: +- The physical rig hardware, in which the sample sits. +- A controller, which plugs directly into the physical rig hardware, and has an ethernet port on the back +- The ethernet port on the controller is plugged into the control PC via a **direct** ethernet connection. This connection should **not** be going via a switch or the ISIS network in any way. + * Note, this "control PC" is **NOT** the `NDX` machine or equivalent, it is a separate machine. Currently a small-form-factor PC named `enginx-stress-1` + * The network between the stress rig controller and `enginx-stress-1` is a point-to-point private network. The relevant network adapter on `enginx-stress-1` should be set to an IP address of `10.0.0.1` with netmask `255.0.0.0`. The stress rig controller is on `10.0.0.2` and should respond to `ping` if it is working. +- The control PC has a physically separate RJ45 port (currently a USB-to-ethernet converter) used to connect to the ISIS network. +- There is an "MMI" controller which plugs into the control PC via USB - and also simultaneously into the Instron controller via a separate cable. + +The manufacturer software ("Console") **MUST** be running on the control PC for the rig to work. Some functions (calibrations, setup, advanced diagnostics) are only available via the "console" software. + +## IOC + +We will run an IOC on the control PC, which can be viewed and eventually controlled from the NDX. + +Due to needing to link against the 32-bit manufacturer DLL (`Arby.dll`) for communication, we need to deploy a 32-bit EPICS build to the control PC. Ticket https://github.com/ISISComputingGroup/IBEX/issues/7326 will define our approach to this computer in more detail. + +The manufacturer software ("Console") must be running in the background while the IOC is attempting to talk - otherwise the rig will not work. + +## Troubleshooting + +### Cannot connect via remote desktop to `enginx-stress-1` + +- Check the machine is powered on - it travels with the stress rig and may not necessarily always be plugged in! +- Check the machine is connected to the ISIS network via it's motherboard port, and to the stress rig via the USB-ethernet adapter. +- Connect a monitor to find out the machine's IP address - it is sometimes slow to update in the nameserver, but can be accessed via remote desktop if you know the IP directly. +- For username/password details for this machine, see keeper. + +### ArbySPY + +The Instron software contains a module called "ArbySPY" which can be used to trace all commands being sent to the rig, no matter their source. As such it is possible to find out what Console is doing to make sure that IBEX can replicate these same commands. + +### Command rejected - frame in control + +Solution: You need to press the physical button on the front of the stress rig to restore computer control. This cannot be done automatically as it trips the hydraulics intermittently when attempted + +### Cannot change channel - "function inhibited - channel not calibrated" + +Solution: Ask scientists to load a calibration for the relevant channel using the Console application. + +### Setpoints are relative rather than absolute + +Verify that the `C1,1` command is **NOT** being used - this is buggy in the Arby interface. Need to use the following sequence instead (this is already done by the protocol file - but any arbitrary commands sent to the rig may also need to be adjusted): + +``` +C500,1;C38,1 +C916,0 +C914,4 +C916,2 +C372 +``` + +### "TestLog" keeps popping up complaining about "next mode must be enabled to send this command" or similar + +This happens when a move is started from IBEX. We are not sure why the rig believes C38 is invalid - but it **is** required in order to actually start the move. + +Solution: ignore these spurious errors. + +### "TestLog" complains about various things as the IOC is started + +Solution: ignore these errors - they do not impact functionality. + +### "TestLog" complains about a wide variety of problems (frame error, fan error, power state, non-corporate pack detected, Crosshead unclamped, Emergency stop pressed, AC power failure) + +This happens if the rig controller is switched off. + +Solution: turn the rig controller back on (and wait ~2 mins for it to connect to console again) + +### Cannot start the Instron "Console" application + +This has been seen on a previous stress rig control PC. We are unsure of the cause. + +Consider how much time you want to spend on this issue - we can only provide very limited support for Instron's console application, and complex support queries might be best to send to Instron directly. + +It is possible to re-install the software from `\\isis\shares\ISIS_Experiment_Controls\Manuals\Instron_Stress_Rig\MiniTower (Ethernet - Arby)\installer` +- This should only be attempted as a last resort and will take some time to set up, and will need instrument scientist involvement to recommission/recalibrate the rig once done. +- When it asks for a system ID, this is on a sticker on the rig itself, and also stored at `\\isis\shares\ISIS_Experiment_Controls\Manuals\Instron_Stress_Rig\MiniTower (Ethernet - Arby)\ENGIN-X Instron Control Tower System ID.jpg` +- You will need to restore calibrations from `\\isis\shares\ISIS_Experiment_Controls\Manuals\Instron_Stress_Rig\MiniTower (Ethernet - Arby)\ENGIN-X 100kN rig calibration files 2022-09-01` after reinstalling + +### Hydraulics randomly trip + +Cause is currently unknown. + +### Communications sometimes randomly fail + +Cause is currently unknown. \ No newline at end of file diff --git a/doc/specific_iocs/loading_rigs/Instron-stress-rig.md b/doc/specific_iocs/loading_rigs/Instron-stress-rig.md new file mode 100644 index 000000000..652009cc3 --- /dev/null +++ b/doc/specific_iocs/loading_rigs/Instron-stress-rig.md @@ -0,0 +1,87 @@ +# Instron stress rig (GPIB) + +```{note} +This page relates to an older-style GPIB-connected stress rig, which we talk to via a GPIB converter box. For documentation about the ethernet-connected rig (i.e. the 100kN rig on ENGIN-X) see [Instron stress rig (MiniTower)](Instron-stress-rig---MiniTower) +``` + +## Setup + +The Instron stress rig is a National Instruments GPIB device. It requires some special setup to get going: + +- The Ethernet GPIB box requires an NI `488.2` (GPIB) driver. This can be installed from `\\isis\installs\Installs\Applications\LabVIEW\GPIB-ENET bits\GPIB Software`. + - Note, **the version of GPIB MUST be 2.7 (from the share above), later versions WILL NOT WORK WITH THE STRESS RIG!** This is due to some kind of licensing handshake in newer versions of NI `488.2` (GPIB) of which older hardware (such as the stress rig) does not understand. +- Run the installer as an administrator, accept the defaults. It will unzip, then install. It takes a while to install. +- You then need to map the ENET box. Usually this means going into NI-MAX and finding devices - it should come up (otherwise, you may have to find and enter it's IP address manually). +- The driver should now be able to communicate (if not, see troubleshooting section below) + +## Getting the rigs set up + +There are two rigs: 50kN and 100kN. + +To turn on the actuator for the 100kN rig, there are three buttons on the front labelled "O", "I", "II". Press these in order to enable the actuator. + +To turn on the actuator for the 50kN rig, Press and hold the "Hydraulics on" button for at least 10 seconds. You should hear the hydraulic system engage (also, the hydraulic lines will change position slightly as pressurised gas enters the system). Then press actuator "off", "low", "high" (in order) to enable the actuator. + +## Hardware debugging + +### Hydraulics keep tripping off and status is "oil too hot" + +Check if the cooling water circuit for TS1 south side is turned on. If not, the Instron's oil might heat up too much which causes the hydraulics to trip when moving the rig. This happens regardless of whether IOC, LabVIEW, or manual control is used to move the actuator. Additional symptoms are a rig status of "HYD. PUMP SHUTDOWN" and the red status light on the control panel being solidly on (and not being able to clear it). + +If the cooling water is off, there is a circulation pump that can be used to run the rig in low-force mode: ask the scientists. + +### General comms failures / stress rig in invalid state + +This can be triggered by sending commands the stress rig doesn't like, or sending valid commands too quickly. Sometimes the control panel of the stress rig will claim it is simultaneously in two control modes (which is impossible). + +The stress rig can be power cycled using the process below (note: if unsure, check with scientists first, as it may need recalibrating after this process) +- Turn off the stress rig PC (white/beige pc located under the stress rig) +- Turn off the GPIB box (this looks similar to a router, located under the stress rig near to the stress rig PC) +- Wait a few seconds +- Turn on the stress rig PC +- Wait several minutes. The stress rig will go through a self-test phase. You need to wait for this process to be complete. +- If you need to calibrate the rig, the following sequence on the console should be used: "Load channel: setup" -> "cal" -> "cal" -> "auto" -> "go" (if you are unsure about this ask the scientists) +- Turn back on the GPIB box. +- After the GPIB box has been turned on for a short time (e.g. 1 minute) the driver (LabVIEW or IOC) should be ready to connect. +- GPIB box LEDs - PWR should be orange, LNK 10/100 should be green. Other LEDs will be flickering depending on connection. Box might have a slightly dodgy connection (not sure about this, but check it) so ensure the LEDs are as described. + +### Weird comms issues + +The stress rig on ENGIN-X is mapped from both ENGINX and ENGINX_SETUP. **Only one of these computers should talk to the stress rig at a time!** If you get strange comms issues, check that the "other" computer is not also trying to talk at the same time. + +### Hydraulics won't re-enable after panic stop + +A panic stop sends the `C23,0` command to the stress rig. To quote from the Instron manual: + +> If this command is sent with the Off parameter, actuator manifold pressure is turned Off immediately, and the Actuator On button is disabled. + +To re-enable the actuator (hydraulics control), first check that the cause of the panic stop is no longer an issue. Then you may need to send the command `C23,1` to the stress rig via the arbitrary command interface on the stress rig OPI once the rig is back in a safe state. + +If the above command fails, restarting the hardware and then the IOC should boot the stress rig back up with the actuator enabled. + +## Driver + +The stress rig driver uses the following DB files: +- `controls.db` - provides basic PVs such as stopping the rig, getting the status, changing control channels, etc +- `controls_channel.db` - provides PVs that are common to each channel (Position, Stress, Strain), e.g. setpoints, readbacks, step times +- `controls_channel_specific.db` - add on PVs for the above - used to add PVs for some channels that don't exist on others +- `controls_waveform.db` - provides the PVs dealing with the waveform generator +- `logging.db` - provides the PVs to do with logging to a file + +The protocol is defined in [`C:\Instrument\Apps\EPICS\support\instron\master\instronSup`](https://github.com/ISISComputingGroup/EPICS-instron/blob/master/instronSup/devinstron.proto) + +## Gotchas / Unusual things +- Every "write" command (commands starting with C or P) must be preceded by `P909,1` (switch to computer control mode) and `C904,0` (disable watchdog). For convenience there is the `setControlModeCom` function in the protocol file which does these for you. + * If you forget to do this, the rig's hydraulics will trip, requiring a physical button push to reset them +- Every "write" command (commands starting with C or P) must be followed by `P909,0` (switch to front panel control mode) +- We have had issues with PVs dropping off to zero while being read from the stress rig. To solve this, use a `:RAW` record which does a read, and monitor it using `CP` from another record. This fixes the issue of getting zeroes. + * Might be a bug in EPICS and/or the interface between the VISA driver and EPICS. At the moment the above solution seems to be the easiest way to cure the symptoms. +- PVs in the stress rig don't scan by themselves typically, they are triggered from one of two read loops: + * `READ_SLOW` scans at 1 Second interval + * `READ_VAR` reads at a rate that can be varied by the user + * In the `READ_VAR` loop the `.1 Second` option has been removed, this is too fast for the rig to handle +- If something works in LabVIEW but not in EPICS, or vice-versa, NI Input/Output trace (NI Spy) can be very useful to compare the traffic and spot any differences. +- The waveform generator does not like receiving setpoints in quick succession. This can cause a fault with the following symptoms (see also https://github.com/ISISComputingGroup/IBEX/issues/2802): + * The "remote" light on the hardware control panels remains lit + * The hardware control panel crashes (it may display that it's in two control channels simultaneously - this is usually impossible) + * The hydraulics will trip. \ No newline at end of file diff --git a/doc/specific_iocs/magnets/Danfysik.md b/doc/specific_iocs/magnets/Danfysik.md new file mode 100644 index 000000000..433129e4d --- /dev/null +++ b/doc/specific_iocs/magnets/Danfysik.md @@ -0,0 +1,82 @@ +# Danfysik + +A type of large power supply for magnets. + +Current four models supported: 8800, 8000, 8500, 9100 and 9700. The [RIKEN](RIKEN-Power-Supplies) power supplies are also controlled by Danfysiks but these are a special case. Although the 9700 should work with the opi it uses a baud rate of 115200, unlike the 9600 used by other devices. The baud rate macro is not exposed to the GUI and so must be set in the globals.txt, the power supply group are working on getting them switched to 9600 baud. + +All can be in calibrated or uncalibrated mode. Calibration is done within the IOC and enabled via macro. Additionally there is a macro to switch to a local calibration repository instead of the default common one. (see [Calibration Files](/system_components/configurations/Calibration-Files) for more info) + +All can be unipolar or bipolar (controlled by the `POLARITY` macro). + +## Testing + +### All + +Set voltage with `%MYPVPREFIX%DFKPS_0X:SIM:VOLT` +Set current %MYPVPREFIX%DFKPS_01:SIM:RAW + + +### 8000 + +Interlock and power status can be testing by setting status PV: `%MYPVPREFIX%DFKPS_0X:SIM:STATUS`. To switch the power on put a `!` as the first character to turn the interlock on place a `!` as the 10th character. + +### 8800/8500 + +Interlock and power status can be testing by setting status PV: `%MYPVPREFIX%DFKPS_0X:SIM:STATUS`. To switch the power off it must be a binary value with 2 digit set (e.g. 2) for the interlock it must be any value which is greater than 3 digit (e.g. 4). The value 6 would trigger both. + +## Connecting to an 8500 series through serial + +The serial standard (RS232, 422 OR 485) used by a Danfysik 8500 series is determined by the position of a jumper on the control board. This may need to be put in place for a new power supply. + +As the MCR network uses RS422, the jumper position will need to be changed if the PSU was previously controlled by the MCR. + +Only change the **remote line**. The local line changes how the PSU communicates with its front panel. + +The steps to change the serial standard to RS232: + +1. Contact a technician responsible for the PSU. They will open the control panel for you (behind the built-in display). +1. Move the jumper from either `ST10` or `ST11` to position `ST9` (RS232 only). + - The position of the jumper is illustrated in figure 13 of the 8500 series user manual on the manuals share + - Which jumpers to short circuit `(S/C)` or leave as open circuit `(O/C)` are given in table 12. + - Do not change the jumpers listed in table 11. + +Note: There are two 8500 series which required a null modem on the 25-way connector. + +## Value Scaling + +It is worth noting that some Danfysik devices read and write `RAW` values at different scales: given the same `CURRENT` the ratio of `RAW` read:write is 1:10 **except for Danfysik model 8800** where the ratio is 1:1. Configuring the IOC to take this into account is done via two macros `factor_read_i` and `factor_write_i`. The thing to note here is that they are the inverse of each other, i.e. `factor_read_i` = 1 / `factor_write_i` * `r:w ratio`. + +To give an example, for an uncalibrated Danfysik model 8500: +- `factor_write_i` is `1000` +- `factor_read_i` is `0.01` (1/1000 * 10) + +## Auto On/Off + +As the name suggests, this setting in the Danfysik IOC controls whether the device should automatically power itself on and off. If auto on/off is disabled, the device should stay in whatever state it is in unless explicitly instructed otherwise. If auto on/off is enabled, the device: +- Should power off if both: + - The setpoint readback value is within `OFF_TOLERANCE` of 0 + - The readback value is stable (i.e. has held its value for +at least 5 seconds) +- Should power on if: + - The setpoint readback value is greater than 0 + `OFF_TOLERANCE` + +Auto on/off gets automatically enabled when using the "sweep to zero and turn off" functionality. + +Auto on/off is part of the workflow for muon instruments, but it is not desired on neutron instruments using Danfysiks and can in fact turn the power supplies off in unsafe ways if configured badly. Thus it should always be switched off on neutron instruments. Note: The DISABLE_AUTOONOFF macro prevents auto on/off from being turned on at all and hides the GUI elements for setting it on. This macro is on by default, but is turned off by the config upgrade script for the EMU instrument, where this feature is desired. + +## Initialization + +At ioc start, some models of Danfysik need to be explicitly changed into remote mode and/or addressed. The last setpoint and power status must also be resent to the device so that the magnet does not change state as a result of an IOC restart. This is currently being done using autosave. Further details in https://github.com/ISISComputingGroup/IBEX/issues/1209 + +## LOQ/SANS2D Goudsmit magnet + +- This magnet is driven by a 750A power supply, but can only actually take ~600A. The scientists have asked us to apply a limit of 738666 parts per million on this Danfysik (which corresponds to a current of 554A) + * This is true on both LOQ and SANS2D (as of May 2019) +- The magnet is calibrated. Calibration tables are start with `Goudsmit_magnet` and are located in `C:\Instrument\Settings\config\common\magnets` +- The magnet can be run in transverse or solenoid modes. As far as IBEX is concerned, this corresponds to a change of calibration files. + +## Troubleshooting + +### Getting no bytes back from the device +The power supply group have some test software they wrote for communicating with the device. Ask them to try this software and if it doesn't work it's likely a wiring issue. + \ No newline at end of file diff --git a/doc/specific_iocs/magnets/HIFI-Zero-field-controller.md b/doc/specific_iocs/magnets/HIFI-Zero-field-controller.md new file mode 100644 index 000000000..748a75b91 --- /dev/null +++ b/doc/specific_iocs/magnets/HIFI-Zero-field-controller.md @@ -0,0 +1,34 @@ +# Zero field controller (HIFI) + +The zero field controller on HIFI follows a similar design to [the zero field controller used on other instruments such as EMU and MuSR](Zero-field-controller), but is implemented as a separate IOC, `ZFHIFI`, due to a number of differences. + +The main differences are: +- There are 6 field probes, across 2 magnetometer devices (each an instance of a Group3 hall probe, `G3HALLPR`). + * There is HIFI-specific field correction logic used to get a corrected field from these 6 probes +- The zero field system on HIFI uses the shim coils from the HIFI cryomag as it's outputs. These are `CRYOSMS_02`..`CRYOSMS_04`, hosted on `hifi-cryomag` + + +## Expected setup + +The expected setup for this system is: +- `CRYOSMS` IOCs on `HIFI-CRYOMAG` for the shim coils, configured with: + * `"CRYOMAGNET": "No"` (The shim coils should *not* be set up as a cryomagnet) + * `"NON_PERSISTENT_SETTLETIME": 0` (To avoid excessive settle times on ZF writes) + * `"HOLD_TIME": 0` (To avoid excessive settle times on ZF writes) + * `"HOLD_TIME_ZERO": 0` (To avoid excessive settle times on ZF writes) +- `G3HALLPR` IOCs on `NDXHIFI` configured with: + * `"FIELD_SCAN_RATE": "Passive"` (the zero-field system will scan these when relevant) + * `"FLNK0": "IN:HIFI:ZFHIFI_01:MAGNETOMETER:X1:READINGS_UPDATED.PROC CA"` (and similar for `flnk1/2`) +- `ZFHIFI_01` configured with: + * `"PSU_X": f"IN:HIFI-C11:CRYOSMS_02"` (and similar for `PSU_Y`/`PSU_Z`) + * `"MAGNETOMETER_X1": f"IN:HIFI:G3HALLPR_01:0"` + * `"PSU_X_MIN": -15` (and similar for `Y`&`Z`, and `MIN`/`MAX`) + +The flow of control is: +- The `ZFHIFI` ioc will request the magnetometers to take readings by processing `G3HALLPR:...:TRIGGER` +- The `G3HALLPR` ioc will take a reading +- The `G3HALLPR` ioc will process the `READINGS_UPDATED` pv on the `ZFHIFI` ioc +- When all probes are ready, the zero-field ioc will calculate corrected fields +- The zero-field controller will write a new setpoint to the cryosms IOCs +- The cryosms IOCs will ramp the fields on the shim coils using it's internal state-machine +- The zero-field controller will wait for the cryosms to be "within tolerance" of the setpoint and reporting `READY` status. diff --git a/doc/specific_iocs/magnets/HTS-Magnet.jpg b/doc/specific_iocs/magnets/HTS-Magnet.jpg new file mode 100644 index 000000000..f86732dec Binary files /dev/null and b/doc/specific_iocs/magnets/HTS-Magnet.jpg differ diff --git a/doc/specific_iocs/magnets/HTS-Magnet.md b/doc/specific_iocs/magnets/HTS-Magnet.md new file mode 100644 index 000000000..aa77b0dc2 --- /dev/null +++ b/doc/specific_iocs/magnets/HTS-Magnet.md @@ -0,0 +1,27 @@ +# HTS Magnet + +## HTS Magnet System +![20221130_140744](HTS-Magnet.jpg) + + +Consists of an electromagnet powered by two KEPCO PSUs in master/slave configuration with monitoring by a Smartmonitor. + +### KEPCO Power Supplies + +The master KEPCO PSU (the uppermost in the rack) is connected to a MOXA NPort socket via a bespoke cable (_NOT_ standard MOXA cable) with an RJ45 (8P8C) on one end and an RJ11 (6P6C) on the other. This flat grey cable is supplied with the device and is labelled accordingly (believed to be the only one and to date, no working spares exist). The serial/remote/RS232 socket on the KEPCO is labelled "Trigger" and is in the bottom left corner of the rear panel. + +The slave PSU is connected to the master (current output is shared between the two), with both connected to the Smartmonitor. + +### Smart Monitor + +The Smartmonitor is a _READ ONLY_ device which monitors the PSUs and magnet. It shows the safe operating limits, hard limits and the current voltages and temperatures of the connected KEPCO. Its only connection to the control system is via Ethernet on the private instrument network. It has a fixed IP address on the R80 network, which is x.x.38.184. It has not yet been set up for R55 but if so the MAC address is available in the manuals area needed for setting up a new IP reserve. + +Note that you cannot ping the smart monitor, but can telnet into it or view its web server at `` + +The status command response on the Smartmonitor does not match what we have written in the "manual" (which is just an email chain) and adds a timestamp (in the NZST timezone...) before the last status character - [#7397](https://github.com/ISISComputingGroup/IBEX/issues/7397) addresses this issue. + +If the smart monitor can't talk by IP or on it's webserver, but all looks plugged in and otherwise healthy, try power cycling the system (after verifying field is zero) and/or check all ethernet cables are correctly seated. + +### Newport SMC100 Motion Controller + +This system is also used with an [SMC100](../motors/SMC100) single axis stage to position the sample inside the magnet itself. diff --git a/doc/specific_iocs/magnets/MuSR-Steering-Magnets.md b/doc/specific_iocs/magnets/MuSR-Steering-Magnets.md new file mode 100644 index 000000000..5e91eab57 --- /dev/null +++ b/doc/specific_iocs/magnets/MuSR-Steering-Magnets.md @@ -0,0 +1,11 @@ +# MuSR Steering Magnets + +MuSR has two steering magnets, for horizontal and vertical steering of muons. + +An OPI exists for this (`musr_steering.opi`) which takes two Kepco Power supplies as macros for the respective magnet. + +This OPI contains a `STOP` and `START` button as pictured - this sends a PV write to the `IN:NDXMUSR:MUSR_STEERING_OFF` PV. This gets loaded by `INSTETC` on MuSR from `custom_records.db` file in the configurations area. This is stored on the `NDXMUSR` branch. + +![musr_steering_opi](musr_steering_opi.jpg) + +The OPI also offers "saving" and "loading" functionality for the steering magnets and their 'default' values - these point to fields specified in `custom_records.db`. These records effectively trigger a manual autosave for the Kepco current setpoint values and load them back in when the load button is pressed. The OPI shows the current defaults above the buttons used for loading and saving. \ No newline at end of file diff --git a/doc/specific_iocs/magnets/Muon-Active-Compensation.md b/doc/specific_iocs/magnets/Muon-Active-Compensation.md new file mode 100644 index 000000000..87faa43d3 --- /dev/null +++ b/doc/specific_iocs/magnets/Muon-Active-Compensation.md @@ -0,0 +1,35 @@ +# Muon Active Compensation + +```{note} +This is a design document; the functionality is not currently implemented. +``` + +A new opportunity for a small project with a deadline of the 1st of February 2019 was added by the Muon scientists to provide active compensation to the Muon beam-steering magnets on the south side (EMU, MUSR and HIFI). +This page contains the notes from a meeting trying to scope the requirements. This should capture the basics of the requirements, whoever takes the project on should verify that the requirements are complete. + +As muons have a magnetic moment and as the instruments are in close proximity the use of the magnet on one instrument can have an impact on the behaviour of the muons and the flux seen on the other two. + +Typically this is EMU diverting the paths of the other two beams due to location and shielding differences. + +Each instrument needs to be aware of the magnitude and direction (positive/negative and transverse/longitudinal) of their own magnet, and of the other two instruments. + +Each magnet will be considered from the desired value by the scientist/user, and the system should correct the actual value sent. This monitoring needs to occur and be acted upon within a few seconds of each other. + +A set of linear coefficients (to be provided by the scientists, and ideally alterable by them as well) should be enough to provide this information. It should be possible to run the system without the corrections being used (i.e. the coefficients are not actioned). + +The data flow would be as follows: +1. Value requested of A for the steering magnet +1. Value A is passed through the coefficients and altered based on the status of the magnets on the other instruments (this is a continuous loop with a timescale of a couple of seconds), giving value B +1. Value B is sent to the device +1. The value read back from the device should be considered and kept within a tolerance of Value B + +There is a need for an immediate solution, as well as a longer-term solution under IBEX. As such the following changes are needed to the existing LabVIEW system: +1. The control of the septum magnet on HIFI needs to be returned under SECI as well and will be required as part of this work +1. Need to provide an offset field for all steering magnets, this should then be added and subtracted from the set and readback values +1. The offset will be supplied by another piece of code, ideally an IOC ready for later migration to IBEX + +The coefficients should be of a similar layout to the following: + +| Steering Magnet | Set | EMU | MUSR | HIFI | Offset to Apply | +| --- | --- | --- | ---| --- | --- | +| Horizontal | Desired value | Coefficient 1 | 1 (This instrument) | Coefficient 2 | The result of the equation for the offset to apply to the setpoint based on the appropriate input values | \ No newline at end of file diff --git a/doc/specific_iocs/magnets/Muon-NGS-Power-Supply-Firewall.md b/doc/specific_iocs/magnets/Muon-NGS-Power-Supply-Firewall.md new file mode 100644 index 000000000..cc95cb495 --- /dev/null +++ b/doc/specific_iocs/magnets/Muon-NGS-Power-Supply-Firewall.md @@ -0,0 +1,15 @@ +# Muon NGPS Power Supply Firewall + +The three NGS power supplies for the muon front end are controlled via TCPIP but hidden behind a dLink firewall box located in the cabinet to the left. Two of the supplies control SEPTUMA and SEPTUMC, with one (currently the one in the middle) the spare. All the power supplies are accessed on the private network on the same IP port (10001), but the firewall box port forwards so they appear externally under ports 10001,10002,10003 The firewall only allows access from MUONFE to the supplies. The firewall has a web interface for management. Login details for the firewall box are on the sharepoint access page + +From top to bottom the supplies are + +|Supply | Internal IP | gateway | external port| external IP | +|------ | ------------- | -----------| -------------| --------| +|SEPTUMA | 192.168.10.10 | 192.168.10.1 | 10001| firewall IP | +|SPARE | 192.168.10.11 | 192.168.10.1 | 10002| firewall IP | +|SEPTUMC | 192.168.10.12 | 192.168.10.1 | 10003| firewall IP | + +## Swapping a power supply + +If the spare is used, you could either change its IP address to that of the replaced supply (no IOC changes required) or change the port number used by the IOC whichever is easier. Changing IP address is in the advanced network menu and you need to rotate the control to move numbers up and down. diff --git a/images/Muon_Zero-Field_Auto-Feedback_Procedure_Flowchart.png b/doc/specific_iocs/magnets/Muon_Zero-Field_Auto-Feedback_Procedure_Flowchart.png similarity index 100% rename from images/Muon_Zero-Field_Auto-Feedback_Procedure_Flowchart.png rename to doc/specific_iocs/magnets/Muon_Zero-Field_Auto-Feedback_Procedure_Flowchart.png diff --git a/doc/specific_iocs/magnets/NGPS-PSU.md b/doc/specific_iocs/magnets/NGPS-PSU.md new file mode 100644 index 000000000..db9cfb1c8 --- /dev/null +++ b/doc/specific_iocs/magnets/NGPS-PSU.md @@ -0,0 +1,40 @@ +# NGPS Power Supply + +Controls the new PSUs that are going to be used on the Muon Front End. + +## Communications + +### Serial comms + +See comments on https://github.com/ISISComputingGroup/IBEX/issues/3447 for details of what was tried. Short version is that a USB-422/485 adapter plugged into a laptop works but a moxa does not. + +### Ethernet comms + +See details of setup [here](/specific_iocs/magnets/Muon-NGS-Power-Supply-Firewall) + +### Status of the device + +The status of the device is encoded in 32 bits. The status of the device depending on which bits are turned on. See page 30 of the manual. + +The device returns its status as an 8 digit 0-padded hexadecimal number. Each digit is directed into its own `mbbiDirect` PV from which the individual bits can be accessed. + +### Faults + +Currently (2018-07-31) there is a single binary input PV that is '1' if there is a fault and '0' if not. + +A fault corresponds to one of the ones below with the bit corresponding to them in the device's internal status: +- Fault condition (bit #1) +- Mains fault (bit #21) +- Earth Leakage (bit #22) +- Earth Fuse (bit #23 +- Regulation Fault (bit #24) +- DCCT fault (bit #30) + +### Errors + +The device has no error state but will return error codes if it cannot process the command you have given it. The list of error codes can be found on page 49 of the manual. Currently, the IOC captures the last error code as a string and displays it in the OPI. + +If the device is talking slowly i.e. getting lots of command mismatches, only doing writes when on trace then it may need a route deleting. As admin run: +``` +route delete 130.246.52.0 +``` diff --git a/doc/specific_iocs/magnets/RIKEN-Power-Supplies.md b/doc/specific_iocs/magnets/RIKEN-Power-Supplies.md new file mode 100644 index 000000000..b8c5be9fe --- /dev/null +++ b/doc/specific_iocs/magnets/RIKEN-Power-Supplies.md @@ -0,0 +1,65 @@ +# RIKEN Power Supplies + +## Setup + +The serial connections to the RIKEN Front End power supply units (PSUs) were previously made in four daisy chains, which all terminated at a single MOXA NPort (i.e. using four of its serial ports). Since the refurbishment project (concluded December 2022) which replaced all of the PSUs with Danfysik units, the connections are now made in a _radial_ fashion to each individual PSU using five MOXA NPort units placed strategically around the area. + +See the `RIKENFE MOXA NPort Devices and IOCs` spreadsheet in the documentation area mentioned below for PSU <-> COM port <-> IOC mappings. + +## Documentation + +The SharePoint site [`Computing -> ICPdiscussions -> RIKEN FE`](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/RIKEN%20FE) contains documentation relating to the refurbishment project. + +The original documentation on the physical setup is at `\\...\shares\ISIS_Experiment_Controls\Manuals\RIKEN_power_supplies\` with some schematics in `riken psu controls - issue C.ppt`. Most of the information in this area is now out-of-date, particularly that which describes the daisy-chain serial connections to the power supplies. The remote _changeover_ system (configuring RB2 magnet mode to direct beam to specific instruments), has now been removed entirely. + +## IOCs and Macros + +Each of the individual Danfysik power supplies is controlled by an individual [Danfysik](Danfysik) IOC (i.e. a one-to-one mapping). Each IOC is configured accordingly using its macro values, with a different COM port for each (as described above). + +The RB2 PSU is _also_ controlled by a Danfysik IOC (actually two - one for each board, see below). Its protocol is very similar to a model 8500, and so the `DEVTYPE` macro is set to this, with the only difference being the command for setting the current. Rather than writing an entirely separate protocol file, it was decided to create a `protocol_override` macro and set this to a file containing the single command definition. + +The IBEX configuration contains several components, each of which relates to the type of magnet a group of PSUs control e.g. 'Quadrupole', 'Bending', 'Crossfield', etc. (There is also a component for the 11 TPG300 units which control valves and read gauges in the vacuum system). + +## Hardware notes + +### RB2 (**Due to be replaced April 2023(?) with standard Danfysik units.**) + +![Annotated image of RIKEN RB2 PSU Front Panel](RIKEN_RB2_Front_Panel.svg) + +- RB2 is a power supply that can be put into three distinct modes: BEND1 (beam goes one way), BEAM2 (the other way), and SEPTUM (beam splits both ways). RB2, although one physical supply, has two Danfysik-like control boards. The first control board (called "RB2" in our system) supplies current for either BEND1, BEND2, or half of SEPTUM mode. The second control board ("RB2_2" in our system) is *only* used to supply the other half of the current in SEPTUM mode. + +![Annotated image of RIKEN RB2 PSU Control Boards](RIKEN_RB2_Control_Boards.svg) + +- It is the only remaining original PSU from before the refurbishment project. Its replacement(s) were not ready in time to be installed during the long shutdown, and so it has had to be integrated into the new connection topology. Adapters were made to enable each control board to be directly connected to a MOXA NPort, rather than via a daisy chain as previously. Wiring details of these adapters are in the document `RIKEN PSU Communications Cables` on the SharePoint site mentioned above. + +- The PSU itself is inside a restricted area, controlled by swipe access. A Muon scientist, PSU group member, or MCR officer will need to be contacted should access be required. + +## Debugging & Troubleshooting + +For Danfysik PSUs, see separate [Wiki Page](Danfysik) for more specific information. + +### Individual Danfysik PSU won't talk at all + +- Check the PSU is powered on :wink: The control mode should switch to `REMOTE` as soon as a command is received. +- Check the serial cable - it can occasionally become loose. It is attached to the rear of each supply (with null modem) and can be accessed from the rear door of the rack. There is also an RJ45 socket for each PSU in each rack. This is where the MOXA serial lead connects to be patched back to the MOXA NPort unit, and is also worth checking. + +### RB2 PSU won't talk +- Check that the PSU is powered on at the isolator on the EPB wall and at its own local switch under the front control panel. +- Check that remote control mode is selected. This is done by pressing the `COMP` switch in the `CONT.` section of the control panel. +- Double check that the appropriate MOXA port(s) is/are set to RS-422 mode. **This setting needs to be done in the MOXA itself (via the webpage) - IBEX can't do it!** (See spreadsheet mentioned above for port information). + +### Individual PSU won't talk sensibly / I get interleaved replies / I get delayed replies + +This can happen, we are not sure why. The PSU seems to get into a communications mode where it is using a different terminator (I think its terminator when it gets into this state is `\n\r\n\r\n\r`, and no, that isn't a typo...). + +The only (known) way to get out of this state is to power-cycle the power supply at its main switch. + +## Magnet troubleshooting + +This section is from a document by James Lord; the original document on the manuals shared drive + +### RB2 +Sometimes RB2_2 reads back implausible numbers such as 75000A. RB2 may also be wrong by a factor of 2. This is often because the IOC (or all of IBEX) has been restarted and the calibration factors for the power supply have been reset to default (which now corresponds to SEPTUM). +- Run `inst.set_RB2_mode(mode,”CAL”)` with the mode that RB2 is in now. +- Run `inst.set_beamline()` to re-send the setpoints. +Note that this procedure may briefly change the value of RB2, so it is best to pause any runs in progress. The restart of IBEX itself would not have affected the magnets. diff --git a/images/RIKEN RB2 Control Boards.svg b/doc/specific_iocs/magnets/RIKEN_RB2_Control_Boards.svg similarity index 100% rename from images/RIKEN RB2 Control Boards.svg rename to doc/specific_iocs/magnets/RIKEN_RB2_Control_Boards.svg diff --git a/images/RIKEN RB2 Front Panel.svg b/doc/specific_iocs/magnets/RIKEN_RB2_Front_Panel.svg similarity index 100% rename from images/RIKEN RB2 Front Panel.svg rename to doc/specific_iocs/magnets/RIKEN_RB2_Front_Panel.svg diff --git a/doc/specific_iocs/magnets/TDK-Lambda-Genesys.md b/doc/specific_iocs/magnets/TDK-Lambda-Genesys.md new file mode 100644 index 000000000..6a676ef3c --- /dev/null +++ b/doc/specific_iocs/magnets/TDK-Lambda-Genesys.md @@ -0,0 +1,28 @@ +# TDK Lambda Genesys + +The TDK Lambda Genesys is a power supply for conventional (non-superconducting) magnets. It is on use on several muon beamlines as a transverse field magnet, as well as several magnets in the muon front end. + +These power supplies can be daisy-chained in groups of up to 10 power supplies simultaneously; one IOC talks to the entire chain of power supplies. + +## Physical Connection +The connection on the back of the device is an RJ45. However, the device does not communicate through ethernet as this would imply but instead requires a serial adapter to convert it into a 9 pin D type connector that we can then plug the MOXA into. We can make our own adapter using the push-pin adapters (currently in the chalk pits) using the following configuration: +![TDK wiring](tdk_wiring.jpg) + +Note that there is a spare 25 way connector in the office (if you use it please replace it!). I couldn't get it working with a 9 way. + +## Macros + +As well as the typical communication macros such as `BAUD`. `BITS`, `PARITY` etc, there are a number of specialist macros for each power supply in this IOC: + +| Macro | Explanation | +| -- | -- | +| AMPSTOGAUSS | A factor which converts the Amps which are reported by the power supply to Gauss (a measure of magnetic field). This value depends on exactly which magnet the power supply is plugged in to. **Not all magnets use this factor; if it is set to the empty string, the "field" section will be hidden on the OPI. This is the case for all of the magnets on MUONFE (they just control on Amps/Volts for those magnets).** | +| MAX_VOLTAGE | The maximum allowable voltage (in Volts). | +| MAX_CURRENT | The maximum allowable current (in Amps). | +| READ_OFFSET | An offset, in Amps, which is added to the value reported by the power supply before being displayed on the OPI and used to calculate the field. | +| WRITE_OFFSET | An offset, in Amps, to add to any current setpoints before they are sent to the power supply. | + +## Troubleshooting + +If the Genesys fails to communicate and it has recently been replaced this may be due to it's internal address being set incorrectly. This can be changed in the front panel of the device. Devices are often shipped with internal address 6. + diff --git a/doc/specific_iocs/magnets/Transtechnik-Power-Supply.md b/doc/specific_iocs/magnets/Transtechnik-Power-Supply.md new file mode 100644 index 000000000..9c02d48f9 --- /dev/null +++ b/doc/specific_iocs/magnets/Transtechnik-Power-Supply.md @@ -0,0 +1,44 @@ +# Transtechnik Power Supply + +The Transtechnik power supply is a modular system being used on RIKEN for cross-field magnets. Similar models are also used on the ISIS Accelerator. + +Each power supply module provides 100A typically - so for a rack containing a controller and 5 power supply modules, this indicates a 500A supply. Max voltage is 125V. + +## Macros/configuration + +| Macro | Meaning | +| --- | --- | +| `PS_ADDR` | The address of the power supply. Note that addressing "all" power supplies using `000` does *NOT* work for these power supplies. To determine the address of the power supply, navigate through the menus on the front of the physical power supply until you find an option for local mode, then navigate to "PSU address" and check it, then put the power supply back into remote. | +| `VOLT_FULLSCALE` | The maximum voltage that this power supply can deliver, in V. If unknown, ask scientists for advice. | +| `CURR_FULLSCALE` | The maximum current that this power supply can deliver, in A. Each power supply module delivers 100A, so a rack with 5 modules plus a controller is a 500A supply. If unknown, ask scientists for advice. | + +Known settings are: + +| Instrument | Location | `PS_ADDR` | `VOLT_FULLSCALE` | `CURR_FULLSCALE` | Serial comms settings | +| --- | --- | --- | --- | --- | --- | +| Test (for RIKEN) | R6 DCLAB | `005` | `125` | `500` | **RS422** 9600/8/None/1, 9-way cable, no null modem | +| RIKEN XFD1 | Mezzanine | `001` | `125` | `100` | **RS422** 9600/8/None/1, 9-way cable, no null modem | +| RIKEN XFD2 | Mezzanine | `002` | `125` | `100` | **RS422** 9600/8/None/1, 9-way cable, no null modem | +| RIKEN XFD3 | Mezzanine | `003` | `125` | `100` | **RS422** 9600/8/None/1, 9-way cable, no null modem | + + +## Driver logic + +The IBEX driver contains a state machine which waits for the power supply to complete certain actions before proceeding. Physically this corresponds to waiting for an inrush current. + +In particular, the following commands are known to be problematic: +- `N` -> Turns power supply on, no response. Note that a wait of 20s is enforced after this command is sent before setting any currents etc (due to PSU inrush current). In particular, setting new current within 20s of this command may be ignored. +- `RS` -> Note that a wait of 20s is enforced after this command is sent before turning supply on. In particular, turning on the supply within 20s may be ignored. + +The state machine in IBEX enforces appropriate delays between the reset, power, and set current commands. + +## Troubleshooting + +### No interlocks are displayed but power supply can't be turned on + +On this power supply, the interlock lights will go off once the interlock is *physically* cleared, but the power supply will still be in a tripped state until a reset is sent. There is no easy way to display this in IBEX. Solution: send a reset, wait for the enforced delay time, and then try to turn on again. + +### No Communication with device + +- Check address and serial port settings above +- Check that the appropriate port(s) on the MOXA NPort are set to **RS422** mode. This is done via its configuration webpage. (Most likely to occur after replacing a MOXA NPort unit). \ No newline at end of file diff --git a/doc/specific_iocs/magnets/Zero-field-controller-design.md b/doc/specific_iocs/magnets/Zero-field-controller-design.md new file mode 100644 index 000000000..9723179cf --- /dev/null +++ b/doc/specific_iocs/magnets/Zero-field-controller-design.md @@ -0,0 +1,229 @@ +# Zero field controller design + +This page describes our understanding of the requirements for the MUON zero-field controller and serves as documentation for how the system was designed (and therefore, why certain choices were made in the implementation). + +If you only need troubleshooting information rather than design details, see [Zero field controller](Zero-field-controller) + +## Background ## +On a muon instrument, the purpose of the MUON zero-field controller is to maintain a constant (not necessarily zero) magnetic field in the region surrounding the sample. The field should be maintained as external magnetic fields change (i.e. another instrument switches on their magnet). This is achieved by measuring the magnetic field at a fixed sample point. The field should be stable at around the `mG` level and needs updating at a frequency of ~1Hz. + +The magnetic field in the sample region is generated by electromagnets (except on HIFI where they are shims to superconducting magnets). The magnetic field is generated by 3 coils perpendicular to each other, so there are 3 different currents that need controlling. A magnetometer is used to measure the magnetic field (in `mG`) in the sample region; this is called the fixed sensor. The magnetometer provides 3 magnetic field readings; one for the z component (along the beam), one for the y component (vertical and perpendicular to the beam) and x (parallel to the floor). The fixed sensor is close to but not at the sample position. + +On different instruments there is slightly different equipment: + +- EMU, MuSR, ARGUS, CHRONUS + - Magnetometer: flux gate (has lower maximum field than the hall probes) connected to a NI 9125 cDAQ + - Magnets: Conventional magnets attached to Kepco power supplies + - which model of Kepco? + - Manual probe: ? not sure but we are not converting this now +- HIFI + - Magnetometer: Group 3 hall probes connected to a DTM161 + - Magnets: Superconducting, shim values set from 0 field are used as offsets in these magnets + - Manual probe: LS201 hall probe + +## Existing MUON zero-field controller ## +All five muon instruments: ARGUS, CHRONUS, EMU, HIFI and MuSR use a zero-field controller, implemented as a collection of LabVIEW VIs. + +There appear to be 3 variants of the MUON zero-field controller: + 1. EMU and MuSR both use one variant of the MUON zero-field controller. This variant can be found in the folder `C:\LabVIEW Modules\Muon Magnets\Zero Field Controller`. + 1. ARGUS and CHRONUS both use a second variant of the MUON zero-field controller. This variant can be found in the folder `C:\LabVIEW Modules\Instruments\ARGUS\Zero Field Controller`. This variant is similar to the one used by EMU and MuSR. + 1. HIFI has its own unique MUON zero-field controller. This variant is included in the Group 3 Hall probe VI at `C:\LabVIEW Modules\Instruments\HIFI\Group3 Hall probe`. The behaviour of this VI needs to be validated. Note that this includes extra coefficients to allow for the shape of the gradients between the two sets of hall probes. + +### Operation + +The zero-field controller operates in one of two required modes, **_manual_** and **_auto-feedback_** (dead reckoning will not be required). + +#### Manual Mode #### +In Manual mode, the user must adjust the currents manually to achieve the desired magnetic field. +The typical procedure is: + +- Set Auto feedback mode +- Take muon measurements in zero field +- Set Manual mode – this maintains the same currents as have just been used in auto, so the field remains zero (for now) +- Apply a moderate field with the instrument’s main magnet (this could be above the maximum of the flux gate magnetometer, but small enough that the stray field we’ve compensated off would be significant) +- Take muon measurements in this exact applied field. +- Turn off the main magnet. The field will be back to near zero (possible offsets due to the external interfering fields having changed recently, or remnant magnetisation somewhere) +- Re-set Auto Mode. The feedback resumes starting with the currents that were in use in Manual, and smoothly reaches a good zero value. +- Take more zero field measurements. + +#### Auto-Feedback Mode #### +In Auto-Feedback mode, the user specifies the desired field (in `mG`) and a feedback loop continually adjusts the current supplied by the PSUs to achieve and maintain the desired magnetic field. + +## Zero field set up procedure: + +1. Calibrate the system (to relate current to field, undertaken via the `Test Zero Field.vi` within SECI): + - Check for a huge stray field + - Ensure that the absolute maximum of the fields are < 4000 mG 2 seconds after setting a current of 0, if it isn't the magnet is not in range, and there is a huge stray field + - Calculate the magnitude of the field at this point (square root of the sum of the squared fields) + - For each magnet and for a range of currents + - Measure the magnetic field at the fixed sample position + - Plot field against the current + - 21 evenly spaced steps of between the lower current limit and upper current limit, with a 2-second wait before taking the field reading in all three dimensions + - Perform linear regression to work out the calibration coefficients (1 per axis). + - If the result is linear (falls in a tolerance for the RMS - tolerance TBD) these values should be displayed to the user for them to be inputted into the config + - If not linear the user should be warned, warnings still to be defined from the VI + - Check that the noise ratios are acceptable at 0 field in manual mode (3-second delay for settling) + - Set field to 0 + - Put controller in manual mode + - Wait 3 seconds + - 20 readings a second apart + - Calculate the variance for each field + - Calculate the manual RMS value (`square root of the sum of the variance for each field`) + - The system is noisy if the RMS value > 5 + - Check that the noise ratios are acceptable at 0 field in auto feedback mode (6-second delay for settling) + - Set field to 0 + - Put controller in auto feedback mode + - Wait 3 seconds + - 20 readings a second apart + - Calculate the variance for each field + - Calculate the manual RMS value (`square root of the sum of the variance for each field`) + - The system is noisy if the RMS value > 5 + - Plot the noise fields against time + - This is performed once or twice a cycle + - NB Coefficients should not be set automatically; scientists will take these numbers and save them later. +1. Measure offsets (to compensate for the stray field gradients): + - Place a portable probe at the sample position + - Set the field to 0 + - Update the offsets so that the portable probe measures 0. + - The offsets should be printed to the user so that they can update them in the config + - This is performed regularly + +## Zero-Field Controller Feedback Loop ## +The zero-field controller feedback loop uses the following inputs: + * **M** – measured magnetic field (it has three components: longitudinal (L), transverse (T) & vertical (V)) + - in IBEX this should use X, Y, Z, with Z being along the beam, y is vertical transverse and X is horizontal transverse. + * **O** – Offset (to compensate for sensor placement) + * _**C**_ – orientation matrix, derived from engineering drawing, this the direction of the field produced by each coil and because the coils are at right angles has +1, -1 and 0 as its components. + * **S** – setpoints requested + * **P** – magnetic field produced by a coil for a current. Measured in `A/mG` using the calibration procedure. + * p – proportional value (feedback fiddle factor) + * **I** – current on magnets + * **I'** - new current to send + +Quantities in bold are vector quantities. Quantities in italic, bold are matrix quantities. + +The new current to send is calculated as:
+**I'** = (**S** - (**M** - **O**) . _**C**_) . **P** * p + **I** + +The following quantities are: + * **Mc** = (**M** - **O**) . _**C**_ is the corrected field - turns measured field in the basis of the magnetometer into fields produced by the magnet in the basis of the magnets. + * **Mc** - **S** is the difference between the current field and required field + * (**Mc** - **S**) . **P** * p is the change required in the current for this iteration (in current version may need to multiply through by time between samples). + +### Maximum Current + +The process enforces maximum limits on the currents. These are set by the instrument scientist based on + +1. the current likely to be needed to compensate any feasible stray field +1. the current that would hit the voltage limit on the Kepco before its current limit, if the coils have high enough resistance +1. the maximum desired power dissipation of the coils +1. not wanting to exceed the magnetometer full scale. Each axis has its own limit. + +Higher currents are `clamped` to this maximum with the appropriate polarity and ARE then sent to the PSU! But there should be a warning as well. In Auto, the clamped value is used as the basis for calculations next time. + +### Maximum Voltage + +The process sets voltage limits on the PSUs to allow the current setpoints to be achieved. These should be automatically set in the power supply by the zero field controller whenever it is in auto mode. + +### Maximum Field + +When overloaded by a high field, the fluxgate magnetometer can read any random value on all three axes which might look like it's in range and may have the opposite sign to the actual field. Moderate overloads leave the reading close to the limit. The VI attempts to detect fields at or close to the limits and says `sensor overload` if this happens: typically the main magnet coil is still on or there’s a fault with the sensor. + + +**Notes:** + 1. ARGUS appears to use different magnetic field components, labelled LR (left-right), UD (up-down) and FB (forwards-backwards). How do these differ from L, T and V? + - A single set would be good X, Y, Z as described above. + 1. The offset value, O, appears to be a constant. Why is this? + - It is an approximation + 1. Are there limits on the input & output values? If so, what are they? If these limits are breached, what should happen? + - The maximum current and voltage values are set as part of the initialisation file, and values beyond those limits are not sent to the PSU + 1. Is the above expression guaranteed to converge? What if it doesn't? How does the current zero-field controller guard against non-convergence? + - No, we expect to get noise but at the 1 mG level. The signal is declared stable at the 10mG level + +*** + +## Requirements ## +* There is a requirement that Zero field system control needs to be continuous, in the mathematical function sense of not having steps in, when configurations change or control is interrupted.". How is this requirement currently achieved? (It may better to run the zero-field controller on a separate device (e.g. a Raspberry Pi). This approach would eliminate the risk of interruption should IBEX be halted or the control PC re-booted.) What is this requirement in practice? +* Preserving the zero on restart: the usual use-case for this is switching (either way) between a dilution fridge configuration and a cryostat configuration which happens to use the dilution insert as a centre stick. The sample remains in position and cold (1.4K to 10K) throughout and we want to avoid a spike in the applied field which might upset a sensitive magnetic or superconducting state. Being able to preserve zero field through an IBEX crash or computer reboot, and thus avoid having to repeat a time consuming sequence of field/temperature steps to prepare a sample, is also useful assuming everything else in IBEX can be got running. +* There is no enforced maximum rate of change. The calculated new currents are applied immediately – the field change itself may be limited by the inductance of the coils and the Kepcos briefly hitting the voltage limits as a result. Similarly in manual, a new current entered should be applied immediately. In practice in a real experiment there might be a large step on initially zeroing the field for a new sample/cryostat but subsequently going back into Auto will only be a small change. +* The system should be monitored for the currents overloading and this should be fed back to the user + +{#zero_field_implementation} +## Implementation ## +**Suggested operation of magnetometer IOC:** (see [ticket #4838](https://github.com/ISISComputingGroup/IBEX/issues/4838) for more details] +1. On a regular timescale the IOC will read the three axes of the magnetometer. It should then fill in PVs for the **raw field** and **corrected field**: `Mc = (M-O) * C`, as well as its **magnitude**: `Magnitude = sqrt(corrected_X_field^2 + corrected_Y_field^2 + corrected_z_field^2)`. +1. The PVs should go into alarm if the magnetometer is overloaded. + +**Suggested operation of auto-feedback IOC:** (see [ticket #4855](https://github.com/ISISComputingGroup/IBEX/issues/4855) for more details] + +Flowchart of procedure: + +![Flowchart](Muon_Zero-Field_Auto-Feedback_Procedure_Flowchart.png) + +1. The magnetometer values should be checked for overload, if so the feedback is (temporarily) disabled. +1. If in Auto mode and the field is “good” it should then calculate the new currents based on the previous current setpoints, the corrected field and the setpoint. Then check the currents against the limits and if so, clamp the current to the limit value and put the PV into alarm state. It then writes these to the Kepcos (regardless of limit status). +1. If in Auto mode, the "AT_SETPOINT" PV is set to "Yes" if the corrected field is within some value (e.g. 10mG: perhaps set as a macro) of the setpoint, otherwise “No”. If in manual mode, the "AT_SETPOINT" PV is always set to "N/A". +1. In either mode and regardless of any alarms above, it then reads back the actual output current and voltage from the Kepcos and fills more PVs. (Option – put the output current PVs into alarm if the actual current is not close to the setpoint. This would best be done in the Kepco IOC, if not already.) +1. Careful consideration should be given to the case when a PV is unavailable or in alarm. i.e. what should the loop do when data required for a decision isn't available? + +### Notes from instrument scientist (who wrote the original zero-field VI): +* The calibration step may be best implemented as a script that the scientists can then own. [See [ticket #4839](https://github.com/ISISComputingGroup/IBEX/issues/4839)] +* In “manual” offset calibration we need a simple way to vary the offsets and have the already-running zero field IOC take them into account (it will be in Auto Feedback mode). Perhaps this will be numbers we can adjust on an OPI. Typing g.set_pv(‘IN:MUONZF:OFFSET_X’,123.4) each time would be way too tedious. We then might want to save these values for future use perhaps by editing a .ini file, editing IOC macros, or similar (managers only). Or just press a “Save offsets” button? +* The calibrations should not auto-save anything at all without the instrument scientist saying so! It should be possible to calibrate and then run with those calibrated values, without saving them, such that the old values return after a restart of IBEX or a reload of a configuration. (Example – one-off user kit which has a stray field of its own which also needs correcting.) Saving values into one of our standard configurations should need manager level credentials just like any other changes to it. If the calibration is in a script it would have to open the .ini file explicitly and write the values – or perhaps print them to the scripting console for the instrument scientist to copy/paste into the ini file. +* The calibration values are initialised from .ini files, macros, etc but have PVs that the calibration script can read or write, with writes taking effect on the next pass through the feedback loop. The OPI should present these as adjustable values – especially the values of O where up/down buttons would be useful. The maximum currents should be available as read-only PVs. +* Most of the calibration factors are per-instrument and so should be stored somewhere global, or in the “instrument_base” configuration component which presumably loads the IOC. +* The offsets **_O_** (3 values for X,Y,Z) are changed more often and might vary between SE. Is it possible to set default values of **_O_** for the instrument and then any SE configuration which needs a different value can over-ride it? +* On MUSR (and CHRONUS) there are separate sets of calibration factors for the instrument in its longitudinal and transverse orientations. MUSR has a switch, fed into the Field point box, which indicates the orientation. +* Direct PV writes to the Kepco current are allowed only in manual mode and only up to the configured maximum current. +* The timing is important (in Auto mode). The delay from reading the magnetometers to writing to the Kepcos should be kept as steady as possible, as should that from writing to the next magnetometer reading. The loop should also be reasonably fast – aim at around 2 loops per second. The calibration procedure checks this and the value of the feedback factor p is set accordingly. Do not attempt to compensate for varying time delays by adding additional factors into the maths! +* The last-written currents (**_I_**) are kept regardless of mode and updated by the feedback loop or writes to the PVs. +* There will be a “mode” PV that scripts can set to “auto” or “manual”. It may help to have a single PV to summarise the field-is-stable and alarm status (as a string value as well as perhaps going into alarm itself). +* On change from Manual to Auto the value of I is simply picked up and used. On change from Auto to Manual it will simply stop writing values to the Kepco unless the user writes to the current PV. +* On IOC start up I’d suggest try to read the Kepco setpoints (if possible) or the output currents, initialise the last-written variable (I) to this value, and then go into Manual. A subsequent change back to Auto will not cause a significant bump assuming the field was zero before the shutdown and the stray fields have not changed much while the IOC was stopped. +* IOC shutdown should leave the currents on: no special action need be taken. + +## EPICS implementation details + +### High-level implementation + +The Zero field system is usually comprised of 5 different IOCs: +- `ZFCNTRL_01` - the zero field controller. Primarily, it is responsible for taking readings from the magnetometer and figuring out the new fields to write to the power supplies. It also handles most of the top-level user PVs. +- `ZFMAGFLD_01` - the magnetometer IOC. It is responsible for getting new values from the DAQ hardware, offsetting to compensate for magnetometer placement, and converting to the power supplies' coordinate system via a calibration matrix. +- 3x power supply IOCs, e.g. `KEPCO_01` - `KEPCO_03` (on EMU and MuSR). On HiFi, these 3 power supply IOCs are replaced by 3 "shim" coils in the cryomagnet. + +Because of a requirement to keep the delay between magnetometer readings and power supply writes as consistent as possible, it is not possible for the magnetometer IOC to scan independently. Instead, it is explicitly told to take a reading at suitable times from the controller IOC. This is done using a forward link from the controller ioc to `...ZFMAGFLD_01:TAKEDATA`. + +Once the magnetometer has collected readings for all 3 coordinate axes (including the magnitude and whether the magnetometer is overloaded), the magnetometer indicates that new readings are ready by forward linking back to the controller using the PV `...ZFCNTRL_01:INPUTS_UPDATED`. This allows the state machine in the controller IOC to continue to the next step. + +In automatic mode, where the controller is required to write new currents to the power supplies, the following algorithm is implemented: +- If the power supply is in voltage control mode, change it to current control mode. Allow up to 5 seconds (configurable via `ZFCNTRL_01:READ_TIMEOUT`) for this change to happen, and raise an alarm if it doesn't. If the power supply is already in current control mode, skip this step. +- If the power supply is off, send it an "on" command and wait up to 5 seconds (configurable via `ZFCNTRL_01:READ_TIMEOUT`) for it to switch on, and raise an alarm if it doesn't. If the power supply is already on, skip this step. +- Calculate new currents to write, and limit them to the interval `[ZFCNTRL_01:OUTPUT:CURR:SP.DRVL, ZFCNTRL_01:OUTPUT:CURR:SP.DRVH]` +- Send these new currents to the power supply by writing to `$(PSU_X):CURRENT:SP`. +- Allow up to 5 seconds (configurable via `ZFCNTRL_01:READ_TIMEOUT`) for the power supply current setpoint readbacks to get within `ZFCNTRL_01:OUTPUT:PSU_WRITE_TOLERANCE` Amps of the setpoint. If this doesn't happen, raise an alarm. + +### Power supply interface + +The Zero-field controller accepts an arbitrary PV prefix to use as a power supply to drive the X, Y and Z magnet coils. The controller expects whichever IOC it is pointing at to have the following PVs available: + +| PV | Description / caveats | +| --- | --- | +| `CURRENT` | The (measured) current for this power supply. | +| `CURRENT:SP` | The setpoint for the power supply. In automatic mode, new floating-point values will be written to this PV at relatively quick intervals (somewhere between 2-10Hz). The new value should be sent to the hardware as soon as reasonably possible, as long delays can have an impact on the stability of the zero field system. | +| `CURRENT:SP:RBV` | The setpoint readback for the power supply. **This should be updated from hardware as soon as reasonably possible after a write to `CURRENT:SP`** - the zero field control state machine will wait for the setpoint readback to match the setpoint before continuing. For example, it can be updated from the protocol (via record redirection) as part of the setpoint write command. A fast scan is not sufficient to satisfy this requirement, because the delay between writing to `CURRENT:SP` and reading back the new value in `CURRENT:SP:RBV` needs to be consistent. | +| `VOLTAGE` | The (measured) voltage on the power supply. Not required by the state machine, but displayed for information. Therefore it is acceptable to update this relatively slowly. | +| `VOLTAGE:SP:RBV` | The voltage setpoint readback | +| `OUTPUTMODE` | A binary choice PV which returns 0 for voltage control and 1 for current control. If this choice is not applicable for a given PSU, it should always return "1" for current control. | +| `OUTPUTMODE:SP` | "1" is written to this PV by the zero-field system to put the power supply into current control mode - if `OUTPUTMODE` returned 0. This PV is never used if `OUTPUTMODE` is always 1 (see above) | +| `OUTPUTSTATUS` | A binary choice PV which returns 1 if the PSU is switched on, 0 otherwise. If this choice is not applicable, then this PV should always return 1. | +| `OUTPUTSTATUS:SP` | "1" is written to this PV by the zero-field system to turn the supply on - if `OUTPUTSTATUS` returned 0. This PV is never used if `OUTPUTSTATUS` is always 1 (see above) | + +## Performance + +The IBEX and SECI zero-field control systems were compared using an external magnetometer mounted at the sample position. This magnetometer is completely separate from the magnetometer which the zero-field system itself uses, and is plugged into a laptop to monitor the field completely independently (the scientist knows how to set this system up; this is not something we are involved with and is not code we support) + +The following is a plot of the performance of both the IBEX and SECI zero field systems, as tested on EMU in January 2020. Y axis is in Gauss, X axis is in seconds. + +![](emu_zf_compensation_comparison.png) + +When using the external magnetometer (connected to the laptop), be aware that this system has a faulty power supply. If the power supply is not correctly seated in the magnetometer, it can cause the field to appear to jump and/or drift wildly. To fix this, ensure the PSU is seated firmly in the magnetometer and avoid perturbing the magnetometer control box while it is taking data. diff --git a/doc/specific_iocs/magnets/Zero-field-controller.md b/doc/specific_iocs/magnets/Zero-field-controller.md new file mode 100644 index 000000000..c2c5d82a2 --- /dev/null +++ b/doc/specific_iocs/magnets/Zero-field-controller.md @@ -0,0 +1,155 @@ +# Zero field controller + +This page describes some practical troubleshooting information about the muon zero field compensation system. For detailed discussion of the software design, see [Zero field controller design](Zero-field-controller-design). + +## IOC setup + +On EMU, the following IOCs should be running: +- Zero-field compensation power supplies: [KEPCO_01, KEPCO_02, KEPCO_03](../power_supplies/Kepco). These power supplies drive currents around the zero field compensation coils, producing a magnetic field which cancels out stray fields from neighbouring magnets. +- Zero-field magnetometer: [ZFMAGFLD_01](../fluxgates_magnetometers/Zero-Field-Magnetometer-IOC). This measures the magnetic field near the sample position, using probes connected to an NI-DAQ box. +- Zero-field controller: ZFCNTRL_01. This implements the control logic which links the magnetometer and the power supplies. + +The zero-field controller implements most of the control logic using an SNL (sequencer) program. It talks via channel access reads and writes to the magnetometer and power supply IOCs. + +## Controller error states + +### No new magnetometer data + +The zero field controller talks to the magnetometer IOC via channel access. Each time the controller takes a reading, it processes `$(MAGNETOMETER):TAKEDATA` (`$(MAGNETOMETER)` is an IOC macro passed in from the configuration). The zero field controller IOC then expects the `ZFCNTRL_01:INPUTS_UPDATED` pv to be processed when new readings are ready. + +This error state indicates that the `INPUTS_UPDATED` pv was not processed. This may be because: +- The magnetometer IOC is not started or has crashed. +- The magnetometer IOC is taking too long to get new readings (the timeout is defined in `ZFCNTRL_01:STATEMACHINE:READ_TIMEOUT`) + +### Magnetometer overloaded + +The magnetometers have a finite range of fields which they can measure accurately. The magnetometer IOC will attempt to detect this condition and flag an error. + +This error state can be caused by: +- One of the external magnets being switched on - either the longitudinal or transverse fields can overload the magnetometer. Suggested solution: switch off the external magnets +- One of the power supplies used by the zero-field system has been driven too high, and is producing a field which overloads the magnetometer. Suggested solution: + * Put the zero field control system into manual mode + * Manually set the currents in all three zero-field power supplies to zero. + * Check that the zero field no longer reports an overload + * Put controller back into auto mode + * Check that it can stabilize the field to zero. + * Consult with the scientists to define appropriate limits on the Kepcos such that they can no longer overload the magnetometer. + +If neither of the above steps works, consult the scientists. There may be a physical problem with the magnetometer probes or an external field which is too strong to be compensated. The scientists have independent hall probes which can be used to check for large fields at the sample position. + +### Magnetometer data invalid + +This error state is caused when the magnetometer returns data marked with an INVALID alarm to the zero field controller. Check that the magnetometer can talk correctly with the hardware. If it is a DAQMX ioc, the ioc may need restarting after a connection to the hardware is reestablished. See [Zero field magnetometer ioc](../fluxgates_magnetometers/Zero-Field-Magnetometer-IOC) for further troubleshooting details on the magnetometer. + +### Power supply invalid + +One of the readbacks from the power supplies was marked with an INVALID alarm. Check the zero field controller OPI's "advanced" panel to see which power supply is having issues communicating - the readbacks will be highlighted with purple invalid alarms. + +### Power supply on limits + +One of the power supplies has been driven onto a limit, and it's output capped. This may indicate: +- That there is a strong external field present, which the zero-field system is unable to compensate for. Solution: talk with scientists to find out if there is a large external field +- That there is a problem with the magnetometer readbacks. Erroneous magnetometer data can cause the zero field controller to attempt large compensations which would drive the power supplies beyond the limits. +- That one or more of the `AMPS_PER_MG_*` macros has an incorrect sign. This will cause the controller to attempt to compensate for fields in the wrong direction, which will cause out-of-bounds setpoints to be written. If you need to re-determine these macros, see the [calibration](#zero_field_controller_calibration) section below. + +### Power supply write failed + +The zero-field controller writes to the power supply IOCs and then waits for the setpoint readbacks to be within a tolerance of the setpoint. This tolerance is defined by `ZFCNTRL_01:OUTPUT:PSU_WRITE_TOLERANCE` and it will wait with a timeout given by `ZFCNTRL_01:STATEMACHINE:READ_TIMEOUT`. + +This error state means that writes were sent to the power supply but the setpoint readbacks did not get within tolerance. Things to check: +- Is the power supply communicating properly? You can put the zero field controller into manual mode and manually set setpoints +- Is the tolerance too tight? +- If it looks like the readbacks are getting within tolerance but slower than the read timeout, try to update the setpoint readback from hardware immediately after sending a new setpoint in the protocol file. If it's still too slow the read timeout can be increased, but beware that very slow readbacks can affect the stability of the system and only use this as a last resort. +- If the power supply appears to write correctly for a short time, and then one of the Kepcos goes into a non-clearable comms error, verify that `REMOTE_ON_SET=NO` is set in the IOC macros for the three relevant Kepcos. This settings MUST be set for zero-field Kepcos. + +### PSU high limit < low limit + +This state is triggered if the power supply limits have been defined incorrectly in the configuration. Check the configuration macros for sign errors. + +### PSU out of range + +This state is triggered if the power supply setpoint readback value is outside of the power supply limits. + +## Miscellaneous troubleshooting + +### Zero field controller reports "unstable" when the instrument is running at field. + +The "stability" indicator does not truly indicate stability - instead, it indicates that the measured field is within a tolerance of the setpoint. The tolerance is defined in `ZFCNTRL_01:TOLERANCE`. + +This behaviour was specified in the [initial design document - see flowchart](#zero_field_implementation) which was agreed with the muon scientists, and should only be changed by agreement with all scientists using this system. + +### Zero field controller always goes into manual after being restarted (e.g. on config load) + +This behaviour was requested in the [initial design document](Zero-field-controller-design). + +> On IOC start up I’d suggest <...>, and then go into Manual. + +### A 'magnetometer overload' does not always trigger an alarm + +In [ticket 5132](https://github.com/ISISComputingGroup/IBEX/issues/5132), it was requested that the alarms on the magnetometer being overloaded should be suppressed if the zero-field controller is in manual mode. This is because, while the controller is in manual mode, the scientists will intentionally apply large external fields (which overload the magnetometer), but they do not want their users to be worried by alarms. + +In auto mode, the magnetometer being overloaded triggers a MAJOR alarm. + +### The "loop time" is in minor alarm + +The state machine measures the total time it takes for all of it's actions to complete. If this takes too long, a minor alarm is triggered on loop time. This usually suggests that one of the steps in the loop is taking much longer than expected (for example, hitting a timeout when trying to acquire data). + +This is highlighted as an alarm because a very slow loop time can affect the stability of the system and it's response to external fields. However, if the system appears to be operating correctly despite this alarm, then it may be safely ignored. + +### The zero field system is generating large channel access put logs + +Because the zero-field controller IOCs does channel access writes to the power supply IOCs, these writes would usually be logged. However the zero field controller can perform these writes several times per second, causing large put logs to build up. + +To solve this, the power supply setpoint PVs should be placed in the `NOTRAPW` (no trap write) EPICS access security group. This can be done in the configuration using the PV sets mechanism, for example: +``` +IN:EMU:KEPCO_01:CURRENT:SP.ASG NOTRAPW +``` + +In future we may choose to implement a solution which only hides writes initiated by the zero field system, while preserving writes generated by a user. + +### Voltage limits set incorrectly or being reset in auto mode + +The voltage limits are specified via IOC macros. The IOC sends the limits whenever the controller is in auto-feedback mode and the limits read back from the power supply are not within tolerance of the macro value (the tolerance is given in `ZFCNTRL_01:VOLT_LIMIT_TOLERANCE`). + +### Requested current is not being delivered by power supply + +Check that the voltage limits are set high enough. If the readback voltage is equal to the limit, the power supply will not be able to produce any more current and this will impact the stability of the system. Check voltage limit settings in the IOC macros and consult with scientists if you need to raise them. + +IBEX may not have set remote mode correctly or something along those lines (noticed as part of checks on EMU). As a workaround until we have a fix, stop the Kepco X, Y and Z, Zero field controller and magnetometer IOCs and then start up the `zerofield.vi` in `LabView Modules\Muon Magnets\Zero Field Controller` for a few seconds. Then stop the `zerofield.vi` and first start up the Kepcos and then the Zero field controller and magnetometer IOCS. + +### I can set the gas flow to a larger percentage but not to a smaller one + +The needle valve may not be connected correctly, ask the scientists + +{#zero_field_controller_calibration} +## Calibration + +The zero-field controller IOCs has macros for the number of Amps per mG expected of each power supply. These values are obtained by running a calibration routine, which is defined in the [`InstrumentScripts`](https://github.com/ISISNeutronMuon/InstrumentScripts/tree/master/technique/muon) repository and loaded into the instrument scripts. + +The calibration routine will create several plots: +- Plots of Amps against measured mG for each axis. + * These plots should be linear. + * If there is a non-linear section in the graph, you will need to set power supply limits such that the non-linear sections are excluded and then re-run the script. + * The gradient of these lines are the numbers you will need for the `AMPS_PER_MG_*` macros. +- Plots of measured field against time in both auto and manual mode. + * The main thing to look for in these plots is that they contain no particular trend - the values should look like noise. The graphs in auto mode and manual mode should look very similar. + * On EMU a noise level less than 5 mG should be achievable. + * If you see a trend of convergence towards zero, you can try making the feedback factor larger to make the zero field system compensate more aggressively. A value of 0.5 has been found to be a good starting point. Do not make feedback greater than 1 as this is highly likely to create oscillations. + * If you observe oscillations (particularly in auto mode), you can try reducing the feedback factor, however this will make the system slower to respond to real changes. + * If you see a trend of divergence away from zero, it is likely that there is a sign error in one of the Amps per mG numbers or in the magnetometer sensor matrix. + +## cDAQ zero field smoothing + +Averaging can be applied with the use of the `NUM_SAMPLES` macro in the `ZFMAGFLD` IOC. This can help with noise from the magnetometer quickly giving feedback to the zero field controller. + +To apply averaging, set the `NUM_SAMPLES` macro to however many samples you would like to compress. The default is 1, which doesn't do anything and leaves the field values as-is. + +## Debug mode + +The zero-field controller includes a very verbose debug mode, which is off by default. To enable it, use `caput %MYPVPREFIX%ZFCNTRL_01:DEBUG 1`. This will log messages at various points in the state machine, to aid debugging. + +**With debug mode enabled, the zero field controller may log debug messages at the rate of ~100 messages/sec.** This option should not be left on long-term, else the messages will quickly fill available disk space. + +## Autosaving feedback-mode +To autosave the feedback mode so it persists on an IOC restart add `ZFCNTRL_01__SAVEFEEDBACKMODE=YES` to your `globals.txt`. +This is done with a `pass0` autosave which checks for changes every 5 seconds rather than the default when using `autosaveFields` which is 30 seconds. \ No newline at end of file diff --git a/images/emu_zf_compensation_comparison.png b/doc/specific_iocs/magnets/emu_zf_compensation_comparison.png similarity index 100% rename from images/emu_zf_compensation_comparison.png rename to doc/specific_iocs/magnets/emu_zf_compensation_comparison.png diff --git a/doc/specific_iocs/magnets/musr_steering_opi.jpg b/doc/specific_iocs/magnets/musr_steering_opi.jpg new file mode 100644 index 000000000..c2deb5151 Binary files /dev/null and b/doc/specific_iocs/magnets/musr_steering_opi.jpg differ diff --git a/doc/specific_iocs/magnets/tdk_wiring.jpg b/doc/specific_iocs/magnets/tdk_wiring.jpg new file mode 100644 index 000000000..1fef34aad Binary files /dev/null and b/doc/specific_iocs/magnets/tdk_wiring.jpg differ diff --git a/doc/specific_iocs/motor_extensions/ALF-Goniometer-Axes.md b/doc/specific_iocs/motor_extensions/ALF-Goniometer-Axes.md new file mode 100644 index 000000000..b151e27ea --- /dev/null +++ b/doc/specific_iocs/motor_extensions/ALF-Goniometer-Axes.md @@ -0,0 +1,34 @@ +# ALF Goniometer + +The ALF Goniometer is a 6-axis goniometer. There is no specific logic required in controlling it, all that is required is the standard movement across the whole range with appropriate encoder/motor counts, and an ability to home all axes to a known position. + +## Axes Conventions +The numbering of the axes below corresponds to the number of the axes on the [annotated diagram](http://www.facilities.rl.ac.uk/isis/computing/ICPdiscussions/ALF/ALF_Gonio_annotated.png) of the ALF goniometer. + +1. `Z-axis` +1. `theta` (a.k.a. `Rrot`) +1. `Cy` (according to `galil.ini` for LabVIEW control) +1. `Cx` (according to `galil.ini` for LabVIEW control) +1. `Rlower` +1. `Rupper` + +## Axes & Motors +The table below defines how the goniometer axes are connected to motors, and the motor setup data needed: + +Axis Name | `Rupper` | `Rlower` | `Cx` | `Cy` | `theta` | `Z` +------------ | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- +Axis letter (crate) | F | E | D | C | B | A +Units | deg | deg | mm | mm | deg | mm +Acceleration | 4000 | 4000 | 4000 | 4000 | 8192 | 2048 +Deceleration | 4000 | 4000 | 4000 | 4000 | 8192 | 2048 +Speed | 700 | 2000 | 2000 | 2000 | 2048 | 4096 +LabVIEW Motor Steps per Unit | 1358 | 1719 | 640 | 640 | 756 | 641 +Motor Type | Reverse
Active
High | Reverse
Active
High | Reverse
Active
High | Forward
Active
High | Forward
Active
High | Forward
Active
High +Encoder Present | Yes | Yes | Yes | Yes | Yes | Yes +LabVIEW Encoder Steps per Unit | 842 | 1074 | 400 | 400 | 472 | 400 +Encoder Type | Normal Quadrature | Normal Quadrature | Normal Quadrature | Reverse Quadrature | Normal Quadrature | Reverse Quadrature +Home Method | Forward
Limit | Forward
Limit | Forward
Limit | Forward
Limit | None
Available | Reverse
Limit +Home Value | 25.4 | 27.6 | 55.6 | 57.922 | N/A | 0 +Max from Home | 25.4 | 27.6 | 55.6 | 57.5 | N/A | 200 +Min from Home | -25.4 | -27.6 | -55.6 | -57.5 | N/A | 0 +Max Travel | 40 | 80 | 100 | 100 | 360 | 200 \ No newline at end of file diff --git a/doc/specific_iocs/motor_extensions/Axis.md b/doc/specific_iocs/motor_extensions/Axis.md new file mode 100644 index 000000000..a7f1b30ab --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Axis.md @@ -0,0 +1,19 @@ +# Axis + +Axis records add a layer of indirection between the user and the low level motor. This means if the underlying motor needs to change it only need to change in one place, all blocks, motion set points etc use this axis and so don't need to change. + +The record is setup in the configuration directory in the motor directory in a file called axes.cmd (e.g. for galil it would by `galil/axes.cmd` for SM300 it would be `SM300_01/axes.cmd` (NB Galil are not based on the IOC number so you need an extra macro to make sure you only add the axis to the correct galil). + +An example of adding an axis to a galil is: + +``` +$(IFIOC_GALIL_01) dbLoadRecords("$(AXIS)/db/axis.db","P=$(MYPVPREFIX)MOT:,AXIS=MOT:APERTURE,mAXIS=MTR0101") +``` + +More examples can be found motion [set points directory](https://github.com/ISISComputingGroup/EPICS-motionSetPoints/tree/master/settings). + +The parameters needed are: + +* `P` - prefix of the axis +* `AXIS` - name of the axis +* `mAXIS` - underlying motor from the table of motors diff --git a/doc/specific_iocs/motor_extensions/Bump-Strip.md b/doc/specific_iocs/motor_extensions/Bump-Strip.md new file mode 100644 index 000000000..3d5b62d13 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Bump-Strip.md @@ -0,0 +1,18 @@ +# Bump Strips + +Some instruments use bump strips as a safety feature. These are sensors placed on the physical beamline, which "trip" when touched, killing all motion immediately. They can plug into digital input ports on the Galil which then reads `0` for TRIPPED and `1` for NOT TRIPPED. Which port this is is specific to the instrument and you may have to talk to the scientists to find out. + +At the moment, we do not handle the safety aspect of it in software (i.e. disable setting positions on axes), we just provide indicators in the GUI. + +### Setup + +The GUI gets the status from the PV `MOT:BUMP_STOP` which is always loaded by IOC `GALIL_01`. At the moment, we only have one PV for the overall bump strip status of the instrument, i.e. we do not support a signal per controller. + +If no input source is configured, this PV will default to "UNAVAILABLE". The input source can be set via macro by creating a file `\configurations\galil\bumpStop.cmd`. + +This should contain a single line followed by a blank line e.g. +``` +epicsEnvSet "BUMPSTOP_IN" "MOT:DMC03:Galil0Bi4_STATUS" + +``` +In this case this expects the signal to come from digital input port 4 (*Bi04*) on Galil controller 3 (*DMC03*) \ No newline at end of file diff --git a/doc/specific_iocs/motor_extensions/Collision-Detection-Project.md b/doc/specific_iocs/motor_extensions/Collision-Detection-Project.md new file mode 100644 index 000000000..175b0a070 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Collision-Detection-Project.md @@ -0,0 +1,208 @@ +# Collision Detection + +```{note} +The collision avoidance monitor was removed in [this PR](https://github.com/ISISComputingGroup/EPICS-inst_servers/pull/400/files). +``` + +--- + + This project aims to develop a system for detecting collisions within the LARMOR instrument at STFC's ISIS facility, using the IBEX control system. + +## Original Behaviour + +Presently, experiments are executed using Genie Python scripts, which command the various axes of motion, as well as the DAE system and other instrument systems. Care must be taken to ensure that the motion axes are not allowed to collide with each other, or other parts of the instrument. This means that "soft" limits must be placed on each axis of motion, based on the equipment present on the instrument and determined by the instrument scientist. + +There are certain instrument configurations and situations where setting these "soft" limits does not allow the full range of motion required, or where the limits on multiple axes become interdependent. In this case, it would be useful to re-calculate the "soft" limits dynamically, in order to improve equipment safety whilst also giving the scientists greater freedom. + +## Proposed System + +This project aims to produce a system capable of: + + 1. Developing a method for describing the geometry & motion of an instrument + 2. Detecting collisions which have occurred, and stopping motion + 3. Dynamically calculating the range of motion for each axis of motion and updating the "soft" limits for each axis + 4. Detecting a collision that is likely to occur in the future, due to moving multiple axes + +The system must appear transparent to the instrument users, and to this end, should require minimal changes to experiment scripts. + +The system will be prototyped using python, which is already used extensively as part of IBEX. + +The system resides within the [EPICS-inst_servers](https://github.com/ISISComputingGroup/EPICS-inst_servers) repository. Start/stop scripts are in the root, with the code under CollisionDetection. + +## Prerequisites + +This project assumes a development environment with a configured IBEX installation. + +The system has been developed in PyCharm. For development purposes, the environment variables from the standard EPICS `config_env.bat` and from `start_collision_detection_cmd.bat` should be set in the PyCharm run configuration, with the script target set to `main.py`. + +The system interfaces with the IBEX server using EPICS via the Channel Access protocol. Genie python is used for simple setting and getting of PVs, and the python `ca` module is used to monitor changes in PVs. + +For collision detection, the system uses the Open Dynamics Engine, through the python module `pyode`. A wheel for which is located at https://www.lfd.uci.edu/~gohlke/pythonlibs/#ode + +The system also produces a visual rendering of the system, for diagnostic and development purposes. This uses `pygame` and the `pyopengl` bindings, and needs `glut.dll` and `glut32.dll`, which are included in the repository and can be found [here](ftp://ftp.sgi.com/opengl/glut/glut3.html). + +## Related tickets + +- https://github.com/ISISComputingGroup/IBEX/issues/2936: Code refactor and contains significant discussion about the role, scope, and future development of the system as of May 2018. +- https://github.com/ISISComputingGroup/IBEX/issues/3141: Technical debt outstanding following the completion of #2936 +- https://github.com/ISISComputingGroup/IBEX/issues/3126: Required enhancement to allow configuration of the system to work with different instruments without hard coding. +- https://github.com/ISISComputingGroup/IBEX/issues/2040: A list of suggested enhancements which have been moved to this wiki + +## Instrument configurations + +### ZOOM +ZOOM is running the system to detect collisions on its detector and trolley axes (`MTR0501`, `MTR0507`). These move linearly along the same axis. The measurements were provided by Dominic Oram and are shown in the schematic in [ticket 2936](https://github.com/ISISComputingGroup/IBEX/issues/2936). + +The system was deployed here in May 2018 with the intention of reducing the number of times the bump strips are triggered by the two components getting too close. Resetting the bump strips requires a manual reset, so catching it with the system beforehand reduces work for the instrument scientists. Motion control have stated that no damage will result from the system not stopping the motors. + +### SANS2D Tank +The system was reanalysed for the SANS2D tank (https://github.com/ISISComputingGroup/IBEX/issues/4587). The following issues were found that made it unsuitable for an immediate solution: +* By moving two objects towards each other you can get just within the above limits before it comes to a stop, when inside these limits the CAM will halt all motion and not let you move the items apart again. [Ticket](https://github.com/ISISComputingGroup/IBEX/issues/5605) +* When automatically setting limits the CAM is not intelligent enough to realise that an object is currently moving out of the way, this could be avoided by just not using automatically set limits + +## System Overview + +The system comprises four main parts: + +1. Collision detector +2. Limit calculator +3. PV server +4. Graphic visualiser + +Additionally the instrument geometry configuration must be described, and is loaded in from `config.py`. + +### A Note on Dial *vs* User Coordinates +The system uses the *dial* versions of the readback and limit PVs (`DVAL`, `DRBV`, `DHLM`, `DLLM`) to protect the collision detection algorithm from changes in the user coordinate system. See [EPICS Motor Record](https://www3.aps.anl.gov/bcda/synApps/motor/R6-9/motorRecord.html). + +### Collision Detector + +The collision detector is responsible for stopping motion if a collision occurs. The collision detector must be executed frequently, to stop collisions as promptly as possible. The collision detector runs in a separate thread, to allow it to operate independently to the main program, and provides a `CollisionDetector.collisions` list for interfacing with the rest of the program. + +The system uses the [Open Dynamics Engine (ODE)](http://www.ode.org/) to calculate whether any bodies have collided, using the function `collide`. A list of `GeomBox` objects are created, one for each body, each containing an `ode.GeomBox` object. The `ode.Collide` function is used to determine whether each pair of geometries has collided. The `config.ignore` list ensures that only the geometries of interest are analysed, reducing the computational load. + +When a collision is detected, the system sends a `.STOP` message to each motor that is currently moving, using Genie python. + + +### Limit Calculator + +The system dynamically calculates limits by stepping each motor through its range of motion, and checking for collisions, using the `auto_seek_limits` function. This can be approached as a sequential search, where `collide` is evaluated at each step along the range of motion. Each direction is searched first with a coarse step, then with a fine step. + +This works on the assumption that most collisions behave like a [rectangular function](https://en.wikipedia.org/wiki/Rectangular_function), and remain collided for a significant portion of movement. Typically this is true for linear motion, but when objects are inclined or in rotary cases the duration of the rectangle function is short. This means that for any step size, the search is not guaranteed to find collision. + +If the geometries of the computer model are sufficiently larger than the real-world system, the search step can be optimised. + +For a given increase in size `S` applied to each face of the box: +``` +modelled size = actual size + 2S +``` + +Assuming a head on collision and considering only linear movement of the seeking axis, a collision of the real world system occurs once the model has collided by at least `2S`. Furthermore, taking two objects with an actual size of zero, and a modelled size of `2S`, a "head-on" collision is maintained for `4S`. + +![Head on](collision_detection/Oversize.PNG) + +In the case of an inclined collision, the collision will persist for longer as the collision path through the centre of the object increases with angle. + +![Inclined](collision_detection/Inclined.PNG) + +In the case of a glancing collision, whereby the collision of the model does not infer a collision of the real world system, a collision of the model may or may not be detected, but the real-world system will never be at risk. +Therefore any search step of `4S` or less will detect a real-world collision. + +![Grazing](collision_detection/Grazing.PNG) + +For movements which involve rotation however, the search step must be chosen to ensure that no point on the body moves by more than `4S` in any direction. To achieve this, the system calculates the positions of each vertex of the body at each step. The magnitude of the move is calculated, and if greater than `4S`, the search step can be reduced. The magnitude of the move is re-calculated and the step reduced until the step is less than or equal to `4S`. + +Once an appropriate step size has been found, the magnitude of the move need not be calculated further, reducing the computational load. This assumes that the movement of any body can be expressed as a linear function of the seeking axis - each subsequent step in the seeking process produces a movement of magnitude equal to the first step. + +###PV Server + +The system exposes some PVs for controlling operation, and getting feedback for the user through `pv_server.py`. The [pcaspy](https://pcaspy.readthedocs.io/en/latest/tutorial.html) module is used to achieve this, similarly to the BlockServer. + +There is no access security in the EPICS sense. However, writing to read only PVs is ignored by the driver. + +PV Name | Access | Description +:--- | :--- | :--- +`AUTO_LIMIT` | R/W | Automatically send new limits to all motors whenever new dynamic limits are calculated. If 0, the limits from `config.py` are sent instead. Writing to this PV will cause the dynamic limits to be recalculated. +`AUTO_STOP` | R/W | Automatically issue a `.STOP` command to any moving motors, when a collision is detected. If 0, no `.STOP` commands will be sent. Writing to this PV will cause the dynamic limits to be recalculated. +`CALC` | W | Writing any number to this PV will cause the dynamic limits to be recalculated. +`COARSE` | R/W | The initial coarse step used when seeking limits. `OVERSIZE` is also updated so that the relationship `4 * OVERSIZE = COARSE` is maintained. Writing to this PV will cause the dynamic limits to be recalculated. +`COLLIDED` | R | A list of values for each body. Value will be 1 is a collision has been detected on that body. +`FINE` | R/W | The fine step used when seeking limits. Writing to this PV will cause the dynamic limits to be recalculated. +`HI_LIM` | R | A list of the upper dynamic limits for each motor axis. +`LO_LIM` | R | A list of the lower dynamic limits for each motor axis. +`MODE` | R/W | A 3-bit binary number of the operating mode (see `OperatingMode` class): bit 0 = `AUTO_STOP`, bit 1 = `AUTO_LIMIT`, bit 2 = close the program. Writing to this PV will cause the dynamic limits to be recalculated. +`MSG` | R | A human readable message describing the current collision status. This *could* be added to the spangle banner. Typical messages include `No collisions detected.`, `Collision expected in 0.9s - 1.0s`, `Collision on Name1, Name2` +`NAMES` | R | A list of the names read from `config.py` +`OVERSIZE` | R/W | The additional distance added to the faces of each bodies, to make collisions easier to detect. `COARSE` is also updated so that the relationship `4 * OVERSIZE = COARSE` is maintained. Writing to this PV will cause the dynamic limits to be recalculated. +`RAND` | R | A random number for testing that the server is responsive +`SAFE` | R | Value is 0 if any collisions have occurred, otherwise 1. +`TIME` | R | The time taken to calculate the last set of dynamic limits +`TRAV_F` | R | A list of the distance from the current position to the upper limit for each motor axis. +`TRAV_R` | R | A list of the distance from the current position to the lower limit for each motor axis. Should always be a negative number. +`TRAVEL` | R | A list of the distance to the closest dynamic limit for each motor axis. If the axis is at either of its limits, this will be 0. + + +### Graphic Visualiser +The visualiser is started with the main program, and can be found in `render.py`. + +The visualiser runs in its own thread, and iterates every 50 ms to draw each frame. It updates the graphic by monitoring the `.DRBV` PVs of each axis, and through a `RenderParams` object containing the soft limits, the collision status, and the duration of the most recent limit calculation. + +![Screenshot](collision_detection/ScreenShot.png) + +#### Keyboard Controls +The program can be controlled through the visualisation window in the same way as setting the `MODE` PV. + +Key | Function +:---|:--- +1 | `AUTO_STOP` on. +2 | `AUTO_STOP` off. +3 | `AUTO_LIMIT` on. +4 | `AUTO_LIMIT` off. + +The camera can also be re-positioned within the visualisation: + +Key | Function +:--- |:--- +W | Forward +A | Strafe left +S | Backward +D | Strafe right +Q | Down +E | Up +Z | Rotate clockwise +X | Rotate anti-clockwise +Up | Pan up +Down | Pan down +Left | Pan left +Right | Pan right +Space | Reset to initial view + + +### Describing Instrument Geometry + +The geometry of the instrument is described in `config.py`. Generally the content of this file is a set of parameter lists, indexed against either the bodies in the mechanical system, or the motion axes. The various parameters are: + +Parameter | Indexed on | Description +:--- | :--- | :--- +`colors` | Axis | A list of RGB tuples used when drawing the limits and current position of each axis, where `(1, 1, 1)` is white. The colours can be re-used in `geometries`. Red `(1, 0, 0)` ought not be used as bodies are drawn in red if they have collided. +`geometries` | Body | A list of `dict` containing the `name` and `size`, and optionally `color`, and `position` parameters of the body. For moving bodies, the `position` is not required, as it will be overridden whenever the system moves. The `color` defaults to white. +`moves` | Body | A list of functions which take a list of position values, and return a `Transformation` to describe the new position of each mechanical body. Alternatively (and more efficiently) a function which moves everything and yields `Transformation`s at each step, which can then be iterated over in the same way as the list. +`pvs` | Axis | A list of motor PVs for each axis. The order of these PVs is the order of the position values given to `moves`. +`hardlimits` | Axis | The end limits of each axis of motion. Nominally the end of travel, though tighter limits can be imposed. The dynamically calculated limits are always within these values. +`ignore` | N/A | A list of `geometries` index pairs which are not of interest. This is useful for bodies which are mechanically connected - we don't care if the carriage collides with it's slide. As this list can be long for a complicated moving system, this is best generated using a nested `for` loop. +`coarse` | N/A | The initial coarse limit seek step, which can be overridden by PV. +`fine` | N/A | The initial fine limit seek step, which can be overridden by PV (but probably never needs to change from 0.5). +`oversize` | N/A | The initial oversize parameter to apply to the bodies, which can be overridden by PV. The relationship `oversize = coarse / 4` should be maintained. + +## Suggested enhancements + +Originally taken from https://github.com/ISISComputingGroup/IBEX/issues/2040 + +- [ ] Change the `config.ignore` list to a "collisions of interest" list to make it shorter and easier to write out - but the default would be to ignore that body completely if `config.py` was wrong +- [x] Add some diagrams to the "Limit Calculator" section of the Wiki page. +- [ ] Find out if the `TRAVEL` PV will ever be useful to anyone. +- [ ] Make the collision detection thread talk to the visualiser thread for updating the collision status of bodies (making them turn red more promptly). +- [ ] Remove `config.oversize`, and calculate the correct value for oversize in `main.py`. +- [ ] Remove `config.control_pv` as it shouldn't need to change based on the instrument's configuration (see *). +- [ ] * Make the program aware of its current instrument context - load the current PV from the environment variables at startup, and work out the full motor/control PVs appropriately. +- [ ] Load the configuration from an appropriate location based on the instrument configuration (ask the BlockServer) +- [ ] Calculate the oversize of the models based on the stopping distances of the motors in the system diff --git a/doc/specific_iocs/motor_extensions/Creating-soft-motors-to-control-real-motors.md b/doc/specific_iocs/motor_extensions/Creating-soft-motors-to-control-real-motors.md new file mode 100644 index 000000000..ea3e711ac --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Creating-soft-motors-to-control-real-motors.md @@ -0,0 +1,118 @@ +# Motor Coordinate Transformations + +## Overview +This page details how to write soft `motor` records that control two hard `motor` records with a `transform` record converting between the two coordinate systems in between. This is based on experiences writing the Larmor beamstop controller (now defunct) which has two Galil motor axes representing the angle of the arm (`theta`) and a horizontal (`w`) position which is controlled by two soft motors in a Cartesian coordinate system (`x`, `y`). This system is complicated by the fact that the `y` axis in Cartesian space is related to both the `w` and `theta` axes of the motors. + +## Implementation +And example of two soft motors with a non-trivial coordinate transform in between taken from the Larmor beamstop is given below. In this example `$(MTR1)` and `$(MTR2)` are the real `motor` record (defined by the Galil IOC config) and `ARM:X` and `ARM:Y` are soft `motor` records. + +``` +record(motor, "$(P)ARM:X") +{ + field(LOCK, "YES") + field(DESC, "X axis position") + field(SCAN, "Passive") + field(DTYP, "Soft Channel") + field(OUT, "$(P)CONVERTXY.C PP MS") + field(RDBL, "$(P)CONVERTXY.F CP MS") + field(STOO, "$(P)ARM:MOTORS:STOP.A PP MS") + field(DINP, "$(P)ARM:MOTORS:DMOV CP MS") + field(URIP, "Yes") + field(MRES,"0.001") + field(RRES,"1.000") + field(PREC,"3") + field(TWV,"5") + field(RTRY,"0") + field(EGU,"mm") +} + +record(motor,"$(P)ARM:Y") +{ + field(LOCK, "YES") + field(DESC, "Y axis position") + field(SCAN, "Passive") + field(DTYP,"Soft Channel") + field(OUT, "$(P)CONVERTXY.D PP MS") + field(RDBL,"$(P)CONVERTXY.G CP MS") + field(STOO,"$(P)ARM:MOTORS:STOP.B PP MS") + field(DINP,"$(P)ARM:MOTORS:DMOV CP MS") + field(URIP, "Yes") + field(MRES,"0.001") + field(RRES,"1.000") + field(PREC,"3") + field(TWV,"5") + field(RTRY,"0") + field(EGU,"mm") +} + +record(transform, "$(P)CONVERTXY") { + field(DESC, "Transform X then Y") + field(ASG, "READONLY") + + field(INPA, "$(P)$(MTR1) CP MS") + field(INPB, "$(P)$(MTR2).DRBV CP MS") + field(INPE, "$(ARMLEN)") + + field(CLCF, "COS(A)*E + B") + field(CLCG, "SIN(A)*E") + field(CLCH, "ASIN(D/E)") + field(CLCI, "C - SQRT(E**2 - D**2)") + + field(OUTH, "$(P)$(MTR1) PP") + field(OUTI, "$(P)$(MTR2).DVAL PP") +} +``` +There are several fields that need to be considered. The most important fields in the soft `motor` records are the `OUT`, `RDBL`, `STOO`, and `DINP` fields. + - The `RDBL` field reads the current position of the transformed motor positions from the `transform` record + - The `OUT` field writes out the soft motor position to the `transform` record + - The `STOO` field writes out a bit indicating if the user has requested the motors stop, this needs to be transformed to both the underlying motors. + - The `DINP` field reads in a bit indicating if the underlying motors have stopped moving, this needs to read from both the underlying motors. + +The former two fields can be linked directly to a single [`transform`](https://wiki-ext.aps.anl.gov/epics/index.php/RRM_3-14_Transform) record. A `transform` record can handle both the conversion to and from the soft motor's coordinate system. In the above example, the fields `CLCF` and `CLCG` transform the raw `motor` positions to the soft motor positions. `CLCH` and `CLCI` compute the inverse transform from soft motors to the underlying`motors. + +For the latter two fields, two additional records are required if the axes of the soft motors are defined by a combination of the underlying motors. These should output a combination of the states from both underlying motors. For example: +``` +record(transform, "$(P)ARM:MOTORS:STOP") { + + field(CLCC, "A || B") + field(CLCD, "A || B") + + field(OUTC, "$(P)$(MTR1).STOP PP") + field(OUTD, "$(P)$(MTR2).STOP PP") + + field(PINI, "YES") +} + +record(calc, "$(P)ARM:MOTORS:DMOV") { + field(SCAN, "Passive") + field(INPA, "$(P)$(MTR1).DMOV CP") + field(INPB, "$(P)$(MTR2).DMOV CP") + + field(CALC, "A && B") + + # Must include MDEL to avoid a race condition when setting + # either of the soft motors to the position they're already at + field(MDEL, "-1") + field(PINI, "YES") +} +``` + +## Testing using the IocTestFramework +The beamstop uses galil controllers for the underlying motors. Galils do not have an emulator in the IOC test framework. Instead galils have a simulation mode which can be turned on which can be used as a sort of pseudo-emulator. When experimenting with running a galil controller though the IOC test framework I found I needed to be careful of the following things: + + - That the flags enabling simulation mode have been turn on at some point during the galil startup. Specifically the following macros should be set: +``` + epicsEnvSet SIMULATE "1" + epicsEnvSet IFSIM " " + epicsEnvSet IFNOTSIM "#" +``` + - That the autosave feature of the galil device is turned off. If this is not disabled then it is possible for previous device state to interfere with the state of the tests. For example the limits from a previous galil run might be saved which may affect whether the test can successfully move to a given position. + + - That the device is run in dev sim mode, but without any lewis emulator. The example command for the beamstop in the `run_all_tests.bat` is: +``` +call %PYTHON% "%EPICS_KIT_ROOT%\support\IocTestFramework\master\run_tests.py" -pf %MYPVPREFIX% -d xybeamstop -p %EPICS_KIT_ROOT%\ioc\master\GALIL\iocBoot\iocGALIL-IOC-01 +``` + +## Known issues + - For the `DMOV` record the monitor deadband (`MDEL`) field needed to be defined as `-1` to avoid a race condition where the pulse from `1-0-1` would happen so fast that the soft motors would get stuck in the moving state forever. I also found I needed to set a delay on the motors (`DLY`) and a scan of about 1 second to prevent the soft motors missing the stop pulse from one or both motors. + diff --git a/doc/specific_iocs/motor_extensions/Fermi-Chopper-Lifter.md b/doc/specific_iocs/motor_extensions/Fermi-Chopper-Lifter.md new file mode 100644 index 000000000..3e7725260 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Fermi-Chopper-Lifter.md @@ -0,0 +1,9 @@ +# Fermi chopper lifter + +The fermi chopper lifter is a single-axis Galil-controlled system which lifts the fermi chopper in and out of the beam on the EMMA beamline. It runs without an encoder and relies solely on limit switches at each end of its travel. A program running inside the controller moves the lifter between the limits and this is communicated with by a specific OPI. + +A bi-directional interlock exists between the chopper controller (an [SKF MB5150g5](../choppers/SKF-MB4150g5-Disk-Chopper-Controllers), aka G3) and Galil. This uses the inhibit input on the axis amplifier card to prevent motion if the chopper is levitated. Likewise, the chopper is prevented from being levitated if the lifter is in motion. + +There is further documentation in the Motion Control Group's SharePoint site: Beamline Motion -> IDD Motion Control -> Technical Files -> EMMA -> EMMA chopper lifter - Handover.docx + +[Direct link to document](http://www.facilities.rl.ac.uk/isis/Motion/TestDocuments/EMMA%20chopper%20lifter%20-%20Handover.docx) diff --git a/doc/specific_iocs/motor_extensions/IMAT-Lens-Adjustment.md b/doc/specific_iocs/motor_extensions/IMAT-Lens-Adjustment.md new file mode 100644 index 000000000..bf8b5c548 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/IMAT-Lens-Adjustment.md @@ -0,0 +1,13 @@ +# IMAT Lens Adjustment + +The lens adjustment axis on IMAT is on a Newport axis being controlled by a Galil on MTR0104. This axis is peculiar as the infra-red electronics from the axis encoder interferes with the neutron camera. To prevent this a relay has been placed on the power feed to the encoder as shown below: + +![](IMAT_relay.jpg) + +where the blue highlighted LEDs and green highlighted key show the status of the relay. + +The relay can be controlled via the digital output of the Galil. Sending `SB 1` to the controller will cause the power to be cut and `CB 1` will turn the power back on again. This is done in between moves in the axis record using the PREM and POST fields. `CB 1` is sent on `PREM` and `SB 1` on `POST`. This means that the encoder should always be off when the camera is stationary. + +Unfortunately the feed for the encoder also powers the limits meaning that power will also be cut to these when stationary and they will appear as if a limit has been hit. The homing routine does not send a `CB 1` on start and so will not work. Therefore to do a home the relay will have to be manually switched. To do this there are some helper functions in the instrument scripts `IMAT_library.py` called `turn_on_encoder` and `turn_off_encoder`, use `turn_on_encoder` prior to a home and `turn_off_encoder` afterwards + +During a move `IN:IMAT:MOT:DMC01:Galil0Bo0_STATUS` will show `Low` (from `CB 1`) and then return to `High` (from `SB 1`) afterwards. When not moving, the motor will show as both limits engaged as power will have been cut by the relay. \ No newline at end of file diff --git a/doc/specific_iocs/motor_extensions/IMAT_relay.jpg b/doc/specific_iocs/motor_extensions/IMAT_relay.jpg new file mode 100644 index 000000000..834661c2a Binary files /dev/null and b/doc/specific_iocs/motor_extensions/IMAT_relay.jpg differ diff --git a/doc/specific_iocs/motor_extensions/Jaws.md b/doc/specific_iocs/motor_extensions/Jaws.md new file mode 100644 index 000000000..41b411f55 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Jaws.md @@ -0,0 +1,96 @@ +# Jaws + +The jaws are an abstraction layer in the form of a set of records that can be placed onto 4 motors which represent the North, South, East and West blade. They will allow a centre and gap to be set in both horizontal and vertical directions. It is a requirement that if an underlying motor position is set then the gap and centre should adjust to be in sync. + +The jaws are a `.db` record which can be added to most motors. The `.db` is assembled from records for individual jaw blades, and header records which describe the jawset as a whole (e.g. the overall lock status depending on the lock status of each individual blade). They are loaded via a `jaws.cmd` file in the configuration area which is read by the motor IOCs. The exact location is dependent on the type of motor. + +## Standard Jaws + +This is a standard full set of jaws (N,S,E,W blades) that are entirely controlled by one motor IOC. + +### Example + +For a galil they might be in a file in the instrument settings called `\configurations\galil\jaws.cmd`: + +``` +# Define a set of jaws from first 4 motors on first controller (if present) +$(IFIOC_GALIL_01) dbLoadRecords("$(JAWS)/db/jaws.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS1:,mXN=MTR0101,mXS=MTR0102,mXE=MTR0103,mXW=MTR0104") +$(IFIOC_GALIL_01) dbLoadRecords("$(JAWS)/db/jaws.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS2:,mXN=MTR0105,mXS=MTR0106,mXE=MTR0107,mXW=MTR0108") + +# Define a set of jaws from first 4 motors on second controller (if present) +$(IFIOC_GALIL_02) dbLoadRecords("$(JAWS)/db/jaws.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS3:,mXN=MTR0201,mXS=MTR0202,mXE=MTR0203,mXW=MTR0204") +$(IFIOC_GALIL_02) dbLoadRecords("$(JAWS)/db/jaws.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS4:,mXN=MTR0205,mXS=MTR0206,mXE=MTR0207,mXW=MTR0208") +``` + +The motor axes would then be mapped as follows: +![Standard Jaws Setup](standard_jaws.jpg) + + +## Split Jaws + +Jaws can sometimes be set up such that individual blades of the same jaw set are controlled by separate motor controllers, e.g. North and South are controlled through `GALIL_01` and East and West are controlled through `GALIL_02`. There is a separate set of `.db` records that can be loaded by each IOC individually for this case. + +### Example + +A `jaws.cmd` analog to the one above, but where all vertical blades are controlled by motor controller 01, and all horizontal blades are controlled by motor controller 02, would look like this: + +``` +# JAWS1 +$(IFIOC_GALIL_01=#) dbLoadRecords("$(JAWS)/db/split_jaws_header_vertical.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS1:,mXN=MTR0101,mXS=MTR0102,mXE=MTR0201,mXW=MTR0202") +$(IFIOC_GALIL_02=#) dbLoadRecords("$(JAWS)/db/split_jaws_horizontal.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS1:,mXW=MTR0201,mXE=MTR0202") + +# JAWS2 +$(IFIOC_GALIL_01=#) dbLoadRecords("$(JAWS)/db/split_jaws_header_vertical.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS2:,mXN=MTR0103,mXS=MTR0104,mXE=MTR0203,mXW=MTR0204") +$(IFIOC_GALIL_02=#) dbLoadRecords("$(JAWS)/db/split_jaws_horizontal.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS2:,mXW=MTR0203,mXE=MTR0204") + +# JAWS3 +$(IFIOC_GALIL_01=#) dbLoadRecords("$(JAWS)/db/split_jaws_header_vertical.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS3:,mXN=MTR0105,mXS=MTR0106,mXE=MTR0205,mXW=MTR0206") +$(IFIOC_GALIL_02=#) dbLoadRecords("$(JAWS)/db/split_jaws_horizontal.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS3:,mXW=MTR0205,mXE=MTR0206") + +# JAWS4 +$(IFIOC_GALIL_01=#) dbLoadRecords("$(JAWS)/db/split_jaws_header_vertical.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS4:,mXN=MTR0107,mXS=MTR0108,mXE=MTR0207,mXW=MTR0208") +$(IFIOC_GALIL_02=#) dbLoadRecords("$(JAWS)/db/split_jaws_horizontal.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS4:,mXW=MTR0207,mXE=MTR0208") +``` + +The motor axes would then be mapped as follows: +![Split Jaws Setup](split_jaws.jpg) + +#### Notes: +- `split_jaws_header_vertical.db` and `split_jaws_horizontal.db` each expect the other one to be loaded and the jaws will not work correctly if this is not the case. +- We currently do not support one direction (e.g. vertical) to be split across separate controllers, e.g. J1 North on `MTR0101` and J1 South on `MTR0201` + +## Vertical Jaws + +Some Jaw sets consist of only vertical jaws (i.e. east and west blades do not physically exist). A separate `.db` exists for this type of device which contains header and individual blade records assuming vertical blades only. + +### Example + +A `jaws.cmd` for one jaw set only consisting of vertical blades: + +``` +$(IFIOC_GALIL_01) dbLoadRecords("$(JAWS)/db/jaws_vertical.db","P=$(MYPVPREFIX)MOT:,JAWS=JAWS1:,mXN=MTR0101,mXS=MTR0102") +``` + +Note: We do not currently support horizontal only jaws. + +## Alias Jaws + +In some cases the jaws logic is performed by the controller itself. Noticeably for the [Beckhoff](../motors/Beckhoff). In this case there will be normal motor axes of the form `MOT:MTR0X0X` that describe the centres and gaps. The alias jaws `db` and `cmd` will load aliases for these so they look like conventional jaws. + +## Archive deadbands + +On jaw sets which have fluctuating positions, it was seen that the databases were getting filled up with updates from the jaws. To stop this happening, the archive deadband (`ADEL`) field is copied from the underlying motor to the jaw. + +For the jaws positions that depend on more than one jaw blade (the centres and gaps), it was decided that the archive deadband should be taken from one of the jaws only. They follow this convention: + * **Vertical** (`VCENT`, `VGAP`) positions take `ADEL` from the **North** jaw motor + * **Horizontal** (`HCENT`, `HGAP`) positions take `ADEL` from the **East** jaw motor + +## Instrument-specific Jaws information + +```{toctree} +:glob: +:titlesonly: +:maxdepth: 1 + +jaws/* +``` \ No newline at end of file diff --git a/doc/specific_iocs/motor_extensions/MARI-Sample-Changer.md b/doc/specific_iocs/motor_extensions/MARI-Sample-Changer.md new file mode 100644 index 000000000..9ab196675 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/MARI-Sample-Changer.md @@ -0,0 +1,18 @@ +# MARI Sample Changer + +MARI has two different sample changers. They are very similar in that they are both run by a Mclennan with the same settings for motor/encoder resolution. They are mounted on the end of sample sticks and each have 4 samples at 90 degrees from each other that can be placed into the beam. The sample holder is attached to both by screwing into the end of the sample stick. However, as this holder is rotating and is on this screw if it gets stuck during a rotation it will either tighten itself or loosen itself and fall off. Thus **the sample changer must always be rotated in the same direction to stop it falling off**, including during homing. If you are unsure which direction will result in it falling off you can start the device moving and then hold on to the sample holder and see if it starts tightening or loosening (you always want it tightening). + +Unfortunately, although the sample changers are very similar they are mechanically different in that one is directly driven by a universal joint (UJ) and the other is driven by a belt. Due to these mechanics the sample changers run in different directions (e.g. a positive move on the Mclennan causes a clockwise rotation in one sample changer and anti-clockwise in the other). For both the belt and the universal joint the correct direction is **clockwise from the top** (the top being the side where the home datum is). See the videos below. + +Confusingly there was an arrow on the bottom of each sample changer indicating which way it turns that was incorrect. The IS has corrected this arrow in blue marker pen. For the Mclennan this means that the belt must be driven in the positive direction and the UJ in the negative direction. The positions on the cans were originally incorrectly labelled, they have now been correctly labelled by the instrument scientist in pen. + +There is currently an instrument script on MARI called `changer.py` which moves everything in the correct direction, including during homing, and selects the correct samples. Due to homing directions being different for both they can currently only be homed through that script, not the GUI. These issues will be corrected in https://github.com/ISISComputingGroup/IBEX/issues/5729. + +Finally, because of inaccuracies between converting the motor resolution in the Mclennan to one in IBEX the script does not move 90 degrees for each sample and instead has a small correction. To experimentally measure this correction there is a `test_sample_changer.py` script in C:\Scripts on MARI which will move the cans a number of full revolutions and ask the user to confirm they are still correctly aligned. + +### Correctly moving universal joint sample changer +![](mari_sample_changer_joint.gif) + +### Correctly moving belt sample changer +![](mari_sample_changer_belt.gif) + diff --git a/doc/specific_iocs/motor_extensions/MERLIN,-LET-and-WISH-Oscillating-radial-collimators.md b/doc/specific_iocs/motor_extensions/MERLIN,-LET-and-WISH-Oscillating-radial-collimators.md new file mode 100644 index 000000000..c4d2e4ecd --- /dev/null +++ b/doc/specific_iocs/motor_extensions/MERLIN,-LET-and-WISH-Oscillating-radial-collimators.md @@ -0,0 +1,189 @@ +# MERLIN, LET and WISH Oscillating Radial Collimators + +The oscillating radial collimator on MERLIN is similar but not quite the same as the one on LET. + +## Starting it + +Occasionally the oscillating collimator stops oscillating, you can tell because the `current angle` on the OPI will not be changing. + +To restart this is sometimes tricky try the following: + +1. On the OPI +1. Click Stop +1. Alter the `swept angle` and `operating frequency` to a **valid** and **different** values so it will resend the setting + 1. The valid values for the swept angle as currently 0-2 (you can see by right click on the input text box and `Show pv info`) + 1. The valid values of operating frequency are 0-0.5 +1. Set the swept angle and operating frequency back to the required settings +1. Click start +1. The `Mode` should now read `Homing` then `Oscillating` and the current angle should start changing. + - this was fairly quick (less than 1 minute) + +## Hardware quirks: +- The thread will die if zero is sent for distance and velocity. The driver must ensure that distance and velocity are not sent at IOC start. +- The oscillation will stop if the galil is power cycled. To restart the oscillation, the galil driver will need to be restarted to re-upload the oscillation code, and then the collimator restarted as usual via it's OPI. Once the collimator is started via to OPI it will home and then switch to oscillating. + +## Galil Code + +The oscillations are managed by the galil code different on [LET](https://github.com/ISISComputingGroup/EPICS-galil/blob/master/GalilSup/Db/galil_Oscillating_Collimator.gmc) and [MERLIN](https://github.com/ISISComputingGroup/EPICS-galil/blob/master/GalilSup/Db/galil_Oscillating_Collimator_Merlin.gmc). This section is an rough explanation of the code. + +``` +'************MERLIN Collimator********* +'*********************************** +'DETERMINE INITIAL POSITION +'*********************************** +``` +Comments + +``` +#HOMER +'*******STEP 1 - Retract Motor****** +mode=0 +count=0 +MT~a=2 +DC~a=1024;AC~a=1024;SH~a +``` + +For homing set up some variables: + +- mode (what galil is doing) +- count (how many oscillations it has performed) + +Then set the acceleration and deceleration and switch motor on. + +``` +JP #HOME, @IN[6]=1 +SP~a=250;PR~a=2600 +mode=1 +BG~a +#INI +JP #INI, @IN[6]<>1 +WT 100 +ST~a +AM~a +``` + +If at home switch then move off it otherwise go directly to `HOME` step. + +``` +#HOME +'*******STEP 2 - Find Gigital Home******* +mode=1 +AC~a=67107840;DC~a=1024;SP~a=250;PR~a=-2600 +SH~a;BG~a +#HEO +JP #HEO, @IN[6]=1 +ST~a +AM~a +WT100 +``` +Move in negative direction and stop as soon as we hit the home switch. + +``` +AC~a=1024;DC~a= 67107840;SP~a= 20;PR~a=2600 +SH~a;BG~a +#HR +JP #HR, @IN[6]<>1 +ST~a +AM~a +WT1000 +``` +Move off the home switch and stop as soon as it disengages. + +``` +DP~a=-200 +PA~a=0 +SH~a +SP~a=1024 +BG~a +AM~a +WT1000 +``` +Set the position of the home switch as -200 and move to 1024. + +``` +#OSCILSU +'****Step 3 - Oscillation Setup******* +mode=2 +count=0 +AC~a=accel;DC~a=accel;SP~a=vel +SH~a +PA~a=-dist/2 +MC~a +``` +Move to half the oscillation distance in the negative direction. See [below](#oscillating_collimator_distance_velocity_calculation) for how distance is calculated. + +``` +#OSCIL +'****Step 4 - Oscillations************ +time1=TIME +SP~a=vel +SH~a;PA~a=dist/2;BG~a; +#CHECK1 +JP #CHECK1, @IN[6]=0 +check=_RP~a +AM~a +SH~a;PA~a=-dist/2;BG~a;AM~a +count=count+1 +time=TIME-time1 +'JP #HOMER, check>125 +JP#OSCIL +EN +``` +Move to positive and then negative positions of oscillation, recording time for total movement. There is a disabled check that we clear the limit switch. Repeat this motion forever. See [below](#oscillating_collimator_distance_velocity_calculation) for how distance is calculated. + +The code on LET is the same except that: + +1. It uses a different input for the home switch, +1. It performs jogs instead of limited moves +1. The home switch is defined to be at 0 (not -200) +1. It does not switch on the motor before each move +1. After the home and before the setup oscillation it does not move the motor away from the home switch position + +{#oscillating_collimator_distance_velocity_calculation} +## Distance and Velocity Calculation + +The distance and velocity the ORC is asked to move is calculated based on the provided swept angle and frequency. This calculation is done in the db inside `motorExtensions`. The purpose of these calculations is to ensure that twice the swept angle is travelled with a constant velocity. First the time taken for half a cycle, T, is calculated: +``` +T = 1/(2*freq) +``` +Then swept distance in steps, D, is calculated: +``` +D = 2*Radius*(Steps/mm)*tan(swept_angle) +``` +The required velocity, V, is then calculated based on the equation: +``` +(2/A)*(V^2) - V*T + D = 0 +``` +The solution to which is calculated as: +``` +V = A * (T - sqrt(T^2 - 8*(D/A))) / 4 +``` +which is sent to the controller as `vel`. From this velocity the distance required to get to this velocity is calculated: +``` +D_v = V^2 / (2*A) +``` +And so the total `dist` sent to the control is: +``` +dist = 2*D_v + D +``` + +The explanation for these equations can be seen in the following diagram (courtesy of Ben Withers) ![ORC maths explanation](ORC.png) + +### Moving indicators + +The `MOT:OSCCOL:MOVING` PV (i.e. whether the collimator is moving or not) is calculated differently for MERLIN/LET and WISH. + +Merlin and LET both are fitted with a laser which is plugged into the galil's digital input. We can deduce the movement of the collimator from whether this PV value (`$(P)MOT:DMC01:Galil0Bi5_STATUS`) is fluctuating or not. +The Db scans this laser PV every .1 second taking 100 samples, so it takes 10 seconds to calculate collimator movement. For example, if the laser PV has changed value within the last 10 seconds then it is `Moving`, and otherwise reports `Not moving`. +Due to the way movement is calculated, when the laser pv is in alarm, the collimator will report `Moving` but in an **invalid state**. + +WISH is not fitted with this laser, and so uses the already previously existing logic which checks that (largest motor position in last 10s) != (smallest motor position in last 10s). + +### WISH specifics + +The WISH collimator is slightly unusual in the way that it: +1. Does not have timing information in the Galil code, so frequency can not be easily read back. To get around this in the Db it is aliased to the setpoint. +1. Every x amount of oscillations it rotates 360 degrees - this is referred to as a "maintenance rotation". This is not required on LET/MERLIN as they use different bearing types +1. It calculates the required swept angle using the gearbox ratio etc. rather than the tan of the angle as defined above. The velocity calculations are done in the same way. + +There is some logic in the OPI that will show the fields for the maintenance rotation such as how long until the next one and what the set value is before rotating. This is enabled on load of the wish Db file. \ No newline at end of file diff --git a/doc/specific_iocs/motor_extensions/Motion-Set-points.md b/doc/specific_iocs/motor_extensions/Motion-Set-points.md new file mode 100644 index 000000000..96f42cbc7 --- /dev/null +++ b/doc/specific_iocs/motor_extensions/Motion-Set-points.md @@ -0,0 +1,82 @@ +# Motion Set Points + +Motion set points allow you to label set positions for any number of axis, though currently db files are only created for single, double or 10 axes. Other numbers of axis can also be easily created by following the example of these. The code for this is in support in the directory [motionSetPoints](https://github.com/ISISComputingGroup/EPICS-motionSetPoints). The configuration for a motion set point is in a number of parts: + +1. First, you must set up axes on the motors that you want to configure with the set points +1. St file called `motionsetpoints.cmd` which sets up the db file which is stored in the configuration under the motor name: + - galil is `Settings\config\\configurations\galil` + - Mclennan is `Settings\config\\configurations\mcleanan` + - SM300 is `Settings\config\\configurations\` +1. The positions which are referenced from the st file. are stored in `Settings\config\\configurations\motionSetPoints` + +The `motionsetpoints.cmd` contains the following lines: + +1. **Point at motion setpoint config:** `epicsEnvSet "LOOKUPFILE" "$(ICPCONFIGROOT)/motionSetPoints/"` +1. **Configure setpoints:** `motionSetPointsConfigure("LOOKUPFILE","LOOKUPFILE", N)` (where N is the number of axes) +1. **Load Motion Setpoint Records:** + * *For 1D setpoints* `dbLoadRecords("$(MOTIONSETPOINTS)/db/motionSetPointsSingleAxis.db","P=,NAME0=,AXIS0=,TOL=,LOOKUP=LOOKUPFILE")` + * *For 2D setpoints* `dbLoadRecords("$(MOTIONSETPOINTS)/db/motionSetPointsDoubleAxis.db","P=,NAME0=,AXIS0=,NAME1=,AXIS1=,TOL=,LOOKUP=LOOKUPFILE")` + * *For 10D setpoints* `dbLoadRecords("$(MOTIONSETPOINTS)/db/motionSetPoints10Axis.db","P=,NAME0=,AXIS0=,NAME1=,AXIS1=,NAME2=,AXIS2=,NAME3=,AXIS3=,NAME4=,AXIS4=,NAME5=,AXIS5=,NAME6=,AXIS6=,NAME7=,AXIS7=,NAME8=,AXIS8=,NAME9=,AXIS9=,TOL=,LOOKUP=LOOKUPFILE")` +1. **Load _In Position_ records:** This is a `dbLoadRecordLoop` instruction (one for each `dbLoadRecords` above), which loads an extra `db` file for one setpoint per iteration, which contains a record for indicating whether the motor is at this particular setpoint. The line should look like `dbLoadRecordsLoop("$(MOTIONSETPOINTS)/db/inPos.db","P=,NAME0=,AXIS0=,TOL=,LOOKUP=LOOKUPFILE", "NUMPOS", 0, 2)` (where the 2 at the end is how many different sample positions there are) +1. **A blank line at the end** + +Where: +* `X` - enumeration of lookup files, e.g. 1, 2 +* `motion setpoint file` - the lookup motion setpoint file +* `motion set point prefix` - the prefix you want to create for the motion setpoint, e.g. `$(MYPVPREFIX)LKUP:MON3:`, `$(MYPVPREFIX)LKUP:SAMPLE:`, `$(MYPVPREFIX)LKUP:ANALYSER:` (ending in a colon) +* `tolerance` - tolerance with which the position has to comply with the positions in the lookup file +* `axis0` - the axis to use for the first/only motor e.g. `$(MYPVPREFIX)MOT:SAMPLE:LIN` +* `name0` - the name of axis 0, e.g. "linear" (defaults to `axis0`) +* `axis1` - the axis to use for the second motor e.g. `$(MYPVPREFIX)MOT:SAMPLE:ROT` +* `name1` - the name of axis 1, e.g. "rotational" (defaults to `axis1`) +etc. + +For examples see Larmor, Demo or SANDALS. + +The lookup motion setpoint file has the following format: + + # Comment, lines starting with hashes are comments +