Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,4 @@ book.pdf
*~
build
venv
.DS_Store
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ If you'd just like to learn about how to use the FCC Software,

```{eval-rst}
.. toctree::
:numbered:
:numbered: 4
:maxdepth: 2
:includehidden:
:caption: Contents:
Expand All @@ -49,14 +49,13 @@ If you'd just like to learn about how to use the FCC Software,
CONTRIBUTING.md

.. toctree::
:numbered:
:maxdepth: 2
:includehidden:
:caption: External links:

FCC Software <https://fccsw.web.cern.ch/>
Key4hep <https://key4hep.web.cern.ch>
FCC-ee Detector Full Sim <https://fcc-ee-detector-full-sim.docs.cern.ch/>
FCCAnalyses <https://hep-fcc.github.io/FCCAnalyses/>
Analysis essentials <https://hsf-training.github.io/analysis-essentials/>
HSF Analysis essentials <https://hsf-training.github.io/analysis-essentials/>
FCC Software glossary <https://hep-fcc.github.io/glossary>
```
20 changes: 14 additions & 6 deletions distributed-computing/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@
# Distributed computing

These pages provide --- and dissect --- examples of workflows to be run on distributed resources with the [DIRAC Interware system][dirac]
through [iLCDirac][ilcdirac], the extension developed by the Linear Collider community and used also by CALICE.
>
> Original author: Gerardo Ganis
>

The procedure to be enabled to use the FCC resources through DIRAC is first described. Additional information about the use of
iLCDirac can be found in the [CLIC][wikiclic] and [ILC][wikiilc] dedicated Wiki pages.
These pages provide &mdash; and dissect &mdash; examples of workflows to be run
on distributed resources with the [DIRAC Interware system][dirac] through
[iLCDirac][ilcdirac], the extension developed by the Linear Collider community
and used also by [CALICE][calice].

The procedure how to use the FCC resources through DIRAC is described.
Additional information about the use of iLCDirac can be found in the
dedicated [iLCDirac Documentation][ilcdirac-docs].

Unless specified, in the rest of this section the word `DIRAC` refers to the `iLCDirac` extension introduced above.

[dirac]: https://dirac.readthedocs.io/en/latest/
[ilcdirac]: https://iopscience.iop.org/article/10.1088/1742-6596/513/3/032077/meta
[wikiclic]: https://twiki.cern.ch/twiki/bin/view/CLIC/DiracForUsers
[wikiilc]: https://flcwiki.desy.de/ILCDirac
[calice]: https://twiki.cern.ch/twiki/bin/view/CALICE/
[ilcdirac-docs]: https://ilcdirac-doc.web.cern.ch/index.html

```{eval-rst}
.. toctree::
:maxdepth: 2
:caption: Contents:

RegisteringToFccVO.md
Expand Down
50 changes: 30 additions & 20 deletions distributed-computing/RegisteringToFccVO.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
# Getting started with FCC distributed computing

>
> Original author: Gerardo Ganis
>

For more detailed documentation how user's can use the distributed resources
managed by iLCDirac, refer to the iLCDirac's
[User Guide](https://ilcdirac-doc.web.cern.ch/DOC/Files/UserGuide/index.html).

## Registering to the FCC VO

The [standard Grid VO registration procedure][signup]
should be followed to be enable to use the resources connected with the FCC VO.
The [standard Grid VO registration procedure][signup] should be followed to be
able to use the resources connected with the FCC Virtual Organization (VO).

:::{admonition} Note
:class: callout

You need to use a browser where you have installed your certificate and the
[CERN CA][cernca] certificates. Firefox usually works fine, Google Chrome
usually does not work. Safari might also work.
As for the VO registration, you need to use a browser where you can import your
GRID certificate together with the [CERN CA][cernca] certificates. Firefox
usually works fine, Google Chrome usually does not work. Safari might also work.
:::

[signup]: https://fcc-auth.cern.ch/dashboard#!/newClient

## Enabling DIRAC

DIRAC is available on CernVM-FS. To enable the relevant applications and
scripts, the following setup script needs to be sourced first
scripts, the following setup script needs first to be sourced

```bash
source /cvmfs/clicdp.cern.ch/DIRAC/bashrc
Expand All @@ -29,7 +35,9 @@ To submit jobs through DIRAC a proxy needs to be created and uploaded:
```bash
dirac-proxy-init -g fcc_user
```

A successful creation looks like this:

```
Generating proxy...
Enter Certificate password:
Expand All @@ -51,11 +59,15 @@ Proxies uploaded:
DN | Group | Until (GMT)
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ganis/CN=393971/CN=Gerardo Ganis | | 2022/05/13 12:12
```

The last section shows the valid proxies upload to the DIRAC system. It can also be checked with

```bash
dirac-proxy-info -m
```

with output similar to

```
subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ganis/CN=393971/CN=Gerardo Ganis/CN=2178341058/CN=3000266373
issuer : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ganis/CN=393971/CN=Gerardo Ganis/CN=2178341058
Expand All @@ -78,12 +90,14 @@ If everything worked fine, your proxy should be mapped to the `fcc001` user. Thi
export EOS_MGM_URL=root://eospublic.cern.ch
XrdSecPROTOCOL=gsi,unix eos whoami
```

the result should look similar to this:

```
Virtual Identity: uid=140035 (99,140035) gid=2855 (99,2855) [authz:gsi] host=lxplus743.cern.ch domain=cern.ch geo-location=0513
```

At CERN the uid of `fcc001` is 140035.
At CERN the uid of `fcc001` is `140035`.

## Copying, browsing, accessing files

Expand All @@ -107,7 +121,7 @@ $ ls -lt /eos/experiment/fcc/prod/fcc/user/g/ganis
total 9545
-rw-r--r--. 1 fcc001 fcc-cg 9768981 Nov 3 2021 edm4hep_test_output.root
```
Files can be replicated to another STorage Element, e.g. CNAF-DISK, using
Files can be replicated to another Storage Element, e.g. CNAF-DISK, using
`dirac-dms-replicate-lfn`:
```
$ dirac-dms-replicate-lfn /fcc/user/g/ganis/edm4hep_test_output.root CNAF-DISK
Expand All @@ -134,17 +148,13 @@ root [1]

## The web portal

The [DIRAC web portal][diracweb] is available to check the status of things. It shows all the jobs submitted and the
files registered.
The [DIRAC web portal][diracweb] is available to check the status of things. It
shows all the jobs submitted and the files registered. An example screenshot
is shown below.

![ILCDirac Web](images/dirac-web-job-listing.png)

:::{admonition} Note
:class: callout

As for the VO registration, you need to use a browser where you have installed
your certificate and the [CERN CA][cernca] certificates. Firefox usually works
fine, Safari might also work. Google Chrome usually does not work.
:::

[diracweb]: https://voilcdiracwebapp2.cern.ch/DIRAC/?view=tabs&theme=Crisp&url_state=1|*DIRAC.JobMonitor.classes.JobMonitor
[signup]: https://fcc-auth.cern.ch/start-registration
[diracweb]: https://voilcdiracwebapp2.cern.ch/DIRAC/?theme=Classic&url_state=1|*DIRAC.JobMonitor.classes.JobMonitor:,
[cernca]: https://ca.cern.ch/cafiles/certificates/Grid.aspx
13 changes: 10 additions & 3 deletions distributed-computing/Workflows.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
# FCC DIRAC example workflows

These pages provide - and dissect - examples of workflows to be run on distributed resources with the [DIRAC Interware system][dirac].
>
> Original author: Gerardo Ganis
>

The python scripts discussed in these pages are available at [https://github.com/HEP-FCC/FCCDIRAC][fccdirac].
These pages provide &mdash; and dissect &mdash; examples of workflows to be run
on distributed resources of FCC with the [DIRAC Interware system][dirac].

The python scripts discussed in these pages are available at
[https://github.com/HEP-FCC/FCCDIRAC][fccdirac].

[dirac]: https://dirac.readthedocs.io/en/latest/
[fccdirac]: https://github.com/HEP-FCC/FCCDIRAC
[fccdirac]: https://github.com/HEP-FCC/FCCDIRAC/tree/master/workflows

```{eval-rst}
.. toctree::
:maxdepth: 2
:caption: Contents:

workflows/Overview.md
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading