InfiniteSpace / dCache¶
The DESY-hosted InfiniteSpace service, based on the dCache system, allows Helmholtz scientists and their collaboration partners to store scientific data and have access to it through different protocols across all Helmholtz Centres and associated institutes. It was one of the first services connected to the Helmholtz AAI. It is provided as part of the Helmholtz Cloud. See dcache.org and the dCache GitHub project for more details on the project.
When using this service, you agree to comply to the service’s Acceptable Use Policy.
Please be aware:
- This service is not like Sync&Share. If you want to use an interactive, easy-to-use file storage and sharing service, please do use Sync&Share instead.
- By design, files cannot be changed on-the-fly in dCache. Instead, every attempted change will result in deleting and rewriting that file.
- dCache InfiniteStorage is meant for large-scale (>>1T) data storage and transfer in mostly automated data processing environments. It provides a graphical user interface, but only for very basic functionality.
- Basic command-line and scripting knowledge is a must-have to make reasonable use of this service. You need to have people with such basic knowledge in your user group. HIFIS can only provide very limited support.
Getting Access for your Group¶
Usage of the service is not possible (and not reasonable) for single users. Instead, groups that are coordinated by a Helmholtz member can request access to the storage and then invite (also non-Helmholtz) users to their so-called “virtual organisations (VO)”.
See here on how to request storage usage as a group manager.
Using and Managing¶
Users must belong to a Helmholtz-led collaboration group (“VO”) that has been granted access to the service.
Web frontend¶
Most conveniently, one can use the Web frontend and log in via Helmholtz ID:
- Go to dCache InfiniteSpace,
- select “Log in via OpenID-Connect Account”,
- select “Helmholtz AAI(Keycloak)”,
- select your home institution (or Google, Github, ORCID),
- identify yourself at your home institution.
See also our illustrated tutorial on how such AAI login works in general.
Other ways¶
- Via rclone and oidc-agent
- API overview (Swagger UI)
- Using the
gfal
tool kit - others
Usage Scenarios¶
- Direct storage and publication using the web frontend, rclone etc.
- Streaming of data through WAN using suitable tools and protocols
- Read-Only access to DESY compute ressources if applicable
- Large scale data transfer with HIFIS: High-level intro to HTS
- Automation of distributed data processing pipelines: See demonstration on Helmholtz Imaging use case. As always: Contact us!
- More to come.
Have fun!¶
In case of problems or if you have suggestions, contact us at support@hifis.net.