Partners: European Synchrotron Radiation Facility (ESRF), Deutsches Elektronen-Synchrotron (DESY), Paul Scherrer Institute (PSI)
Project description
Serial (femto-second) X-ray-Crystallography (SFX) is a special variant of macromolecular X-ray crystallography aiming at rapid structural studies at room temperature. This highly innovative technology permits investigation of bio-structures not tractable with conventional X-ray crystallography, and is capable of studying fast in-situ biochemical processes.
Emerging technologies are allowing scientists to study bio-structures and fast in-situ biochemical processes that could not be detected through traditional methods like X-ray crystallography. These new processes have a rather complex data processing pipeline though, which relies especially on the fast and accurate image classification. The main goal of this project was to establish and develop a data-processing platform that integrates services allowing cross-disciplinary users with modest expertise gain rapid and convenient access to tools and documentation of the newest developments.
Technical challenge
Not all serial crystallographers have access to high-performance computing and storage facilities at their home institution. This lack of IT infrastructure can significantly limit their data analysis capabilities.
The ID29 beamline at ESRF is specifically built for serial macromolecular crystallography. Since its commissioning within a 6-month period, it has generated over 1.2 petabytes (PB) of raw data. Scientists using the ID29 beamline require access to high-performance computing (HPC) or storage infrastructure even when they are not physically present at ESRF to analyze this raw data. To address this need, a Virtual Infrastructure for Scientific Analysis (VISA) service has been deployed at ID29. This service enables users to access their raw data through the ESRF data portal and analyze it using the VISA service remotely.
ID29 beamline at ESRF is dedicated to serial macromolecular crystallography. Since its commissioning within 6 months period, it has generated over 1.2PB of raw data. The ID29 users or the scientist need access to HPC or storage infrastructure even when they on not present at ESRF on the site to analyse this raw data. By deploying Virtual Infrastructure for Scientific Analysis (VISA) service at the ID29, users can now access their raw data from the ESRF data portal and analyse this data using the VISA service.
EOSC Future added value
As a part of EOSC Future, the VISA service could be deployed at the ID29, this required a interaction between the beamline scientist as well as ESRF IT services to provide necessary software on the VISA service to ID29 users.
Main results
- The VISA service and ESRF data portal have been integrated into EOSC as part of EOSC Future activities
- Both of these services are accessible to serial crystallographers at ESRF, as well as other users of Photon and neutron infrastructure, and even the general public who wish to analyse ESRF’s open data
- In addition to providing access to open data and analysis software, ESRF is also opening up access to its HPC infrastructure