EOSC Future plans to make incremental improvements to the European Open Science Cloud platform in three phases.
Phase I
Researchers can access a rich set of EOSC resources, including some regional and thematic ones. They can also use a rich and diverse set of services, such as requesting resources through the INFRAEOSC-07 projects.
Phase II
Researchers can request access to centrally funded resources from multiple locations as we improve the integration functions and enables sharing between portals.
Phase III
Researchers can access a full spectrum of the available resources, from several funding streams and commercial sources. They can search data, recompose data into new data sets and discover new resources through smart, personalised recommendations. Users have access to services that support the full lifecycle of data processing, storage, and analysis, including a growing number of end-to-end workflows.
Each of the EOSC releases will see improved functionalities for users and providers, allowing more and better resource availability, sharing, discovery and recomposability. To stay on track, we aim to follow a detailed technological roadmap to improve all different aspects of our EOSC environment.
Resource sharing and discovery
- Researchers can see and reach all thematic and regional portals from the EOSC platform.
- Researchers can see services from the thematic clusters through EOSC.
- A researcher from PaNOSC can seamlessly use compute and/or storage resources provided by the e-Infrastructures to analyse data from the PaNOSC Research Infrastructure using the PaNOSC ID and without having to re-register across infrastructures.
- A researcher from a new community not involved in EOSC can get a virtual tour of available resources.
- A researcher searching on EOSC sees a comprehensive set of resources from multiple communities and clusters.
- Usage statistics for datasets will be collected and made available.
- Researchers using resources through EOSC will have common elements such as authentication and authorisation infrastructure, support, monitoring, and accounting.
- A researcher from PaNOSC can access an ESCAPE resource with the PaNOSC ID.
- Researchers using a thematic portal see services and data pulled in from the central EOSC registry.
- A richer set of horizontal services to support science is offered to researchers.
- A researcher can do the full lifecycle of data processing, storage, analysis, and publishing supported by resources available and transparently integrated through EOSC.
- EOSC allows research communities to build cross-disciplinary portals.
- Researchers can gauge the quality and suitability of resources based on usage statistics and feedback from other services.
Resource allocation
- A researcher can request computing or storage resources offered by EOSC Future and other projects related to EOSC (e.g. INFRAEOSC-07 and ARCHIVER)
- A researcher requesting resources or services can request access to EC-funded resources.
- Requesting resources through EOSC includes the possibility to access commercial resources.
- Researchers can request high-performance computing resources (e.g from EuroHPC).
Resource composability
- Researchers can see example cases of complex workflows using multiple resource providers.
- Researchers can access and use EOSC computing and storage resources.
- Researchers can see a rich range of horizontal resources and request access to them.
- Researchers can orchestrate data analysis on computing resources provided by multiple e-Infrastructure resource providers and transfer back the output to their storage system.
- Researchers can compare and select resources based on how easy they are to compose and connect to.
- Communities can offer their users fully integrated, end-to-end workflows and a number of these are available for various research topics.
EOSC-Core - enabling services to operate the EOSC
- The EOSC Marketplace shows all data collected from provider and resource profiles. Allows for scoring and rating.
- The EOSC platform allows direct onboarding (web + API) but as transfer of provider and resource records from other registries and vice-versa.
- EOSC AAI Federation operational.
- Monitoring and accountingallows checking of services based on availability of their web pages/endpoints. Can support better integration via specific metrics. Accounting supports usage tracking to support virtual access reimbursement by the EC.
- The helpdesk covers core services. Customisation based on new/updated technology and connected to the Service Management System.
- Order management is set up and includes integration of providers. Already used for services inherited from EOSC-hub and others that previously enabled it.
- The EOSC metrics dashboard provides statistics about services and requests of access and is fully integrated with the EOSC platform.
- The EOSC Marketplace is connected to monitoring to be able to enrich feedback with availability data. Includes AI/ML-based suggestion engine interfaces to automatically pull in resources from other catalogues in the EOSC central registry.
- Increased automatic validation tools and automatic flagging of resources which are likely to require review on the provider platform and resource registry
- e-infrastructure SP-proxies and cluster community authentication and authorisation infrastructures (AAIs) fully integrated into EOSC’s AAI Federation. Community AAIs can integrate.
- Automated/self-service integration of monitoring probes and metrics offered to providers. Monitoring can track availability and reliability and accounting of usage based on parameters in provider and resource profiles (location, sector, organisation type).
- The helpdesk covers core services and can redirect tickets to providers who have their own helpdesk. Helpdesk can be tested by providers who wish to use it.
- Order management can deploy and provision resources from selected providers, including from 07 projects.
- The EOSC metrics dashboard is enriched with additional information on EOSC resource providers and on activities of researchers in EOSC.
- Describe supported interfaces, standards, workflow languages, and metadata supported by EOSC resources for the marketplace. An advanced user dashboard is available.
- Inbuilt or integrated management/workflow engine to support management of applications, review of records, auditing and quality control as well as automatic provider communication for the provider platform and resource registry
- Community authentication and authorisation infrastructures (AAIs) seamlessly integrate with EOSC AAI federation through self-service onboarding.
- Automated monitoring includes automated thresholds, raising issues, or alarms in the Service Management System based on results. Accounting is related to capacity data offered by providers.
- Helpdesk-as-a-service available as optional add-on during onboarding. Integrated with central helpdesk functions.
- Order processing for thematic services can bundle orders for horizontal or basic services needed to deliver the thematic service and include deployment and provisioning.
- The EOSC metrics dashboard provides statistics related to the combined/integrated usage of EOSC resources.
EOSC-Exchange - registering resources and services
- Resources onboarded through prior projects (e.g. EOSC-hub, EOSC Enhance, eInfraCentral) remain available.
- ‘Horizontal’ resources from INFRAEOSC-07 projects are onboarded.
- A selection of resources from prior projects and 07 projects are integrated with functions from EOSC-Core.
- Initial cluster services have been onboarded and are listed and accessible through the EOSC Marketplace.
- Horizontal services, such as publishing workflows, data transfer, data packaging and container deployment, are demonstrated through 07 service instances.
- Resource requests are passed to the provider.
- First release of EOSC File Transfer Service as first service arising from the clusters to become a horizontal service.
- The majority of Science Cluster resources, services and data are available through the central resource registry and EOSC Marketplace.
- All main INFRAEOSC-07 horizontal services are integrated with EOSC-Core functionalities.
- Numerous other horizontal services, such as publishing workflows, data transfer, data packaging, virtual machine/container deployment and orchestration, are integrated with EOSC-Core functionalities.
- EOSC-Exchange offers AI/ machine learning-enabled suggestion functions for using the platform(s).
- EOSC-Exchange highlights which services can be easily integrated or composed.
- The onboarding process is extended to include validation of data sources to align to community (FAIR) metadata guidelines.
- The onboarding process for resources is extended to not only list them but also to include optional integration steps in the same workflow.
- Resource requests can lead to automatic provisioning of resources.
- Production release of EOSC File Transfer Service.
- First releases of other horizontal services arising from Science Clusters and communities.
- Ability to create thematic execution environment and virtual research environments based on integration of compliant thematic, horizontal, and core resources.
- The onboarding process allows for automated/self-service integration with some EOSC-Core functionalities.
- The onboarding process for resources is extended to include more optional integration steps in the same workflow.
- Resource requests integrated with procurement as well as provisioning functions.
- EOSC-Exchange includes numerous services from communities other than those represented by the Science Clusters
Science Clusters and Science Projects
- Cluster science projects started integration and adaptation of cluster/Research Infrastructure tools and services into the broader EOSC framework, including integration with authentication and authorisation infrastrucure, data sources moved into EOSC (FAIR) data stores, and catalogued, software development and exchange platforms available, workflow deployment mechanisms adapted.
- List of candidate services from clusters which can be generalised to be EOSC-wide offerings, with several already being developed to be included in EOSC.
- Most of the science projects have fully operational workflows using integrated EOSC services and tools, make use of EOSC Interoperability Framework, provide feedback from the science communities to the service deployment and operation, and can use resources made available through the clusters/research infrastructures.
- EOSC software catalogue/repository populated with identified cluster-provided services and tools.
- Full scale operation of science projects to the point where many have full scientific analyses ready or close to publication as full demonstrations of open cross-disciplinary science. Demonstrations of the full lifecycle of data processing, storage, analysis and publishing supported by resources available and transparently integrated through EOSC. Workflows deployed across cluster/Research Infrastructure resources and where appropriate on commercial cloud and/or European HPC resources.
- General availability of research infrastructure-originated horizontal services visible through EOSC platform and catalogues.
EOSC Interoperability Framework
- Provider and resource description framework v3.2 including data sources.
- Initial overview of available process identifier (PIDs) frameworks and guidelines for selecting PID types.
- Initial technical guidelines for Research Infrastructures and e-Infrastructures to connect AARC-compliant authentication and authorisation infrastructures (AAIs) proxies to EOSC’s federated AAI. EOSC AAI federation guidelines accepted by the Science Cluster communities and e-Infrastructures.
- Initial guidelines for metadata discovery and exchange on the basis of existing generic guidelines
- Initial guidelines for reporting accounting metrics for virtual access by INFRAEOSC-07
- Initial guidelines for monitoring service URLs registered in the EOSC Catalogue.
- Initial guidelines for managing orders specifying interfaces to forward orders to providers.
- Initial guidelines for handling user requests for services registered in the EOSC Catalogue.
- Initial guidelines for data ingesting and movement for processing in hybrid cloud environment.
- Initial guidelines for data repository.
- Initial guidelines for virtual machine/container management and orchestration.
- Initial guidelines for high-performance and high-throughput computing clusters on demand and multi-tenant containerised job submission.
- Initial Interoperability guidelines for machine learning and review of existing interoperability guidelines
- Provider and resource description framework v3.5 including research products and interoperability guidelines/best practices for horizontal services in EOSC-Exchange.
- Initial guidelines for new process identifier (PID) types and standards to connect PID frameworks to PID Graphs. Draft guidelines for PID service providers for minimum kernel type information
- Initial technical guidelines to connect identity provide r(IdP) and authentication and authorisation infrastructures (AAIs) proxies from public and private sector service providers to the EOSC Federated AAI should become technical guidelines for cross-infrastructure credential delegation and verification for supporting multi-step agent-driven workflows.
- Initial guidelines for communities to publish community-specific metadata and ontologies in EOSC.
- Initial interoperability framework for service providers for automated reporting of accounting metrics for VA.
- Initial interoperability framework for service providers for monitoring service availability.
- Initial interoperability framework for service providers for automatic dispatching of the orders for services registered in the EOSC Catalogue.
- Initial interoperability framework for service providers for automatic dispatching and handling of user requests for services registered in the EOSC Catalogue.
- Guidelines for data ingesting and movement for processing in hybrid cloud environment improved according to user communities’ feedback.
- Guidelines for data repository improved according to user communities’ feedback.
- Guidelines for virtual machine/container management and orchestration improved according to user communities’ feedback.
- Guidelines for high-performance and high-throughput computing clusters on demand and multi-tenant containerised job submission improved according to user communities’ feedback.
- Guidelines for machine learning/deep learning data analytics services improved according to user communities’ feedback.
- Provider and resource description stable release v4.0 incorporating new features requested by the user and provider communities.
- Draft interoperability framework for a process identifier (PID) meta-resolver and guidelines for PID service providers for minimum kernel type information.
- Technical interoperability guidelines for supporting cross-sector access to the EOSC Federated authentication and authorisation infrastructure (AAI)
- Guidelines for minimum metadata to support the discovery, metadata exchange, and crosswalks of research products across communities.
- Extended interoperability framework for service providers for automated reporting of accounting and usage metrics.
- Extended interoperability framework for service providers for monitoring service availability of services registered in the EOSC Catalogue.
- Extended interoperability framework for service providers for automatic dispatching of orders for services registered in the EOSC Catalogue.
- Extended interoperability framework for service providers for automatic dispatching and handling of user requests for services registered in the EOSC Catalogue.
- EOSC-endorsed guidelines for data ingesting and movement for processing in hybrid cloud environment adopted by one or more horizontal services.
- EOSC-endorsed guidelines for data repository adopted by one or more horizontal services.
- EOSC-endorsed guidelines for virtual machine/container management and orchestration adopted by one or more horizontal services.
- EOSC-endorsed guidelines for high-performance and high-throughput computing clusters on demand and multi-tenant containerised job submission adopted by one or more horizontal services.
- EOSC-endorsed interoperability guidelines for machine learning.