Global Open Science Cloud(GOSC)
Practice at CNIC
Background of the project
At present, open scientific cloud platforms such as China Science and Technology Cloud (CSTCloud), European Open Science Cloud (EOSC), and African Open Science Platform (AOSP) have been built around the world to promote the popularization and application of open scientific research models. However, platforms such as CSTCloud, EOSC, and AOSP are independent of each other and cannot support global scientific and technological cooperation to solve problems such as new crown pneumonia, carbon neutrality, Global issues and challenges such as natural disasters. Therefore, in 2019, China Science and Technology Cloud proposed to build the "Global Open Science Cloud" (GOSC) initiative at the International Data Committee (CODATA), in order to realize the interconnection and sharing of open science cloud platforms in different fields, institutions, countries and regions [3]; In 2021, the General Conference of the United Nations Educational, Scientific and Cultural Organization (UNESCO) pointed out that the future open science The Xueyun platform must achieve international interconnection and high interoperability, so as to avoid the fragmentation of scientific and technological resources and narrow the digital, technological and knowledge gaps between countries and within countries.
Related research
At present, for cloud platform interoperability and resource sharing scenarios, academia and industry have proposed multiple conceptual models such as multi-cloud, hybrid cloud, cloud federation, InterCloud, Sky Computing, cross-cloud, and inter-cloud computing. For example, the "multi-cloud" system manages and monitors the resources and operating status of multiple cloud platforms (usually referred to as shared clouds), and selects different cloud platforms to deploy and run application services, so that users can avoid platform lock-in and obtain maximum cost-effectiveness; "hybrid cloud" system It is a special case of a "multi-cloud" system, which schedules application workloads in real time according to the system's running status and dynamically migrates and deploys them between public clouds and private clouds; Monitor the running status of the cloud platform, manage the metadata information of the cloud platform, and provide functional services such as resource discovery, monitoring and billing, and identity authentication for applications.
Our understanding of the concept of the cloud federation system is as follows: the cloud federation system is a middleware system for multi-source heterogeneous cloud platform collaboration, which enables cloud platforms and applications to be interconnected to form a cloud network, and applications can be orchestrated and run on cloud network nodes Business load, using cloud network resources (communication, computing, data, etc.) to solve problems such as cost-effectiveness, platform locking and scalability of a single cloud platform.
Conceptual Model
The cloud platform in the cloud federation system is divided into two types: physical cloud platform and virtual cloud platform. The physical cloud platform refers to the cloud computing system running on the physical server, and the virtual cloud platform refers to the resource integration management middleware system of the physical cloud platform. As shown in the figure, the peer-to-peer agent mode cloud federation system realizes large-scale cloud platform interconnection and resource sharing through three types of middleware: cloud federation agent platform, interface gateway, and cloud federation intermediary platform.
service provider
The interface gateway and the cloud platform form the basic unit of cloud federation resource supply (federation members), in which the cloud platform hosts resources to the cloud federation agent platform, and the interface gateway provides functional services such as interface adaptation and identity authentication for the resource hosting process
Cloud Federation Proxy
The cloud federation proxy platform and federation members constitute the basic unit of resource sharing (cloud federation proxy point) of the cloud federation system, in which the cloud federation proxy platform manages the resources of federation members and provides proxy services for federation members and applications respectively: (1) Provided as resources Or, the proxy federation member supplies cloud computing resources to other cloud federation proxy points. (2) As a resource consumer, the proxy application schedules the resources of federation members to provide unified integrated cloud services for the application. From the application point of view, the cloud federation proxy point shields the multi-source heterogeneity of the system cloud platform, making the cloud service calling method exactly the same as that of a single cloud platform.
Cloud Federation Broker
The cloud federation intermediary platform coordinates the cloud federation agent points to establish a peer-to-peer resource sharing relationship, and provides safe and reliable intermediary services for the resource sharing process. The cloud federation intermediary platform needs to provide four types of basic intermediary services (1) Metadata management: maintain the cloud service metadata information of the global agency points, and enable the agency points to perform resource release, search and update operations (2) Identity authentication: establish Trust relationship, so that different agents can mutually verify identity tokens (3) Service billing: provide billing guarantee services for cross-platform resource consumption behaviors, ensuring the security and accuracy of the billing process (4) Contract management: use services The contract method establishes the peer-to-peer resource sharing relationship between cloud federation agent points, ensuring that both parties to the contract perform resource sharing operations in accordance with the terms of the contract.
The key issue
Heterogeneous cloud platform interoperability
When providing federated cloud services, federated cloud service interface calls need to be resolved into a federated cloud service integration plan composed of heterogeneous cloud platform operations. This process mainly includes three basic operations: resource supply selection, heterogeneous cloud service interface adaptation, and resource transfer (1) Resource supply selection: There are multiple resource supply paths in the peer-to-peer agent resource sharing network. The federated cloud service integration system can learn from the commercial supply chain idea to model the resource sharing network, and use network flow optimization strategies such as minimum cost and maximum flow Dynamically select a reasonable resource supply cloud platform for federated cloud services; (2) Heterogeneous cloud service interface adaptation: proxy middleware needs to uniformly manage federated cloud service metadata in accordance with OCCI, CDMI and other intermediate cloud service models [28], and establish heterogeneous cloud service interfaces. The adaptation and mapping relationship between the cloud platform and the intermediate cloud service service model provides users with a unified federated cloud service interface; (3) resource transmission: federated cloud services need to exchange and transfer resources between heterogeneous cloud platforms, through standard specification formats Files such as OVF, AMI, VHD, and JSON persist resource objects such as images, data, virtual machines, and containers, and transfer resource objects to the target cloud platform and synchronize the runtime status of resource objects [29].
Federal cloud service metering and billing
The resource sharing process of the federal cloud platform is essentially a value-based transaction or exchange process, in which the cloud service metering and billing process should combine cloud service functions and scientific research indicators at the same time to model the value of cloud computing resources in open scientific scenarios. Use blockchain technology, smart contracts and incentive mechanisms to enable participants to trade resources in a closed-loop system.
Unified certification of federal cloud services
The unified certification of federal cloud services provides a new model for the collaboration of information resources and services in the process of new cross-domain scientific research cooperation. Scientific research collaboration in a peer-to-peer and independent environment is the key to all kinds of information resources and services. The basis for collaborative work. The federal cloud service unified authentication system needs to provide overall technical solutions for different authentication protocol identity providers accessing the federation, as well as heterogeneous service (resource) providers, involving a series of key technologies, software systems, tool sets, software Serve.
Unified monitoring of federal cloud services
Federated cloud service unified monitoring is a set of production systems based on the federated business system that can cover the operation and maintenance of the federated system; it supports three groups of people: ordinary users, professional operation and maintenance personnel, and management personnel, and realizes from collection, intelligent analysis, massive An operation and management subsystem integrating storage, multi-dimensional display, unified authorization, and detailed processes.