Speaker
Description
Scientific endeavors of unprecedented scale, such as Square Kilometre Array, Large
Synoptic Survey Telescope, Large Hadron Collider, will be or are already generating
colossal amounts of data. Furthermore, in a fundamental manner, OMICS and AI
depend on the availability and easy access to big data. This forces us to rethink the
hierarchy and relative importance of "compute - store - transport" elements. All these
components must be considered on equal footing, integrated and architected as almost
organic, coherent and coalesced federation.
In this talk I will chronicle InfiniCortex project run between 2014-2016.
Subsequently the U.S. DoE devised a concept of Superfacilities; a Global Research
Platform was established to facilitate big scientific data collaborations at the global
scale; and in Europe EuroHPC Joint Undertaking is funding three pre-exascale
computing infrastructures, each serving communities of users in many countries. All
this requires the tight integration of compute-store-transport and access components.
We are witnessing the emergence of Federated Supercomputing.
InfiniCortex was a supercomputer spanning the entire globe. It combined
supercomputing and storage resources located in Asia, Australia, USA and Europe into
one concurrent, seamless, global electronic brain. Over the three years of project
duration A*CRC in Singapore together with dozens of partners around the world
implemented:
i) ring-around-the-world connectivity with most links at 100Gbps using exclusively
InfiniBand transport, allowing almost 100% efficient, lossless and encrypted
communication between all processors, everywhere (RDMA) and InfiniBand
routing
ii) InfiniCloud: “designer choice” HPC Cloud instances with no limits on size of
computing resources, spanning four continents
iii) InfiniCortex enabled applications in critical areas of science and technology.
I will also report on continuation of this work at ICM, University of Warsaw with
partners around the globe to transmit data between Europe and Australia, and Europe
and Singapore, on establishing networks with efficient Data Trensfer Nodes (DTNs),
inter-city or inter-continental connected infrastructure enabling high-priority
computing, distributed supercomputing, collaboration, and scientific research.
