Mini-Symposium on Data over Distance:
Convergence of Networking, Storage, Transport, and Software Frameworks
July 19, 2018
Hanover, MD, USA
BACKGROUND
Data is ubiquitously generated at unprecedented rates and volumes in science and engineering scenarios, which is to be used to extract information critical to discovery and action. This data is expected to be transparently available, as and when needed, to users and automated codes across distance, over the increasingly sophisticated wide-area infrastructures. We consider a class of scenarios with high performance computing systems and instruments that generate and consume data, which are connected over high-capacity, agile networks. These are in sharp contrast with IoT and social media scenarios with a large number of data sources and highly distributed clouds; their solutions being extensively developed by the industry do not scale to high-performance requirements of our scenarios.
OVERVIEW
The effectiveness of Data over Distance solutions critically depends on finer optimizations of networking, storage, transport, and software frameworks, and on their interactions. We have reached the limits of the conventional approaches of storing data at a few sites and transporting them on-demand across wide-area networks. These solutions are simply too brittle and inefficient to support the emerging demands of higher data volumes and sophistication, for example, streaming, computational monitoring and steering. Furthermore, the sheer complexity of powerful computing systems, large data sources, and storage sites, all connected over high-performance networks, makes it extremely challenging to design, implement and operate them, and also to understand and continually optimize their performance. Small mismatches and inefficiencies could cascade and escalate to significant performance degradations. For example, credit limits of a file system can lead to severe limits on network throughput. Indeed, fundamentally new thinking and approaches are needed to realize these capabilities and the associated tools and technologies.
GOALS
The goal of this symposium is to assemble experts to discuss Next Big Things, including challenges, concepts and solutions that contribute to this new paradigm without the constraints of immediate programmatic considerations. The focus is on the possible and how to get there, including quantum leaps, game changers, and out of the box ideas.
TECHNICAL AREAS OF INTEREST
- Infrastructures and Frameworks
- Science of Data Transfers
- Analytics and Analytical Models
- Network Protocols
- Data Transfer Methods and Tools
- File and IO Systems
- Emerging Applications
- Performance Analysis and Optimization
- Security and Resilience
ORGANIZING COMMITTEE
- Neena Imam, ORNL
- Nagi Rao, ORNL
EXTERNAL ADVISOR
- Rich Carlson, DOE ASCR