��app demonstrates fast data transfer to Japan and the US

Terabytes

��app demonstrates fast data transfer to Japan and the US

1 Dec 2025 - Pat Brans

The ability to move experimental data instantly and securely will determine how thousands of researchers participate in near-real time when the machine comes online in the 2030s.

Sharing data with partners worldwide. The red lines show the two 100 Gbps physical network paths from France to Japan via the US (Leg 1) and Asia (Leg 2). (For simplicity, links to centres not participating in the test are omitted.)

While Member scientists around the world will follow ��app experiments from their home countries, they will not be able to operate plant systems remotely. They will, however, be able to analyze data within seconds of an experiment and provide feedback to operators.

“It’s a kind of indirect participation,” explains Denis Stepanov, Computing Coordinating Engineer in ��app’s Control Program. “We quickly extract scientific data from the plant network and make it widely available so researchers can run calculations and feed results back during operations.”

Building the global data backbone

At the heart of this arrangement is the onsite Scientific Data and Computing Centre and its backup data centre in Marseille about 50 km away. This data centre has a dual purpose: it holds a redundant copy of all data generated by ��app and will serve as the distribution point to partners worldwide.

“By locating our backup and distribution hub in Marseille, we can protect the master data stored at ��app while providing high-speed, secure access for our international partners,” says Peter Kroul, Computing Center Officer at ��app.

The Cadarache site is connected to the Marseille centre via a redundant pair of dedicated 400 Gbps lines. In turn, the centre is connected, via the French network , to the pan-European , which provides access to other research and education networks—including (USA) and (Japan). This overall structure ensures that, even during intensive experimental campaigns, data can move at full speed while the primary plant network remains isolated and protected.

A summary of the August-September data transfers across a single 100 Gbps link. While long transfers were artificially capped to allow cooperative use of networks, the September peak illustrates full 100 Gbps consumption using the MMCFTP tool. (Image courtesy of ��É��.)

To move terabytes of data efficiently across 10,000 kilometres of fibre optics, the team needs software and hardware that can handle diverse systems without running the risk of vendor lock-in. “We cannot dictate what technologies our partners use on their side,” says Kroul. “So we built something flexible—able to connect to whatever they have, while still achieving high parallelization and efficiency even on high-latency links.”

The result is ��app.sync, a high-performance, open-source-based data-replication framework developed at the Scientific Data and Computing Centre. Drawing on the principles of rsync but heavily optimized, ��app.sync automatically parallelizes data streams, tunes network parameters, and maintains near-saturation speeds even over long-distance connections where latency is high.

��app.sync was designed to operate in environments with tools used by some of the partners—for example, the Massively Multi-Connections File Transfer Protocol (MMCFTP), which was developed by Japan’s National Institute of Informatics (NII).

Gbps: A measure of network transmission speed, representing one billion bits of data transferred per second. Because one byte equals eight bits, 100 Gbps corresponds to roughly 12.5 GB/s of data throughput under ideal conditions.

High-latency: High-latency describes network links where data packets experience long delays—typically due to the physical distance between systems or the number of hops through intermediate routers. Transferring data between France and Japan, for instance, involves a 10,000-kilometre journey across multiple submarine cables, introducing latency of more than 200 milliseconds. Such delays can degrade throughput unless the transfer software compensates with optimized buffering and parallel streams. ��app.sync is specifically engineered to maintain high performance even on these long, high-latency paths.

Massively Multi-Connections File Transfer Protocol (MMCFTP): A high-performance data transfer protocol engineered to maximize throughput by opening and coordinating a very large number of simultaneous network connections. Unlike traditional file-transfer tools, which typically rely on a handful of parallel streams, MMCFTP can orchestrate hundreds or even thousands of concurrent flows.

Parallelization: The process of dividing a large data transfer into multiple simultaneous streams to increase overall throughput. Instead of sending one massive file sequentially, ��app.sync splits and distributes the data across several network threads or channels, each handled independently.

Rsync: An open-source software utility designed to synchronize files and directories between two systems efficiently by sending only the data blocks that have changed. It’s widely used for backups and mirroring because it minimizes network load while ensuring data integrity. ��app.sync builds on rsync’s core principles but adds advanced tuning and multi-streaming capabilities for long-distance, high-speed scientific data transfers.

Saturation speeds: The point at which a network link is fully utilized, with data flowing as fast as the physical and protocol limits allow.

Global data network put to the test

This summer, ��app engineers carried out two large-scale data-transfer campaigns: one with Japan’s Remote Experimentation Centre (REC) in Rokkasho and the other with the DIII-D National Fusion Facility in San Diego (United States). For the purpose of the tests, ��app simulated the projected data acquisition scenarios.

The , conducted from mid-August to early September, built on a 2016 demonstration that reached 10 Gbps, which was what was available at the time. The new tests achieved two simultaneous 100 Gbps links—a twenty-fold increase. Engineers demonstrated continuous throughput, multi-path transfers, and resilience by simulating a submarine-cable outage between Marseille and Rokkasho. Both ��app.sync and MMCFTP were used in the tests, providing valuable insight into data transfer strategies and specific tuning for long-distance transfers.

It is expected that only a fraction of the data will be needed in near-real time by remote experimentalists. This data will be transferred as soon as it reaches primary storage. The bulk of the data, however—which needs to be available for off-line analysis—will be transferred via quiet overnight syncs. This second scenario was also tested.

“The key was to test not just network speed but the whole chain—hardware, software, and reliability,” says Stepanov. “Building the technical link is one challenge, but coordinating with all the network providers across Europe and Asia is just as complex. It takes time, alignment, and trust.”

Network throughput graph showing the dedicated 100 Gbps path for the DIII-D data challenge, the data transfer performance for one individual 176 TB test, and the ESnet portion of the end-to-end path between ��app and DIII-D.

In parallel, the ��app computing centre completed its full-scale data challenge with ESnet and the DIII-D fusion facility at General Atomics in San Diego (United States), supported by a trans-Atlantic link operated at 100 Gbps. Over ten full-scale runs, the teams achieved consistent end-to-end performance close to the link’s theoretical maximum. The test also demonstrated interoperability between ��app’s IBM Spectrum Scale storage and DIII-D’s BeeGFS-based Science DMZ infrastructure, again confirming ��app.sync’s ability to bridge heterogeneous environments.

“These results show that ��app’s international data ecosystem will scale and be ready for the operations we will face in the 2030s,” says Kroul. “We can already, with current technology, ensure that scientific data moves efficiently and reliably between ��app and partner institutions worldwide.”

See press releases from the and networks.

Return to newsline

��app