Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

e-VLBI: Science over High-Performance Networks 7717590

24 views

Published on

Fundamental Physics in Radio Astronomy
Radio astronomical measurements allow the study of a variety of questions in fundamental physics. Those questions range from the equation-of-state of super-dense matter to the investigation of fundamental forces such as gravity and magnetism. In particular, the research group searches for and exploits fast-rotating neutrons stars that are visible as radio pulsars. Their observations allow us to test general relativity and alternative theories of gravity and works towards the detection of a long-wave cosmological gravitational wave background.

Further studies exploit the information that is imprinted in radio emission about cosmic magnetic fields. Pulsars are used to study the magnetic milky way, while far-distant, external galaxies allow us to study cosmic magnetism. Magnetic fields are also important during the formation and the evolution of neutron stars, so that supernovae as well as the properties of a variety of radio-loud neutron stars are studied.

The research group is also involved in the development of innovative radio telescopes and works on projects such as the Low Frequency Array (LOFAR) and the Square Kilometre Array (SKA) and the science that will be enabled with these gigantic telescopes.


Radio Astronomy / VLBI

https://www.mpifr-bonn.mpg.de/research/vlbi

Published in: Science
  • Be the first to comment

  • Be the first to like this

e-VLBI: Science over High-Performance Networks 7717590

  1. 1. 1 Masaki Hirabaru and Yasuhiro Koyama {masaki,koyama}@nict.go.jp APEC-TEL APGrid Workshop September 6, 2005 e-VLBI: Science over High-Performance Networks
  2. 2. 2 Radio Telescopes NICT Kashima Space Center 34m Onsala Space Observatory 20m (left) Perks 64m (right) Australia Telescope National Facility MIT Haystack 18m Shanghai 25m
  3. 3. 3 • Geophysics and Plate Tectonics VLBI ApplicationsVLBI Applications 鹿島 ハワイ アラスカ 5700km5700km 5400km5400km 4700km4700km 鹿島-ハワイの基線長変化 - 400 - 200 0 200 400 1984 1986 1988 1990 1992 1994 年 基線長(mm) アラスカ-ハワイの基線長変化 - 400 - 200 0 200 400 1984 1986 1988 1990 1992 1994 年 基線長(mm) 鹿島-アラスカの基線長変化 - 400 - 200 0 200 400 1984 1986 1988 1990 1992 1994 年 基線長(mm) -63.5 ± 0.5 mm/year -46.1 ± 0.3 mm/year 1.3 ± 0.5 mm/year Kauai Fairbanks Kashima Kashima-Kauai Baseline Length Fairbanks-Kauai Baseline Length Kashima-Fairbanks Baseline Length
  4. 4. 4 VLBI Applications (2)VLBI Applications (2) Halca ( Muses-B ) NGC4261 Radio Telescope SatelliteRadio Telescope Satellite ‘Halca’ and its images‘Halca’ and its images Earth Orientation ParametersEarth Orientation Parameters • Radio Astronomy : High Resolution Imaging, Astro-dynamics • Reference Frame : Celestial / Terrestrial Reference Frame • Earth Orientation Parameters, Dynamics of Earth’s Inner Core
  5. 5. 5 VLBI (Very Long Baseline Interferometry) •e-VLBI geographically distributed observation, interconnecting radio antennas over the world ASTRONOMY GEODESY •Gigabit / real-time VLBI multi-gigabit rate sampling delay radio signal from a star correlator A/D clock A/D   Internet clock ~Gbps ~Gbps A B A B d Large Bandwidth-Delay Product Network issue
  6. 6. 6 VLBI System Transitions K5 Data Acquisition Terminal 1st Generation 2nd Generation 1983~ Open-Reel Tape Hardware Correlator 1990~ Cassette Tape Hardware Correlator e-VLBI over ATM 3rd Generation 2002~ PC-based System Hard-disk Storage Software Correlator e-VLBI over Internet K3 Correlator (Center) K3 Recorder (Right) K4 Terminal K4 Correlator 64Mbps 256Mbps 1 ~ 2Gbps
  7. 7. 7 Recent e-VLBI System DevelopmentsRecent e-VLBI System Developments K5 by NICTK5 by NICT ADS1000 (1024Msample/sec 1ch 1bit or 2bits) ADS2000 (64Msample/ch·sec, 16ch, 1bit or 2bits) IP-VLBI Board (~16Msample/ch·sec, ~4ch, ~8bits) PC : Data Acquisition Correlation VSI Correlator other DAS Internet PC-VSI Board (Supports VSI-H specifications) VSI VSI-E RTP/RTCP
  8. 8. 8 Motivations • MIT Haystack – NICT Kashima e-VLBI Experiment on August 27, 2003 to measure UT1-UTC in 24 hours – 41.54 GB NICT → MIT 107 Mbps (~50 mins) 41.54 GB MIT → NICT 44.6 Mbps (~120 mins) – RTT ~220 ms, UDP throughput 300-400 Mbps However TCP ~6-8 Mbps (per session, tuned) – BBFTP with 5 x 10 TCP sessions to gain performance • HUT – NICT Kashima Gigabit VLBI Experiment - RTT ~325 ms, UDP throughput ~70 Mbps However TCP ~2 Mbps (as is), ~10 Mbps (tuned) - Netants (5 TCP sessions with ftp stream restart extension) There was bandwidth available but we could not utilize.
  9. 9. 9 • Observing Bandwidth ∝ Data rate ∝ (Precision of Time Delay) -1 ∝ (SNR) 1/2 • Wave Length / Baseline Length ∝ Angular Resolution • Baseline Length ∝ (EOP Precision) -1 VLBI - CharacteristicsVLBI - Characteristics Faster Data Rate = Higher Sensitivity Longer Distance = Better Resolution
  10. 10. 10 Long Distant Rover Control (at least) 7 minutes one way delay Image Command Earth Mars When operator saw collision, it was too late.
  11. 11. 11 Long-Distance End-to-End Congestion Control Merge (Bottleneck) A+B > C Overflow Sender (JP) Receiver (US) Feedback BWDP: Amount of data sent but not yet acknowledged 64Kbps x 200ms = 1600B ~ 1 Packet 1Gbps x 200ms = 25MB ~ 16700 Packets 200ms round trip delay A B C
  12. 12. 12 Average TCP Throughput less than 20Mbps Q=50 Example How much speed can we get? ReceiverSender High- Speed Backbone L2/L3 SW 1G 100M Delay at light speed: 100ms 1G
  13. 13. 13 Analyzing Advanced TCP Dynamic Behavior in a Real Network (Example: From Tokyo to Indianapolis at 1G bps with HighSpeed TCP) The data was obtained during e-VLBI demonstration at Internet2 Member Meeting   in October 2003. Throughput RTT Window Sizes Packet Losses
  14. 14. 14   Kwangju Busan 2.5G Fukuoka Korea                                                         2.5G SONET KOREN Taegu Daejon 10G 1G (10G)1G 1G Seoul XP Genkai XP Kitakyushu Kashima 1G (10G) Fukuoka Japan 250km 1,000km 10G JGN II 9,000km 4,000km Los Angeles Chicago Washington DC MIT Haystack 10G 2.4G APII/JGN II Abilene Koganei 1G(10G) Indianapolis 100km bwctl server Performance Measurement Platform for High-Performance Scientific Data Transfer 10G Tokyo XP / JGN II I-NOC *Performance Measurement Point Directory http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html perf server e-vlbi server JGNII 10G GEANT SWITCH 7,000km TransPAC Pittsburgh U of Tokyo Locate the problem International collaboration to support for science applications U. Hawaii
  15. 15. 15 Solutions by Advanced TCPs • Loss-Based ► AQM (Advanced Queue Management) Reno, Scalable, High-Speed, BIC, … • Delay-Based Vegas, FAST • Explicit Router Notification ECN, XCP, Quick Start, SIRENS, MaxNet How can wee foresee collision (queue overflow)?
  16. 16. 16 TCP Performance with Different Queue Sizes
  17. 17. 17 * set to 100M for measurement Measuring Bottleneck Queue Sizes Switch / Router Queue Size Measurement Result ReceiverSender Capacit y C packet train lost packet measured packet Queue Size = C x (Delaymax – Delaymin) Device Queuing Delay (µs) Capacity (Mbps) Estimated Queue Size (1500B) Switch A 6161 100* 50p/75KB Switch B 22168 100* 180p/270KB Switch C 20847 100* 169p/254KB Switch D 738 1000 60p/90KB Switch E 3662 1000 298p/447KB Router F 148463 1000 12081p/18MB Router G 188627 1000 15350p/23MB cross traffic injected for measurement
  18. 18. 18 RouterSwitch 1Gbps (10G) 100Mbps (1G) b-1) Typical Bottleneck Cases RouterSwitch a) Queue ~100 Queue ~1000 VLANs Switch/ Router 10G LAN-PHY Ethernet Untag b-2) 9.5G WAN-PHY 802.1q Tag
  19. 19. 19 e-VLBI Demonstration in JGN II Osaka (Jan. 2005)  e-VLBI data transfer achieved ~700Mbps from Haystack to Osaka ~900Mbps from Kashima to Osaka  Software Cross Correlation ~240Mbps per station Dr. Koyama 4 Apple G5 Server machines (8 CPUs in Total) Osaka #7,#8 1G 1G Raid Disks 1G x4 1G x4 Raid Disks #5,#6 #1,#2 #3,#4 CPU x8 10G Tokyo NICT Kashima Abilene (10G) MIT Haystack CHI WAS JGN II Int’l (10G) 1G /10G 1G/2.5G *TCP parameters were tuned for the path.
  20. 20. 20                                                         VLBI Antenna Locations in North-East Asia Shintotsukawa 3.8m Tomakomai 11m, FTTH (100M) 70km from Sapporo Mizusawa 10m 20m 118km from Sendai Tsukuba 32m, OC48/ATMx2 SuperSINET Kashima 34m, 1Gx2 JGN II, OC48/ATM Galaxy Yamaguchi 32m 1G, 75M SINET Gifu 11m 3m, OC48/ATMx2 SuperSINET Usuda 64m, OC48/ATM Galaxy Nobeyama 45m OC48/ATM Galaxy Nanshan (Urumqi) 25m 70km from Urumqi Koganei 34m, 1Gx2 JGN II, OC48/ATM Galaxy Miyun (Beijing) 50m 50km from Beijing 2Mbps 2Mbps Yunnan (Kunming) 3m (40m) 10km from Kunming Sheshan (Shanghai) 25m 30km from Shanghai Observatory is on CSTNET at 100M Jeju 20m Tamna U Seoul 20m Yonsei U Ulsan 20m U Ulsan Daejon 14m Taeduk Ishigaki 20m Ogasawara 20m Chichijima 10m Iriki 20m Kagoshima 6m Aira 10m Legend connected not yet connected antenna under construction
  21. 21. 21 e-VLBI Data Transfer Real-time e-VLBI – flat-rate live data transfer Internet Synchronize Correlation Common e-VLBI – file transfer Carry the disk to the nearest station to put on-line Correlate among many combinations concurrently to get more precise data (like a virtual huge antenna) Future – e-VLBI infrastructure multicast and automated
  22. 22. 22 Summary • High-performance scientific data transfer faces on network issues we need to work out. • Big science applications like e-VLBI and High- Energy Physics need cooperation with network and Grid researchers. • Deployment of performance measurement Infrastructure over research networks is on- going on world-wide basis.

×