What are the challenges facing astronomy? How can astronomy cope with the changing landscape for both data and collaborations?
Here I tried to frame the problem and give some possible answers to what astronomers and others needs to address.
Cyberinfrastructure And Astronomical ResearchCybera Inc.
Chris Pritchett
Professor, Physics and Astronomy,
University of Victoria
Presented at the Cybera/CANARIE National Summit 2009, as part of the session "Research Transformed by Cyberinfrastructure." This panel featured researchers who have seen their work transformed through cyberinfrastructure – ie. collaborations made possible, mountains of data rendered intelligible, remote instrumentation accessed.
The Large Interferometer For Exoplanets (LIFE) II: Key Methods and TechnologiesAdvanced-Concepts-Team
The LIFE initiative has the goal to develop the science, the technology and a roadmap for an aspiring space mission that will allow humankind to detect and characterize, via nulling interferometry, the atmospheres of hundreds of nearby extrasolar planets including dozens that may be similar to Earth. This follow-up talk will tackle more of the techniques and technologies that will enable such an ambitious undertaking. I will outline the underlying measuring principle, and provide some overview over essential technologies, their current status and necessary developments.
Cyberinfrastructure And Astronomical ResearchCybera Inc.
Chris Pritchett
Professor, Physics and Astronomy,
University of Victoria
Presented at the Cybera/CANARIE National Summit 2009, as part of the session "Research Transformed by Cyberinfrastructure." This panel featured researchers who have seen their work transformed through cyberinfrastructure – ie. collaborations made possible, mountains of data rendered intelligible, remote instrumentation accessed.
The Large Interferometer For Exoplanets (LIFE) II: Key Methods and TechnologiesAdvanced-Concepts-Team
The LIFE initiative has the goal to develop the science, the technology and a roadmap for an aspiring space mission that will allow humankind to detect and characterize, via nulling interferometry, the atmospheres of hundreds of nearby extrasolar planets including dozens that may be similar to Earth. This follow-up talk will tackle more of the techniques and technologies that will enable such an ambitious undertaking. I will outline the underlying measuring principle, and provide some overview over essential technologies, their current status and necessary developments.
Galaxy Forum USA 2016 - Prof Imke de Pater, UC BerkeleyILOAHawaii
Background:
Galaxy Forum is the primary education and outreach initiative of ILOA, it is an architecture designed to advance 21st Century science, education, enterprise and development around the world.
Galaxy Forums are public events specifically geared towards high school teachers, educators, astronomers of all kinds, students and the general public. Presentations are provided by experts in the fields of astrophysics / galaxy research, space exploration and STEM education, as well as related aspects of culture and traditional knowledge. Interactive panel discussions allow for community participation and integration of local perspectives.
Stats:
Almost 70 Galaxy Forums, with a total of about 300 presentations to date.
Held in 26 locations worldwide including Hawaii, Silicon Valley, Canada, China, India, Southeast Asia, Japan, Europe, Africa, Chile, Brazil, Kansas and New York.
Started with Galaxy Forum USA, July 4, 2008 in Silicon Valley, California.
International Lunar Observatory Association (ILOA) is an interglobal enterprise incorporated in Hawaii as a 501(c)(3) non-profit to expand human knowledge of the Cosmos through observation from our Moon and to participate in internationally cooperative lunar base build-out, with Aloha – the spirit of Hawai`i.
In February, 2017 the Space Horizons workshop at Brown University focused on sending spacecraft to the Alpha Centauri stellar / planetary system. The first questions that come to mind include: getting there in some relevant amount of time, i.e. less than a human lifespan; communicating back over the 4+ light years distance; surviving the space environment en route; What do you do when you get there, and; how much is all that going to cost?
We also asked: what could we learn by going there vs. wouldn’t we learn more directing those resources to developing instruments here in the earth and near-earth environment to observe the star system (and probably other star systems) and exoplanets remotely? Would going there have some special meaning to people on earth as America’s landing of people on the moon did, versus the Soviet robotic missions? And what might we learn along the way - thinking our way to another star - that might change how we do other possibly completely unrelated things - closer to home?
We barely managed to even address all those questions in a day and a half at Brown, but undaunted I will do the best I can to distill that concentrated dose of interstellar travel down another factor of 30 into a talk 0.0014% of the trip time to Alpha Centauri at the speed of light. I.e. 30 minutes.
On March 8,9 and 10, NASA, the Space Telescope Science Institute, Northrop Grumman Aerospace Systems, and Microsoft Research organized the James Webb Space Telescope (JWST) experience exhibit as part of the South By SouthWest Interactive Festival. The full scale model of the JWST was positioned between the Long and Palmer center right outside downtown Austin, TX. Over 3 days more than 15,000 people attended our events and witnessed astronomers and engineers give talks about the science and challenges of building Hubble's successor in front of a 6 meter wide screen provided by Microsoft Research. The event was a tremendous success.
The James Webb Space Telescope is NASA's next flagship mission. Webb will revolutionize astronomy in the infrared like the Hubble Space Telescope has done for the visible portion of the spectrum over the last 22 years. Webb will reveal the story of the formation of the first starts and galaxies, investigate the processes of planet formation, and trace the origins of life.
Scalable Data Mining and Archiving in the Era of the Square Kilometre ArrayChris Mattmann
Keynote presentation at the HPC User Forum 2012 in Darborn, MI, September 19, 2012. http://www.hpcuserforum.com/registration/dearborn2012/dearbornagenda.pdf
• “Detecting radio-astronomical "Fast Radio Transient Events" via an OODT-based metadata processing pipeline”, Chris Mattmann, Andrew Hart , Luca Cinquini, David Thompson, Kiri Wagstaff, Shakeh Khudikyan. ApacheCon NA 2013, Februrary 2013
Galaxy Forum USA 2016 - Prof Imke de Pater, UC BerkeleyILOAHawaii
Background:
Galaxy Forum is the primary education and outreach initiative of ILOA, it is an architecture designed to advance 21st Century science, education, enterprise and development around the world.
Galaxy Forums are public events specifically geared towards high school teachers, educators, astronomers of all kinds, students and the general public. Presentations are provided by experts in the fields of astrophysics / galaxy research, space exploration and STEM education, as well as related aspects of culture and traditional knowledge. Interactive panel discussions allow for community participation and integration of local perspectives.
Stats:
Almost 70 Galaxy Forums, with a total of about 300 presentations to date.
Held in 26 locations worldwide including Hawaii, Silicon Valley, Canada, China, India, Southeast Asia, Japan, Europe, Africa, Chile, Brazil, Kansas and New York.
Started with Galaxy Forum USA, July 4, 2008 in Silicon Valley, California.
International Lunar Observatory Association (ILOA) is an interglobal enterprise incorporated in Hawaii as a 501(c)(3) non-profit to expand human knowledge of the Cosmos through observation from our Moon and to participate in internationally cooperative lunar base build-out, with Aloha – the spirit of Hawai`i.
In February, 2017 the Space Horizons workshop at Brown University focused on sending spacecraft to the Alpha Centauri stellar / planetary system. The first questions that come to mind include: getting there in some relevant amount of time, i.e. less than a human lifespan; communicating back over the 4+ light years distance; surviving the space environment en route; What do you do when you get there, and; how much is all that going to cost?
We also asked: what could we learn by going there vs. wouldn’t we learn more directing those resources to developing instruments here in the earth and near-earth environment to observe the star system (and probably other star systems) and exoplanets remotely? Would going there have some special meaning to people on earth as America’s landing of people on the moon did, versus the Soviet robotic missions? And what might we learn along the way - thinking our way to another star - that might change how we do other possibly completely unrelated things - closer to home?
We barely managed to even address all those questions in a day and a half at Brown, but undaunted I will do the best I can to distill that concentrated dose of interstellar travel down another factor of 30 into a talk 0.0014% of the trip time to Alpha Centauri at the speed of light. I.e. 30 minutes.
On March 8,9 and 10, NASA, the Space Telescope Science Institute, Northrop Grumman Aerospace Systems, and Microsoft Research organized the James Webb Space Telescope (JWST) experience exhibit as part of the South By SouthWest Interactive Festival. The full scale model of the JWST was positioned between the Long and Palmer center right outside downtown Austin, TX. Over 3 days more than 15,000 people attended our events and witnessed astronomers and engineers give talks about the science and challenges of building Hubble's successor in front of a 6 meter wide screen provided by Microsoft Research. The event was a tremendous success.
The James Webb Space Telescope is NASA's next flagship mission. Webb will revolutionize astronomy in the infrared like the Hubble Space Telescope has done for the visible portion of the spectrum over the last 22 years. Webb will reveal the story of the formation of the first starts and galaxies, investigate the processes of planet formation, and trace the origins of life.
Scalable Data Mining and Archiving in the Era of the Square Kilometre ArrayChris Mattmann
Keynote presentation at the HPC User Forum 2012 in Darborn, MI, September 19, 2012. http://www.hpcuserforum.com/registration/dearborn2012/dearbornagenda.pdf
• “Detecting radio-astronomical "Fast Radio Transient Events" via an OODT-based metadata processing pipeline”, Chris Mattmann, Andrew Hart , Luca Cinquini, David Thompson, Kiri Wagstaff, Shakeh Khudikyan. ApacheCon NA 2013, Februrary 2013
Astronomical Data Processing on the LSST Scale with Apache SparkDatabricks
The next decade promises to be exciting for both astronomy and computer science with a number of large-scale astronomical surveys in preparation. One of the most important ones is Large Scale Survey Telescope, or LSST. LSST will produce the first ‘video’ of the deep sky in history by continually scanning the visible sky and taking one 3.2 giga-pixel image every 20 seconds. In this talk we will describe LSST’s unique design and how its image processing pipeline produces catalogs of astronomical objects. To process and quickly cross-match catalog data we built AXS (Astronomy Extensions for Spark), a system based on Apache Spark. We will explain its design and what is behind its great cross-matching performance.
The first "Insights in Technology Conference" was in Schaffhausen on December 16, 2019. The event is organized by the Schaffhausen Institute of Technology SIT. Special guest is Nobel Prize winner Wolfgang Ketterle.
Schaffhausen Institute of Technology website: http://sit.org
High Performance Cyberinfrastructure Required for Data Intensive Scientific R...Larry Smarr
11.06.08
Invited Presentation
National Science Foundation Advisory Committee on Cyberinfrastructure
Title: High Performance Cyberinfrastructure Required for Data Intensive Scientific Research
Arlington, VA
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 3
Wolfram Data Summit: New Frontiers in Astronomy
1. New Frontiers in Astronomy Dr Alberto Conti
Space Telescope Science Institute
2. • Visible – Hubble Space Telescope
• Gamma rays - Compton Gamma Ray Obs.
• X-rays - Chandra X-ray Observatory
April 2006
• Infrared - Spitzer Space Telescope
3. Optical & UV
Data Archive
Optimize the science from community-led
astrophysics missions and projects.
Develop, nurture, and share innovations in
space astronomy science operations.
Collaborate on the next generation of space
astrophysics programs.
4. Astronomy Project Timeline
A Partial List of Key Astrophysics Facilities
Ares V Flights
Beyond Einstein
INTEGRAL WISE JWST
SWIFT SIM? TPF?
WMAP Herschel - Planck
Kepler
GLAST
GALEX
FUSE
XMM
Chandra
Spitzer
HST
SOFIA
SDSS
VLT & Gemini Observatories
PANSTARRS
LSST
TMT
ALMA
NVO Development NVO Operations
2000 2005 2010 2015 2020
Start date and Probable Duration
5. Astronomy Project Timeline
STScI Project and Mission Activity
Ares V Flights
Beyond Einstein
INTEGRAL WISE JWST
SWIFT SIM? TPF?
WMAP Herschel - Planck
Kepler
GLAST
GALEX
FUSE
XMM
Chandra
Spitzer
HST
SOFIA
SDSS
VLT & Gemini Observatories
PANSTARRS
LSST
TMT
ALMA
NVO Development NVO Operations
2000 2005 2010 2015 2020
Start date and Probable Duration
6. Space is big!
For one picture you need a
2 Trillion pixels camera!
Challenges for the Future
7. Monochrome : 4 Terabytes
or
5% of the Library of Congress
Challenges for the Future
8. Color: 100 Terabytes
or
the 21% more than the entire
Library of Congress
Challenges for the Future
9. Time: 10 Petabytes
or
120 times the entire
Library of Congress
Challenges for the Future
10. New analysis & visualization tools are required
Challenges for the Future
11. Astronomy is changing
Detectors follow
Moore’s Law
Total data doubles
every year
Growth over 25
years is a factor of
30 in glass, 3000 in
pixels
12. Computer
Biology Economics
Science
Medicine Government Astronomy
Massive amounts
of information
13. e-S
Computer
Science
Biology Economics
c ien
Medicine c
Government
e
Astronomy
Massive amounts
of information
14. Data Volume
Time
New Science Paradigm for Astronomy
15. New Paradigm Issues
• Moving data around is hard
• Extracting knowledge is hard
• Complex, difficult to use
• Hard for user to publish their own data
• Many distributed services are unreliable
16. ASTRONOMY IS SPECIAL!
No commercial value
Ideal testbed for
complex algorithms
Interesting problems
Plenty of data
17. ADAPT OR PERISH
Google Earth, Microsoft
Virtual Earth have
revolutionized the way
we look at our planet
We need a new
synergistic approach to
the challenge of bringing
the universe to our
desktops
18. New Science Paradigm:
First Iteration
Few Data Standards, Some Protocols
Mission A
Mission B
Observatory X
Mission C
Observatory Y
19. New Science Paradigm:
Second Iteration
Data Standards, Protocols
Simple Mining Tools
Mission A
Metadata Mission B
Observatory X
Mission C
Observatory Y
20. New Science Paradigm:
Science 2.0
Individual
Users
MAST @ STScI
Intʼl Data Centers
Kitchen Sink
NASA Data Centers
21. Global Challenges
• Reduce obstacles to Capturing, Organizing,
Summarizing, Analyzing,Visualizing, and Curating
• Consider data and algorithms as “the product”
• Adopt semantic technologiesclustering and
automated metadata tagging,
to enable
mining
• Transition to the new astronomy
• Sociological issues
22. Technological Challenges
• Infrastructure not available for intensive
data mining
• Solutions for handling large datasets are
lacking
• Cloud hosting solutions still expensive
• Unclear if commercial solutions can fit
science needs
23. • We must partner with other academic
disciplines: Computer Science, Statistics, ...
• We must leverage partnerships with
industry interested in enabling Science 2.0
• We must learn to be humble and ask for
help
• We must remember that we have the
greatest datasets in the world (universe
really!)