This document provides an overview and agenda for a Python fundamentals session, including discussions of Jupyter notebooks, magics, running scripts from Jupyter cells, and virtual environments. It also briefly introduces NumPy, describing NumPy arrays as the main data structure and noting NumPy is fast due to C library bindings. Key topics covered are Jupyter project overview, line and cell magics, running scripts from Jupyter using various interpreters, creating and activating virtual environments in virtualenv and Anaconda, and basic NumPy array types.
An introduction to the OpenMP parallel programming model.
From the Scalable Computing Support Center at Duke University (http://wiki.duke.edu/display/scsc)
An introduction to the OpenMP parallel programming model.
From the Scalable Computing Support Center at Duke University (http://wiki.duke.edu/display/scsc)
Josh Williams
You may be familiar with connecting to a database using Python, but did you know it's possible for your database itself to use Python? PostgreSQL has the ability to write functions and procedures in PL/Python, which can use Python to both work with data inside your database, and bring in information from the outside world. This tutorial-format demonstration is designed to show you how to get up and running on PL/Python, and some of the cool things you can accomplish using it.
Clime is a Python library which lets you convert any module into a multi-command CLI program without any configuration.
It is a short tour of Clime.
The full documentation of Clime: http://clime.mosky.tw/.
Workshop slides originally given at the WOPR Summit in Atlantic City. Use JavaScript parsers and generators like Shift combined with Puppeteer and Chrome to reverse engineer web applications
Jupyter notebooks have arrived to stay as a means to document the scientific analysis protocol, as well as to provide executable recipes shared seamlessly among the community. This has triggered the rise of a plethora of complementary tools and services associated to them. This talk will cover different possibilities to use Jupyter notebooks and JupyterLab interface. We will start with the description of their basic functionalities, as well as functionality extensions not widely known by the community. We will describe how to take advantage of their cross-language capabilities to enhance collaborative work, and also use them as complementary assets in the paper publication process to provide reproducibility of the results. Other aspects on how to deal with modularity and scalability of long complex notebooks will be covered, and we will see several platforms for rendering and execution others then the browser and the local desktop. We will finish on how they are actually being used together with Docker and Binder as part of the versioned executable documentation of a project like Gammapy.
Josh Williams
You may be familiar with connecting to a database using Python, but did you know it's possible for your database itself to use Python? PostgreSQL has the ability to write functions and procedures in PL/Python, which can use Python to both work with data inside your database, and bring in information from the outside world. This tutorial-format demonstration is designed to show you how to get up and running on PL/Python, and some of the cool things you can accomplish using it.
Clime is a Python library which lets you convert any module into a multi-command CLI program without any configuration.
It is a short tour of Clime.
The full documentation of Clime: http://clime.mosky.tw/.
Workshop slides originally given at the WOPR Summit in Atlantic City. Use JavaScript parsers and generators like Shift combined with Puppeteer and Chrome to reverse engineer web applications
Jupyter notebooks have arrived to stay as a means to document the scientific analysis protocol, as well as to provide executable recipes shared seamlessly among the community. This has triggered the rise of a plethora of complementary tools and services associated to them. This talk will cover different possibilities to use Jupyter notebooks and JupyterLab interface. We will start with the description of their basic functionalities, as well as functionality extensions not widely known by the community. We will describe how to take advantage of their cross-language capabilities to enhance collaborative work, and also use them as complementary assets in the paper publication process to provide reproducibility of the results. Other aspects on how to deal with modularity and scalability of long complex notebooks will be covered, and we will see several platforms for rendering and execution others then the browser and the local desktop. We will finish on how they are actually being used together with Docker and Binder as part of the versioned executable documentation of a project like Gammapy.
Try to imagine the amount of time and effort it would take you to write a bug-free script or application that will accept a URL, port scan it, and for each HTTP service that it finds, it will create a new thread and perform a black box penetration testing while impersonating a Blackberry 9900 smartphone. While you’re thinking, Here’s how you would have done it in Hackersh:
“http://localhost” \
-> url \
-> nmap \
-> browse(ua=”Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.346 Mobile Safari/534.11+”) \
-> w3af
Meet Hackersh (“Hacker Shell”) – A new, free and open source cross-platform shell (command interpreter) with built-in security commands and Pythonect-like syntax.
Aside from being interactive, Hackersh is also scriptable with Pythonect. Pythonect is a new, free, and open source general-purpose dataflow programming language based on Python, written in Python. Hackersh is inspired by Unix pipeline, but takes it a step forward by including built-in features like remote invocation and threads. This 120 minute lab session will introduce Hackersh, the automation gap it fills, and its features. Lots of demonstrations and scripts are included to showcase concepts and ideas.
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakNETWAYS
Uphill battle against large enterprise it environments and IT corporate culture. How those difficulties turned out opportunities and clever implementations. Interesting modules, integrations and workflow pieces.
SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL SAMCSCMLA SCACLSALS CS L LSLSL
Shared Object images in Docker: What you need is what you want.Workhorse Computing
Docker images require appropriate shared object files (".so") to run. Rather than assume Ubuntu has the correct lib's, use ldd to get a list and install the ones you know you need. This can reduce the underlying images from GB to a few MB.
these are my presentation files I use in my PyML Course, for videos + voice records (in Persian language) contact me (they are all FREE for everyone)
Session 1: Introducing Python
.NET Core, ASP.NET Core Course, Session 4aminmesbahi
Session 4,
What is Garbage Collector?
Fundamentals of memory
Conditions for a garbage collection
Generations
Configuring garbage collection
Workstation
Server
.NET Core, ASP.NET Core Course, Session 3aminmesbahi
Session 3,
Introducing to Compiler
What is the LLVM?
LLILC
RyuJIT
AOT Compilation
Preprocessors and Conditional Compilation
An Overview on Dependency Injection
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
3. Jupyter, Overview
Jupyter:
Project Jupyter exists to develop open-source software, open-standards, and services for
interactive computing across dozens of programming languages (over 40 programming
languages, including Python, R, Julia, and Scala)
https://jupyter.org/try
4. Jupyter, Magics
Magics:
provide a mini command language that is orthogonal to the syntax of Python and is
extensible by the user with new commands.
Command: lsmagic
5. Jupyter, Magics - Line magics
Line magics:
these are commands prepended by one % character and whose arguments only
extend to the end of the current line.
%timeit np.linalg.eigvals(np.random.rand(100,100))
100 loops, best of 3: 7.06 ms per loop
6. Jupyter, Magics - Cell magics
Cell magics:
use two percent characters as a marker (%%), and they receive as argument both
the current line where they are declared and the whole body of the cell. you can
only use one cell magic per cell).
%%timeit a = np.random.rand(100, 100)
np.linalg.eigvals(a)
100 loops, best of 3: 7.4 ms per loop
7. Running Scripts from Jupyter
Running Scripts from Jupyter:
Jupyter has a %%script cell magic, which lets you run a cell in a subprocess of any
interpreter on your system, such as: bash, ruby, perl, zsh, R, etc
%%script python
import sys
print('hello from Python: %s' % sys.version)
hello from Python: 3.7.0 (default, Jun 28 2018, 08:04:48)
[MSC v.1912 64 bit (AMD64)]
8. Running Scripts from Jupyter
%%ruby
puts "Hello from Ruby #{RUBY_VERSION}“
Hello from Ruby 2.0.0
%%bash
echo "hello from $BASH“
hello from /usr/local/bin/bash
9. Running Scripts from Jupyter
Background Scripts
These scripts can be run in the background, by adding the --bg flag.
When you do this, output is discarded unless you use the --out/err flags to store output as above.
%%ruby --bg --out ruby_lines
for n in 1...10
sleep 1
puts "line #{n}"
STDOUT.flush
End
Starting job # 0 in a separate thread.
10. Virtual Environments
Virtual Environments
Allow us to set up virtual installations of Python and libraries on our machines.
We can have multiple versions of Python or libraries and easily activate or deactivate these
environments
11. Virtual Environments
Virtual Environments
virtualenv library for normal Python distributions
python -m venv sample-env
On Windows, run:
sample-envScriptsactivate.bat
On Unix or MacOS, run:
source sample-env/bin/activate
14. Numpy, Overview
Numpy
Is a Linear Algebra Library for Python, the reason it is so important for Data Science with Python is
that almost all of the libraries in the PyData Ecosystem rely on Numpy as one of their main building
blocks.
NumPy is also incredibly fast, as it has bindings to C libraries.
conda install numpy
pip install numpy
15. Numpy, Overview
Numpy
NumPy arrays are the main way we will use NumPy.
NumPy arrays essentially come in two flavors: vectors and matrices
Vectors are strictly 1-d arrays and matrices are 2-dC libraries.