Solving BIG problems with Open Source: Condor

•

0 likes•443 views

This is an introduction to the queue distribution system Condor and how are we using it at the I3A (http://i3a.unizar.es)

Technology Business

> Solving Big problems with OS: Condor
> Antonio Sanz (ansanz@unizar.es)
> 09 / Nov / 11

3
> Antonio Sanz
> I3A System Manager
> HERMES HPC cluster sysadmin
> ansanz@unizar.es
> @antoniosanzalc

Problema inicial 6
> Dr. Good
> Neurologist
> Alzheimer research
> Process 20000
brain image scans
(1h/image)
> A thousand times.
Maybe two.

3. Sistemas de gestión de colas : Condor

Problema inicial 7
> Mrs. Nice
> Santa’s Logistic Officer
> Gift transportation
> Analize 6x10e7 possible
load/reindeers/routes
(10min/analysis)
> Before Christmas!

Condor Basics 10

Single queue

3. Sistemas de gestión de colas : Condor

Condor Basics 12

Multiple queues

3. Sistemas de gestión de colas : Condor

Problem can be broken into independent pieces 15

Condor Basics 17

For loops are
your best friends

3. Sistemas de gestión de colas : Condor

Condor Basics 18

While loops …
can sometimes be
convinced

3. Sistemas de gestión de colas : Condor

Condor Basics 19

Do it yourself !
3. Sistemas de gestión de colas : Condor

Condor Basics 21

Heterogeneous computing

Condor Basics 25

Fair use of resources

3. Sistemas de gestión de colas : Condor

Condor Basics 32

3. Sistemas de gestión de colas : Condor

Condor Basics 36

Job list ClassAd

3. Sistemas de gestión de colas : Condor

Condor Basics 41

Job running
3. Sistemas de gestión de colas : Condor

45

#!/bin/sh
# I’m hola.sh
echo Hola mundo desde `hostname`

#
# A Hello World .. In Condor!
#
# I’m hello.sub
Universe = vanilla
Executable = hola.sh
Log = hola.log
Output = hola.out
Error = hola.err
Queue

Hello, World !!

Lanzar el cálculo 46

condor_submit

4. Condor Basics – Un cálculo fácil

$48 Something tastier… #!/bin/sh # I’m hello2.sh OUTPUT=hello${1}.result cat hello.input >> $OUTPUT cat echo Hello world, I’m job $1 here from `hostname` > $OUTPUT # Execute n times with different outputs Universe = vanilla Executable = hello2.sh Transfer_input_files = hello.input WhenToTransferOutput = ON_EXIT_OR_EVICT Arguments = $(Process) Log = hello.log Output = hello.out Queue 10$

49

Perfect
Simulation

4. Condor Basics – Un cálculo fácil

Condor Basics 55

Hadoop Integration

3. Sistemas de gestión de colas : Condor

Condor Basics 63

3. Sistemas de gestión de colas : Condor

Condor Basics 64

> Conclusiones

3. Sistemas de gestión de colas : Condor

66

Antonio Sanz
ansanz@unizar.es
@antoniosanzalc

Slides here:
http://web.hermes.cps.unizar.es/doc/condor.pdf

Solving Big problems with OS: Condor is a presentation about Condor, an open-source workload management system for computer clusters and grids. Condor provides features for heterogeneous computing, job monitoring, and fair sharing of resources. It allows users to run long-running or short jobs and supports technologies like Hadoop, MPI, and GPU computing. While flexible and powerful, Condor has some limitations like a lack of native MPI support or slot-wise preemption. However, it remains a free and popular system for managing distributed workloads.

Cibercrimen qué hace tu dinero cuando no lo miras

Antonio Sanz Alcober

Este documento describe las diferentes etapas del cibercrimen y cómo se obtiene, distribuye, usa y blanquea el dinero de forma ilícita en la deep web. Explica cómo funcionan los ataques de phishing, troyanos bancarios y botnets, y cómo se venden datos robados. También analiza tendencias como el malware móvil y la mezcla del cibercrimen con otros delitos. Concluye que los ataques son cada vez más sofisticados y organizados, y se necesitan más expertos en seguridad para combatir estas

Computer Forensics: You can run but you can't hide

Antonio Sanz Alcober

This document discusses digital forensics and incident response (DFIR). It covers the key phases of digital forensics like identification, acquisition, preservation, analysis and dissemination. Acquisition involves collecting evidence from various sources like mobile devices, cloud storage and game consoles. Preservation of the evidence is critical following techniques like write blocking. Analysis techniques are discussed like recovering deleted data and analyzing file system metadata. The challenges of DFIR are also covered like virtual machines, network forensics and issues with the cloud. The document ends emphasizing the importance of ethics, certification and keeping up with new tools and techniques in this field.

Waft cloud

sabura

PRINT™ - Maximising Social Media ROI

Sociagility

The document discusses a social media measurement and planning system called PRINTTM that helps marketers maximize their return on investment from social media. PRINTTM analyzes social media performance across multiple channels, identifies areas for improvement, and predicts the business impact of optimizing engagement strategies. It provides strategic recommendations on where to focus resources based on brand objectives and known correlations between social media metrics, brand value, growth, and sales.

Introduction to PRINT™

Sociagility

This document introduces PRINTTM, a social media measurement system that assesses and tracks the competitive social brand performance of companies. It measures five key areas of social media success and provides scorecards, insights reports, and a tracker to analyze performance over time. The benefits of PRINTTM include comparative, actionable, objective, independent and credible social media metrics. It is used by both brands and agencies to understand strengths, weaknesses, and improve social strategies. Contact information is provided to learn more and request a demo.

Waft cloud

sabura

Six Great Minds – Enterprise 2.0

Sociagility

PRINTTM is a social media performance measurement system that analyzes five key attributes of social media across multiple platforms and channels. It provides comparative scores and insights for competitors to identify areas for improvement. The PRINTTM reports include a scorecard with overall and attribute scores for brands, breakdowns of performance by channel and attribute, and charts mapping relationships between attributes. Reports are customized for each client and can be conducted as one-time or ongoing analyses.

Domain driven design in a nutshell

Toni Esteves

Domain-driven design is a software development approach that focuses on modeling the core domain and problem space. It values collaboration between developers and domain experts to create a ubiquitous language for discussing the domain. The goal is to develop a deep understanding of the problem and build software that meets the needs of the business domain through an iterative process of learning and modeling.

Informatica forense

mausmr

Por qué un Pecha Kucha es mejor que un Iphone (y casi tanto como el chocolate)

Antonio Sanz Alcober

Manual oms[1]

Elizza Quezzad

Este documento presenta una guía práctica para la prevención de infecciones nosocomiales. Proporciona información sobre la epidemiología de las infecciones nosocomiales, programas de control de infecciones, vigilancia de infecciones, prevención de brotes e infecciones endémicas comunes, y precauciones para el control de infecciones. El objetivo es ayudar a los hospitales a implementar programas efectivos para reducir las tasas de infecciones nosocomiales y mejorar la seguridad de los pacientes.

La ciber guerra en las relaciones internacionales

carinna_07

Capacidades de China para la ciberguerra

Antonio Sanz Alcober

El documento describe las capacidades de inteligencia militar y civil de China para la ciberguerra y el ciberespionaje. China ha invertido fuertemente en estas áreas a través de su ejército, agencias de inteligencia como el MSS y el MPS, universidades e industrias tecnológicas. China ha llevado a cabo numerosas operaciones de ciberespionaje contra objetivos militares y comerciales de otros países para robar secretos e información tecnológica. China se ha posicionado como una potencia líder en el ciberespacio

07 decreto ap.julio 2007.joaquin

mariajoseadamuzgracia

Este documento describe la estructura y organización de los servicios de atención primaria de salud en una región. Se divide en cinco capítulos que detallan la organización territorial, las estructuras organizativas como los distritos y unidades de gestión clínica, el régimen de personal, y los mecanismos de participación profesional. Además, incluye disposiciones transitorias para la implementación gradual de los cambios propuestos.

Phising

Rossy Carhuaricra Celis

Este documento describe el phishing o suplantación de identidad en el ciberespacio. Explica que el phishing implica el robo de información confidencial como contraseñas o datos bancarios mediante ingeniería social. Describe las técnicas comunes de phishing como el uso de enlaces manipulados y dominios falsos, y cómo ha evolucionado desde los años 90 cuando se usaba para robar cuentas de AOL. También analiza los intentos recientes de phishing dirigidos a bancos y servicios de pago en línea, así como estrategias

Seguridad para Pymes "Guerra de Guerrillas"

Tuvika Tuvika

Zen y el arte de pescar APT

Antonio Sanz Alcober

Los "malos" instalaron un keylogger en el equipo del director de marketing y usaron este para robar credenciales de OWA y VPN, así como documentos confidenciales de Sharepoint. Aunque se realizó un análisis forense completo, no pudieron demostrarse las identidades de los atacantes debido a las técnicas avanzadas utilizadas para ocultar sus huellas. La lección aprendida es la necesidad de sistemas de detección de anomalías y correlación avanzados.

Administracion electronica: Tecnologías al servicio del ciudadano

Antonio Sanz Alcober

Este documento describe cómo las tecnologías electrónicas pueden usarse para mejorar la administración pública y los servicios al ciudadano. Explica diferentes tecnologías como la nube, dispositivos móviles y herramientas de Google. También presenta casos de éxito de administraciones que han adoptado estas tecnologías. Finalmente, discute la importancia de la interoperabilidad y la seguridad al implementar soluciones de administración electrónica.

Propuestas para mejorar la calidad de vida de los enfermos terminales

EIDEC

Este documento presenta propuestas para mejorar la calidad de vida de los pacientes terminales a través de los cuidados paliativos en 6 hospitales de la Ciudad de Buenos Aires. Incluye un marco teórico sobre los cuidados paliativos y la legislación existente, y analiza la situación actual en cada hospital a través de entrevistas con directores y referentes, abordando aspectos como la concepción, estructura, funcionamiento y necesidad de mejora de los dispositivos de cuidados paliativos.

Seguridad de la banca online - Consejos para usuarios

Antonio Sanz Alcober

Ciberguerra y terrorismo digital

Hacking Bolivia

(H)ac(k)tivismo, Ciberterrorismo y Ciberguerra

Joel A. Gómez Treviño

El documento habla sobre delitos informáticos. Explica que un delito informático es una actividad ilícita que usa computadoras o sistemas informáticos como medio o tiene como objetivo causar daños a sistemas. También describe diferentes tipos de delincuentes informáticos como hackers, crackers y phreakers, así como diferentes tipos de delitos como robo de identidad, fraude electrónico y ataques cibernéticos. Finalmente, clasifica los delitos informáticos por su motivación, ya sea entretenimiento, lucro o caus

Pablo González & Juan Antonio Calles – Cyberwar: Looking for… touchdown! [Roo...

RootedCON

Este documento presenta una conferencia sobre ciberguerra y ciberamenazas. La conferencia discute temas como el estado actual de las ciberamenazas, los componentes de la ciberguerra como el ciberespionaje, la ciberdefensa y los ciberataques, la tecnología y los costos involucrados, y cómo los ciudadanos pueden convertirse en cibersoldados. También incluye ejemplos de ciberataques recientes y escenarios sobre cómo se llevan a cabo el ciberespionaje y los ciberataques.

Triagedeurgencias MÉDICA Y ENFERMERÍA

Anelly May

El documento presenta una guía para implementar el método de triage en las salas de urgencias de los hospitales. El triage es un método de clasificación y priorización de pacientes basado en sus requerimientos y la disponibilidad de recursos. La guía describe los objetivos, procesos, criterios y niveles de prioridad del triage, con el fin de mejorar la atención de pacientes y optimizar los recursos en salas de urgencias.

condor_user_tutorial.ppt

vlaaade

Frieda has 600 simulations to run that will each take 6 hours. She learns about Condor from colleagues and installs a "personal Condor" on her workstation. This allows Condor to manage her 600 jobs and run them across available resources over time. The document outlines how Frieda organizes her files and directories, writes a submit description file to describe the jobs to Condor, and submits the jobs to her personal Condor pool.

Basics_of_Kernel_Panic_Hang_and_ Kdump.pdf

stroganovboris

"A rootkits writer’s guide to defense" - Michal Purzynski

PROIDEA

Michal will take you on a journey all the way to 90’s and back, sharing the Mozilla detection framework - a systematic way to detect and hunt down threat actors. Why did we spend hours digging through some old Phrack issues? How does a blue team's member approach writing rootkits? What is better - a fail negative or a false positive? I will share answers to these questions plus a lot of alerting and evil-doing code.

Solaris Kernel Debugging V1.0

Jarod Wang

This document provides an overview of kernel debugging on Solaris systems using the modular debugger Mdb and dynamic tracing framework DTrace. It discusses debugging live kernels with Mdb, analyzing system crash dumps with Mdb, and using DTrace to monitor the kernel at runtime by enabling probes published by different providers. The document outlines the key tools, techniques, and challenges involved in kernel debugging and crash analysis on Solaris.

Viewers also liked

Print™ Sample Analysis

Sociagility

Domain driven design in a nutshell

Toni Esteves

Informatica forense

mausmr

Por qué un Pecha Kucha es mejor que un Iphone (y casi tanto como el chocolate)

Antonio Sanz Alcober

Manual oms[1]

Elizza Quezzad

La ciber guerra en las relaciones internacionales

carinna_07

Capacidades de China para la ciberguerra

Antonio Sanz Alcober

07 decreto ap.julio 2007.joaquin

mariajoseadamuzgracia

Phising

Rossy Carhuaricra Celis

Seguridad para Pymes "Guerra de Guerrillas"

Tuvika Tuvika

Zen y el arte de pescar APT

Antonio Sanz Alcober

Administracion electronica: Tecnologías al servicio del ciudadano

Antonio Sanz Alcober

Propuestas para mejorar la calidad de vida de los enfermos terminales

EIDEC

Seguridad de la banca online - Consejos para usuarios

Antonio Sanz Alcober

Ciberguerra y terrorismo digital

Hacking Bolivia

(H)ac(k)tivismo, Ciberterrorismo y Ciberguerra

Joel A. Gómez Treviño

Pablo González & Juan Antonio Calles – Cyberwar: Looking for… touchdown! [Roo...

RootedCON

Triagedeurgencias MÉDICA Y ENFERMERÍA

Anelly May

Viewers also liked (18)

Print™ Sample Analysis

Domain driven design in a nutshell

Informatica forense

Por qué un Pecha Kucha es mejor que un Iphone (y casi tanto como el chocolate)

Manual oms[1]

La ciber guerra en las relaciones internacionales

Capacidades de China para la ciberguerra

07 decreto ap.julio 2007.joaquin

Phising

Seguridad para Pymes "Guerra de Guerrillas"

Zen y el arte de pescar APT

Administracion electronica: Tecnologías al servicio del ciudadano

Propuestas para mejorar la calidad de vida de los enfermos terminales

Seguridad de la banca online - Consejos para usuarios

Ciberguerra y terrorismo digital

(H)ac(k)tivismo, Ciberterrorismo y Ciberguerra

Pablo González & Juan Antonio Calles – Cyberwar: Looking for… touchdown! [Roo...

Triagedeurgencias MÉDICA Y ENFERMERÍA

Similar to Solving BIG problems with Open Source: Condor

condor_user_tutorial.ppt

vlaaade

Basics_of_Kernel_Panic_Hang_and_ Kdump.pdf

stroganovboris

"A rootkits writer’s guide to defense" - Michal Purzynski

PROIDEA

Solaris Kernel Debugging V1.0

Jarod Wang

GOD MODE Unlocked: Hardware backdoors in x86 CPUs

Priyanka Aash

Complexity is increasing. Trust eroding. In the wake of Spectre and Meltdown, when it seems that things cannot get any darker for processor security, the last light goes out. This talk will demonstrate what everyone has long feared but never proven: there are hardware backdoors in x86 processors, and they’re buried deeper than we ever imagined possible. In this talk, we walk through how we discovered a privilege escalation backdoor in a family of x86 CPUs, that allows an unprivileged user, on an unmodified system, to circumvent all processor security checks and escalate from ring 3 to ring 0 – permitting an unprivileged, arbitrary userland program to directly modify and execute code inside of the kernel, regardless of the operating system, security patches, antivirus, firmware, etc. Speakers: Christopher Domas, Cyber Security Researcher

100 bugs in Open Source C/C++ projects

Andrey Karpov

Ako využiť (ukradnúť) kolegove CPU cykly

cloudcampsk

This document discusses Condor, a high throughput computing system developed at the University of Wisconsin-Madison since 1988. It describes Condor's capabilities including job management, prioritization of resources, access controls, and support for Linux, Windows, and other operating systems. An example is provided of configuring and submitting a Condor job using tools like condor_q and condor_submit.

2011 ecoop

bergel

The document discusses using message counting as a profiling technique in Pharo Smalltalk. It shows that counting messages is more stable than traditional execution sampling profiling, as message counts are not impacted by variations in execution environment. An experiment showed a strong correlation between number of messages and average execution time. Counting messages can be used to identify performance bottlenecks and compare performance across versions.

Intro to Arduino Programming.pdf

HimanshuDon1

This document provides an introduction and overview of Arduino and C programming. It covers the following key points: 1. The presentation covers assumptions about the audience's programming experience and goals of being able to create Arduino applications using shields, libraries, control flows, functions, reading connection diagrams, and debugging. 2. The agenda includes introductions to the Arduino hardware, IDE, variables, control flow, loops, arrays, strings, functions, libraries, I/O, connection diagrams, and EEPROM. 3. The Arduino is a microcontroller that can be programmed to read sensors, run motors and lights, and send/receive data via various shields and accessories. The Arduino IDE is

Making Security Invisible

J On The Beach

This talk will cover the problems currently with why applications are not being sandboxed to lessen the attack surface. Mostly this is based upon the existing tools being not user friendly and requiring a low level knowledge of syscalls that is hard to find in application developers. Seccomp is one of these tools. It defines syscall filters that allow an application to define what syscalls it allows or denies. It is commonly used in the highly-regarded Chrome sandbox. Integrating things like seccomp filters into programming languages at build time could allow for creating a perfect set of filters based off the application code. In practice, some try to mock this behavior at runtime but it often fails due to certain functions not being called during testing and missing specific syscalls. Therefore causing the user to turn it off completely. By integrating it into the code at build time we can ensure that all the syscalls are accounted for. This talk will also show a proof of concept with this in Golang.

Exploring Boto3 Events With Mitmproxy

Michael Twomey

Slides from my talk at AWS ComSum 2022 in Manchester. If you are writing code in Python which communicates with AWS APIs you are more than likely using the boto3 library. boto3 isn't extended via inheritance or callbacks, instead it offers an event system which allows you to intercept and modify calls at different stages in their lifecycle. This is a very powerful mechanism but it can be hard to write code for very rare events. Using a tool like mitmproxy you can rewrite responses boto3 receives and simulate rarer events to aid writing code to handle them. In this talk I'll show: - How to write basic code to listen to events - An example of some events you might see in typical S3 requests - How to use mitmproxy to intercept, understand and ultimately rewrite HTTP requests between boto3 and AWS to simulate different scenarios - An example of the events you'll see during request retries - An example putting this together to insert monitoring into the boto3 retry mechanism to diagnose network issues Full source here: https://github.com/micktwomey/exploring-boto3-events-with-mitmproxy

Lab Manual Combaring Redis with Relational

Amazon Web Services

Ruun commands

Queen Lumanta

Monitoring a program that monitors computer networks

PVS-Studio

The document discusses several types of errors found using static analysis on the NetXMS open source project codebase. It describes 4 examples of 64-bit errors where pointers are incorrectly cast to 32-bit types. It also mentions errors handling unsigned types like sockets, half-cleared buffers due to misunderstanding string sizes, copy-paste errors, uninitialized variables, null pointer dereferences, and incorrect variable type usage with variadic functions. The author encourages the NetXMS developers to use the static analysis tool to find and address these kinds of issues in their code.

PythonBrasil[8] - CPython for dummies

Tatiana Al-Chueyr

This document provides an overview of how to contribute to the cPython source code. It discusses running benchmarks to understand performance differences between loops inside and outside functions. It encourages contributing to improve coding skills and help the open source community. The steps outlined are to clone the cPython source code repository, resolve any dependencies during building, review open issues on bugs.python.org, and work on resolving issues - starting with easier ones. Tips are provided such as commenting when taking ownership of an issue, reproducing bugs before working on them, writing tests for code changes, and updating documentation.

What the CRaC - Superfast JVM startup

Gerrit Grunwald

The document discusses Coordinated Restore at Checkpoint (CRaC), a feature of the Java Virtual Machine (JVM) that allows saving the state of a running application and restoring it later to avoid JVM startup overhead. CRaC uses the CRIU userspace checkpoint/restore mechanism and provides a simple API for applications to register resources that need to be notified during checkpoint and restore. This allows restoring application state like open files and sockets. An example demonstrates how CRaC can speed up subsequent runs of an application by restoring a pre-filled cache from a previous checkpoint.

Skiron - Experiments in CPU Design in D

Mithun Hunsur

This document discusses Skiron, an experimental CPU design project implemented in the D programming language. It provides an overview of Skiron, which simulates a RISC-inspired instruction set architecture. It describes the idioms and patterns used in D to define the instruction set and encoding in a way that is self-documenting and allows different parts of the software to stay in sync. It also discusses lessons learned, such as issues with delegates, as well as potential improvements to D's metaprogramming capabilities and standard library support for @nogc code. Realizing Skiron in hardware with an FPGA and making it self-hosting are presented as future goals.

Stealthy, Hypervisor-based Malware Analysis

Tamas K Lengyel

Tamas K Lengyel gave a presentation on stealthy malware analysis using a hypervisor. He discussed how malware uses various techniques to detect sandboxes and analysis tools. Using a hypervisor for monitoring provides greater visibility but malware can still detect virtualization. Various techniques were presented for improving stealth, such as using memory sharing, emulating system properties, and stalling analysis through syscall spamming. Overall, an arms race exists between analysis tools becoming more stealthy and malware improving detection techniques.

WWCode Dallas - Kubernetes: Learning from Zero to Production

Rosemary Wang

The document discusses various Kubernetes concepts and tools including: - Using Minikube to deploy a local Kubernetes cluster for learning. - Using kops to deploy a Kubernetes cluster on cloud infrastructure like AWS. - Key Kubernetes objects like pods, deployments, services, ingress controllers, daemonsets, statefulsets and jobs. - Cluster operations such as logging/metrics, autoscaling, upgrades, backups and testing. - Security practices including secrets management, vulnerability scanning and network policies.

Hands-on Lab - Combaring Redis with Relational

Amazon Web Services

by Jeff Duffy, Database Specialist Solution Architect, AWS Database Week at the AWS Loft is an opportunity to learn about Amazon’s broad and deep family of managed database services. These services provide easy, scalable, reliable, and cost-effective ways to manage your data in the cloud. We explain the fundamentals and take a technical deep dive into Amazon RDS and Amazon Aurora relational databases, Amazon DynamoDB non-relational databases, Amazon Neptune graph databases, and Amazon ElastiCache managed Redis, along with options for database migration, caching, search and more. You'll will learn how to get started, how to support applications, and how to scale.

More from Antonio Sanz Alcober

Ciberamenazas - ¿A qué nos enfrentamos?

Antonio Sanz Alcober

We are increasingly dependent on information technology that leaves us vulnerable to cyber threats from actors seeking to influence events through violence, steal trade secrets and intellectual property, or compromise critical infrastructures like SCADA systems that control utilities. While attribution is difficult, states have been implicated in incidents like Stuxnet against Iran and cyberattacks between Georgia and Russia. Individuals and organizations can help manage these risks by not becoming an easy target, obtaining threat intelligence, assuming compromise will occur, and responding appropriately to incidents.

China y el ciberespionaje: Sun Tzu, APT1 y los tiempos interesantes.

Antonio Sanz Alcober

China lleva más de una década incrementando su presencia en Internet, siendo una de las potencias más activas en ciberataques y estando bajo sospecha de algunas de las APT más sonadas. Pero para conocer los objetivos de China es necesario realizar un análisis global, que más allá de la tecnología tenga en cuenta su particular entorno histórico, social y político. El propósito de la charla es conformar la visión que China tiene como potencia mundial y su aplicación en el dominio del “quinto espacio”. Gracias a este análisis podremos analizar correctamente el pasado de los ciberataques, sabremos detectar mejor los ataques presentes y estaremos mejor preparados para futuros vectores de ataque.

Cyberthreats: Are we screwed ?

Antonio Sanz Alcober

SEA - Syrian Electronic Army - Cyberterrorism

Antonio Sanz Alcober

The SEA (Syrian Electronic Army) is a group of Syrian computer hackers aligned with the Syrian regime. They use cyber attacks like DDoS and hacking to target political opposition and Western websites. While not directly part of the Syrian government, they share similar political agendas of fighting those who destabilize Syria. Notable attacks include hacking the Associated Press Twitter in 2013 and falsely reporting that the White House was bombed, and hacking advertising service Outbrain to place pro-Assad posts on the Washington Post, Time, and CNN. The SEA meets the definition of terrorism by using violence and intimidation for political aims, and cyberterrorism by using technology to cause disruption and fear. They are an organized group that

Sex lies and queues: Confessions of an HPC sysadmin

Antonio Sanz Alcober

This document provides advice for system administrators (sysadmins) working in high-performance computing (HPC) environments. It discusses managing systems, which includes tidying the datacenter, prioritizing tasks, planning in advance, monitoring everything, doing backups with a sound strategy, making systems uniform, and automating as much as possible. It also covers managing people, such as educating and deploying ticketing systems for users, managing bosses by knowing their priorities and getting documentation, and using "Zen" and setting boundaries when managing yourself. Additionally, the document discusses managing projects through setting expectations, thinking long-term, avoiding vendor lock-in, and being nice to suppliers to a point. The overall message is that an H

Cómo comprar seguro en internet

Antonio Sanz Alcober

Apple compra Microsoft : Respuesta ante incidentes de seguridad en redes soci...

Antonio Sanz Alcober

Harry el Sucio te enseña Seguridad en Wordpress

Antonio Sanz Alcober

Sex, lies and magnetic tapes: Confessions of a sysadmin

Antonio Sanz Alcober

This document discusses the role and responsibilities of a system administrator (sysadmin). It is presented as confessions from a sysadmin based on their 10+ years of experience. The summary discusses managing systems, people, and projects as the key responsibilities of a sysadmin. Sysadmins must prioritize tasks, have backup and automation strategies, and learn to manage users, bosses, and consultants. The document emphasizes the importance of planning, documentation, and being able to say "no." It also notes that while the job can be tough, sysadmins have some perks and must find a way to manage themselves.

Supercomputación: Haciendo HOY la ciencia del mañana

Antonio Sanz Alcober

Historia de la inteligencia china

Antonio Sanz Alcober

Este documento proporciona una historia de la inteligencia china desde sus orígenes hasta el siglo XXI. Comienza describiendo los primeros espías en la antigua China y la importancia de la inteligencia durante los períodos de los Estados en Guerra, incluyendo la figura de Sun Tzu. Luego resume los principales hitos de la inteligencia china a lo largo de las dinastías imperiales y durante los cambios políticos del siglo XX, como la guerra contra Japón y la revolución cultural. Finalmente, analiza el desarrollo de la int

Análisis de la capacidad científica, tecnológica e innovadora de la república...

Antonio Sanz Alcober

Seguridad en redes sociales

Antonio Sanz Alcober

Cloud computing security

Antonio Sanz Alcober

Cloud computing provides opportunities for scalability, availability, and performance but also poses risks if not implemented securely. Key risks include vendor lock-in, lack of governance and control, non-compliance, and various technical risks around isolation faults, data leaks, network attacks, and provider compromises. Migrating to the cloud requires carefully analyzing requirements, evaluating cloud models and providers, defining security controls, and having business continuity plans to mitigate risks. With proper planning and risk mitigation, cloud computing can be implemented securely.

Construyendo un cluster de supercomputación - Guía paso a paso

Antonio Sanz Alcober

More from Antonio Sanz Alcober (15)

Ciberamenazas - ¿A qué nos enfrentamos?

China y el ciberespionaje: Sun Tzu, APT1 y los tiempos interesantes.

Cyberthreats: Are we screwed ?

SEA - Syrian Electronic Army - Cyberterrorism

Sex lies and queues: Confessions of an HPC sysadmin

Cómo comprar seguro en internet

Apple compra Microsoft : Respuesta ante incidentes de seguridad en redes soci...

Harry el Sucio te enseña Seguridad en Wordpress

Sex, lies and magnetic tapes: Confessions of a sysadmin

Supercomputación: Haciendo HOY la ciencia del mañana

Historia de la inteligencia china

Análisis de la capacidad científica, tecnológica e innovadora de la república...

Seguridad en redes sociales

Cloud computing security

Construyendo un cluster de supercomputación - Guía paso a paso

Recently uploaded

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Aggregage

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Neo4j

Dr. Sean Tan, Head of Data Science, Changi Airport Group Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...

Neo4j

Leonard Jayamohan, Partner & Generative AI Lead, Deloitte This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

Vladimir Iglovikov, Ph.D.

Presented by Vladimir Iglovikov: - https://www.linkedin.com/in/iglovikov/ - https://x.com/viglovikov - https://www.instagram.com/ternaus/ This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation. Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners. This case study covers various aspects, including: People: The contributors and community that have supported Albumentations. Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions. Challenges: The hurdles in monetizing open-source projects and measuring user engagement. Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration. Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community. Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations. Mental Health: Maintaining balance and not feeling pressured by user demands. Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth. Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects. Explore more about Albumentations and join the community at: GitHub: https://github.com/albumentations-team/albumentations Website: https://albumentations.ai/ LinkedIn: https://www.linkedin.com/company/100504475 Twitter: https://x.com/albumentations

20240609 QFM020 Irresponsible AI Reading List May 2024

Matthew Sinclair

Communications Mining Series - Zero to Hero - Session 1

DianaGray10

This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered: • Communication Mining Overview • Why is it important? • How can it help today’s business and the benefits • Phases in Communication Mining • Demo on Platform overview • Q/A

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Albert Hoitingh

Full-RAG: A modern architecture for hyper-personalization

Zilliz

Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.

UiPath Test Automation using UiPath Test Suite series, part 5

DianaGray10

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Essentials of Automations: The Art of Triggers and Actions in FME

Safe Software

In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation. We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios. Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Neo4j

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Data structures and Algorithms in Python.pdf

TIPNGVN2

Large Language Model (LLM) and it’s Geospatial Applications

Rohit Gautam

How to Get CNIC Information System with Paksim Ga.pptx

danishmna97

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

20240607 QFM018 Elixir Reading List May 2024

Matthew Sinclair

Recently uploaded (20)

Generative AI Deep Dive: Advancing from Proof of Concept to Production

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

20240609 QFM020 Irresponsible AI Reading List May 2024

Communications Mining Series - Zero to Hero - Session 1

Securing your Kubernetes cluster_ a step-by-step guide to success !

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Full-RAG: A modern architecture for hyper-personalization

UiPath Test Automation using UiPath Test Suite series, part 5

PCI PIN Basics Webinar from the Controlcase Team

Essentials of Automations: The Art of Triggers and Actions in FME

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Data structures and Algorithms in Python.pdf

Large Language Model (LLM) and it’s Geospatial Applications

How to Get CNIC Information System with Paksim Ga.pptx

Removing Uninteresting Bytes in Software Fuzzing

20240607 QFM018 Elixir Reading List May 2024

Solving BIG problems with Open Source: Condor

1. > Solving Big problems with OS: Condor > Antonio Sanz (ansanz@unizar.es) > 09 / Nov / 11

2. 2

3. 3 > Antonio Sanz > I3A System Manager > HERMES HPC cluster sysadmin > ansanz@unizar.es > @antoniosanzalc

4. 4

5. 5 Show / Know / Use

6. Problema inicial 6 > Dr. Good > Neurologist > Alzheimer research > Process 20000 brain image scans (1h/image) > A thousand times. Maybe two. 3. Sistemas de gestión de colas : Condor

7. Problema inicial 7 > Mrs. Nice > Santa’s Logistic Officer > Gift transportation > Analize 6x10e7 possible load/reindeers/routes (10min/analysis) > Before Christmas!

8. 8 Hey … ! It’s a 64K one !

9. 9 Queue distribution systems

10. Condor Basics 10 Single queue 3. Sistemas de gestión de colas : Condor

11. 11

12. Condor Basics 12 Multiple queues 3. Sistemas de gestión de colas : Condor

13. 13

14. 14 Problem partitioning

15. Problem can be broken into independent pieces 15

16. Condor Basics 16 Oh Yeah!

17. Condor Basics 17 For loops are your best friends 3. Sistemas de gestión de colas : Condor

18. Condor Basics 18 While loops … can sometimes be convinced 3. Sistemas de gestión de colas : Condor

19. Condor Basics 19 Do it yourself ! 3. Sistemas de gestión de colas : Condor

20. Condor Basics 20 >

21. Condor Basics 21 Heterogeneous computing

22. 22 Resource harvesting

23. 23 Requirements

24. 24 Job Surveillance

25. Condor Basics 25 Fair use of resources 3. Sistemas de gestión de colas : Condor

26. 26 Checkpoints

27. Condor Basics 27 Nested jobs (DAG)

28. Condor Basics 28 Email Notifications

29. 29 Grid & Cloud Computing

30. Condor Basics 30 Flexibility

31. 31 … with Hadoop, MPI, OpenMP, GPU

32. Condor Basics 32 3. Sistemas de gestión de colas : Condor

33. 33 How Condor works

34. 34 Management [Hello, Dave]

35. 35 Compute

36. Condor Basics 36 Job list ClassAd 3. Sistemas de gestión de colas : Condor

37. 37 Resource list ClassAd

38. 38 Matchmaking

39. Condor Basics 39 Priority Management

40. 40 Data Transfer

41. Condor Basics 41 Job running 3. Sistemas de gestión de colas : Condor

42. 42 Job Monitoring

43. 43 Job End

44. 44 Example

45. 45 #!/bin/sh # I’m hola.sh echo Hola mundo desde `hostname` # # A Hello World .. In Condor! # # I’m hello.sub Universe = vanilla Executable = hola.sh Log = hola.log Output = hola.out Error = hola.err Queue Hello, World !!

46. Lanzar el cálculo 46 condor_submit 4. Condor Basics – Un cálculo fácil

47. Lanzar el cálculo 47 condor_q

48. 48 Something tastier… #!/bin/sh # I’m hello2.sh OUTPUT=hello${1}.result cat hello.input >> $OUTPUT cat echo Hello world, I’m job $1 here from `hostname` > $OUTPUT # Execute n times with different outputs Universe = vanilla Executable = hello2.sh Transfer_input_files = hello.input WhenToTransferOutput = ON_EXIT_OR_EVICT Arguments = $(Process) Log = hello.log Output = hello.out Queue 10

49. 49 Perfect Simulation 4. Condor Basics – Un cálculo fácil

50. 50 Extra Bonus

51. 51 Dynamic Partitioning

52. Condor Basics 52 Configurable Jobs

53. 53 Advanced Accounting

54. 54 Dynamic Checkpointing

55. Condor Basics 55 Hadoop Integration 3. Sistemas de gestión de colas : Condor

56. 56 Green Computing

57. Condor Basics 57 GPU Integration

58. 58 I3A & Condor

59. 59 Gaming IA

60. 60 MRI Brain Analysis