- GeoScript is a library for processing spatial data in various scripting languages like Python, Groovy, JavaScript, etc. It uses GeoTools behind the scenes.
- It provides familiar spatial concepts like geometry, features, layers, and projections. Scripts can load, style, analyze, and display spatial data with just a few lines of code.
- Processes written with GeoScript can be published as OGC Web Processing Services and executed remotely through a standard interface. This allows sharing and chaining of spatial analysis workflows.
Slides from the Kubernetes Ahmedabad Kick-Off Meetup. Since it was the first meetup, we discussed briefly about the basics of Kubernetes followed by a demo on Minikube.
Slides from the Kubernetes Ahmedabad Kick-Off Meetup. Since it was the first meetup, we discussed briefly about the basics of Kubernetes followed by a demo on Minikube.
This presentation talks about how to use GlusterFS containers in Docker. If you need more details please refer http://www.humblec.com/building-glusterfs-in-a-docker-container/
Demo Video # https://youtu.be/4Xf8pmDEZYw
CloudZone's Meetup at Google offices, 20.08.2018
Covering Google Cloud Platform Kubernetes Engine in Depth, including networking, compute, storage, monitoring & logging
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...OpenNebula Project
In this talk we'll showcase the new support for LXD containers in OpenNebula. The talk will describe the basic functionality of the new drivers and will provide some hints on the integration internals. LXD support will be released in OpenNebula 5.8 and it will let you manage LXD containers in your cloud using the same interfaces as with VMs, leveraging all the OpenNebula ecosystem and functionality including: Marketplace, multi-tenancy or service composition with OpenNebula Flow.
OSBConf 2015 | Scale out backups with bareos and gluster by niels de vosNETWAYS
During this talk, Niels will explain the basics of Gluster and show how Bareos integrates with it. Gluster provides a Software Defined Storage environment that can scale-out when the backup storage needs to grow. With a live demonstration Niels shows how simple it is to setup a small Gluster environment and configure Bareos to use the native Gluster protocol.
Workshop on Funding Collaborative Research during Indian Technology Congress - 2013. On July 24th and 25th, 2013 @ Nimhans Convention Centre, Bangalore, India.
This presentation talks about how to use GlusterFS containers in Docker. If you need more details please refer http://www.humblec.com/building-glusterfs-in-a-docker-container/
Demo Video # https://youtu.be/4Xf8pmDEZYw
CloudZone's Meetup at Google offices, 20.08.2018
Covering Google Cloud Platform Kubernetes Engine in Depth, including networking, compute, storage, monitoring & logging
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...OpenNebula Project
In this talk we'll showcase the new support for LXD containers in OpenNebula. The talk will describe the basic functionality of the new drivers and will provide some hints on the integration internals. LXD support will be released in OpenNebula 5.8 and it will let you manage LXD containers in your cloud using the same interfaces as with VMs, leveraging all the OpenNebula ecosystem and functionality including: Marketplace, multi-tenancy or service composition with OpenNebula Flow.
OSBConf 2015 | Scale out backups with bareos and gluster by niels de vosNETWAYS
During this talk, Niels will explain the basics of Gluster and show how Bareos integrates with it. Gluster provides a Software Defined Storage environment that can scale-out when the backup storage needs to grow. With a live demonstration Niels shows how simple it is to setup a small Gluster environment and configure Bareos to use the native Gluster protocol.
Workshop on Funding Collaborative Research during Indian Technology Congress - 2013. On July 24th and 25th, 2013 @ Nimhans Convention Centre, Bangalore, India.
Βαθμός αποδοχής της Υπηρεσίας VoIP από εκπαιδευτικούς και σχολικές μονάδες Πρ...Konstantinos Dragogiannis
Η υπηρεσία τηλεφωνίας μέσω διαδικτύου (Voice over Internet Protocol, VoIP) προσφέρει τη δυνατότητα φωνητικής συνομιλίας σε πραγματικό χρόνο, με μηδενικό κόστος, για όποιον διαθέτει σύνδεση στο διαδίκτυο. Η υπηρεσία αυτή παρέχεται και στους πιστοποιημένους χρήστες του Πανελλήνιου Σχολικού Δικτύου. Σκοπός της παρούσας μελέτης είναι η διερεύνηση της υπάρχουσας κατάστασης όσον αφορά τη δημιουργία και τη λειτουργία της συγκεκριμένης υπηρεσίας κυρίως από σχολικές μονάδες Πρωτοβάθμιας και Δευτεροβάθμιας Εκπαίδευσης, αλλά και από εκπαιδευτικούς.
Στην παρούσα έρευνα πραγματοποιήθηκε συλλογή δεδομένων για ποσοτική ανάλυση. Από την ανάλυση αυτή προέκυψε ότι, μέχρι τα τέλη του Δεκεμβρίου 2014, υπήρχαν εξακόσιοι έξι (606) ενεργοποιημένοι λογαριασμοί VoIP του Πανελλήνιου Σχολικού Δικτύου. Η πλειοψηφία των λογαριασμών αυτών ανήκουν σε προσωπικούς λογαριασμούς εκπαιδευτικών και μάλιστα το μεγαλύτερο ποσοστό ανήκει σε άνδρες εκπαιδευτικούς. Αξιοσημείωτη τέλος είναι η υπεροχή των ενεργοποιημένων λογαριασμών των σχολικών μονάδων Πρωτοβάθμιας Εκπαίδευσης έναντι αυτών της Δευτεροβάθμιας.
Παρελθόν και μέλλον υπηρεσίας της Ηλεκτρονικής Σχολικής Τάξης του Πανελλήνιου...Konstantinos Dragogiannis
Η υπηρεσία της Ηλεκτρονικής Σχολικής Τάξης (η-τάξη) του Πανελλήνιου Σχολικού Δικτύου (Π.Σ.Δ.) από το σχολικό έτος 2006-2007 μέχρι και το σχολικό έτος 2014-2015 χρησιμοποιήθηκε, κατά μέσο όρο, από 1061 σχολικές μονάδες υποστηρίζοντας, κατά μέσο όρο, 3979 ηλεκτρονικά μαθήματα. Στην παρούσα εργασία καταγράφονται αφενός ο αριθμός των ηλεκτρονικών μαθημάτων και αφετέρου ο αριθμός των σχολικών μονάδων που διαθέτουν ηλεκτρονικά μαθήματα και γίνεται μία προσπάθεια ερμηνείας των αιτιών της επικρατούσας κατάστασης αλλά και του μέλλοντος της ηλεκτρονικής αυτής υπηρεσίας.
Από την διενεργηθείσα έρευνα προέκυψε ότι ο αριθμός των ηλεκτρονικών μαθημάτων τα τέσσερα πρώτα σχολικά έτη παρέμεινε σχεδόν σταθερός, στη συνέχεια παρουσίασε μία σημαντική άνοδο, ενώ τα τελευταία τρία χρόνια παρέμεινε και πάλι σχεδόν σταθερός. Η μείωση του ρυθμού αύξησης τα τελευταία τρία σχολικά έτη ενδεχομένως οφείλεται στην αναβάθμιση της υπηρεσίας η-τάξη, η οποία μεταξύ άλλων περιελάβανε και την αύξηση των εργαλείων από έξι (6) σε δεκαεπτά (17). Στην περίπτωση που η υπόθεση αυτή ισχύει, τότε και κατά το τρέχον σχολικό έτος δεν αναμένεται αύξηση του αριθμού των ηλεκτρονικών μαθημάτων, δεδομένου ότι έχει γίνει νέα αναβάθμιση στην υπηρεσία, εκτός και αν εντατικοποιηθούν οι προσπάθειες επιμόρφωσης των εκπαιδευτικών, ιδίως της Πρωτοβάθμιας Εκπαίδευσης, στην ενλόγω υπηρεσία του Π.Σ.Δ.
The Indian Technology Congress 2013 is a Platform bringing together Technologists, Entrepreneurs, Academicians, Investors and Policy Makers. Tech Congress provides an opportunity to Network, Collaborate and Innovate for Technology enthusiasts. Leading Technologists and Entrepreneurs from India and abroad will be sharing their thoughts and vision towards making India a Hub of Knowledge and Innovation.
Indian Technology Congress is being made possible by coming together of various Professional Bodies joining hands with Institution of Engineers India (IEI) and National Design Research Forum (NDRF). The Institutions and Organizations include, Institute of Electrical and Electronics Engineers (IEEE), Institution of Engineering and Technology (IET), All India Manufacturing Organizations (AIMO), Indian Institution of Production Engineers (IIPE). Micro Small and Medium Enterprises (MSME) Department, Government of India is supporting the initiative.
Αποτελεί –σε παγκόσμια κλίμακα– τάση στη σύγχρονη Τριτοβάθμια Εκπαίδευση η δημιουργία και λειτουργία, από οικεία Ιδρύματα, Συστημάτων Διαχείρισης Μάθησης, δεδομένου ότι οι σπουδαστές που συμμετέχουν σε προγράμματα μικτής διδασκαλίας (blended learning) επιτυγχάνουν τα ίδια ή και καλύτερα μαθησιακά αποτελέσματα. Δείκτες επιτυχίας των συστημάτων αυτών, σύμφωνα με το μοντέλο αποδοχής τεχνολογίας (Technology Acceptance Model – TAM), αποτελούν η αντιληπτή ευκολία χρήσης (Perceived Ease of Use), καθώς και η αντιληπτή χρησιμότητα (Perceived Usefulness) και σύμφωνα με το μοντέλο συνέχισης πληροφοριακών συστημάτων (Expectation-Confirmation Model of IS Continuance) η ικανοποίηση (Satisfaction), η επιβεβαίωση των προσδοκιών (Confirmation) και η πρόθεση συνέχισης χρήσης (Information System Continuance Intention).
Η παρούσα εργασία αποσκοπεί αφενός στην προσαρμογή των δομών του μοντέλου αποδοχής τεχνολογίας (α) αντιληπτή ευκολία χρήσης και (β) αντιληπτή χρησιμότητα κα-θώς και των δομών του μοντέλου συνέχισης πληροφοριακών συστημάτων (γ) ικανοποίηση, (δ) επιβεβαίωση προσδοκιών και (ε) πρόθεση συνέχισης χρήσης για Συστήματα Διαχείρισης Μάθησης και αφετέρου στην εφαρμογή των τροποποιημένων αυτών δομών στα πλαίσια μίας μελέτης περίπτωσης για τον καθορισμό της επιτυχίας ενός Συστήματος Διαχείρισης Μάθησης, μέσω της μέτρησης των πέντε αυτών δομών.
Στην εργασία αυτή πραγματοποιείται μία μελέτη περίπτωσης Συστήματος Διαχείρισης Μάθησης στην Τριτοβάθμια Εκπαίδευση. Συγκεκριμένα, καταγράφηκαν οι απόψεις ογδόντα οκτώ (88) σπουδαστών, σε τεθέν ερωτηματολόγιο επταβάθμιας κλίμακας τύπου Likert, για την αντιληπτή ευκολία χρήσης, την αντιληπτή χρησιμότητα, την ικανοποίηση, την επιβεβαίωση των προσδοκιών και την πρόθεση συνέχιση
The AZSHRM Annual Conference is the largest human resource-focused conference in the state of Arizona. It provides fantastic keynote speakers, great educational sessions, networking opportunities, and exposure to the newest HR products, techniques and services. Open to all HR professionals looking for an affordable, dynamic program that can help you earn re-certification credits. Business owners and managers can learn from top keynote speakers and leading experts nationwide.
This years conference is being held at the Sheraton Wild Horse Pass.
This is already slated to be a sold out conference! Confirm your attendee, sponsorship or exhibitor spot early or be left out!
Keynote Speakers include:
Stuart Crabb (Facebook)
Keith Ferrazzi
James Key Lim
Robert Van Arlen
Stedman Graham
The AZSHRM Annual Conference is the largest human resource-focused conference in the state of Arizona. It provides fantastic keynote speakers, great educational sessions, networking opportunities, and exposure to the newest HR products, techniques and services. Open to all HR professionals looking for an affordable, dynamic program that can help you earn re-certification credits. Business owners and managers can learn from top keynote speakers and leading experts nationwide.
This years conference is being held at the Sheraton Wild Horse Pass.
Full Conference Early Bird Rate $545.00 (Includes Pre-conference day)
Regular Conference Early Bird Rate $365.00 (Does not include Pre-conference day)
(Visit the fees page for more information & rate details)
AZSHRM HOTEL Rate: $115/night + charges & taxes
Extended and embedding: containerd update & project use casesPhil Estes
A talk given at FOSDEM 2020 in the containers devroom on the current status of the CNCF containerd project as well as a dive into the ways users are extending and embedding containerd in other platforms and projects.
As more workloads move to severless-like environments, the importance of properly handling downscaling increases. While recomputing the entire RDD makes sense for dealing with machine failure, if your nodes are more being removed frequently, you can end up in a seemingly loop-like scenario, where you scale down and need to recompute the expensive part of your computation, scale back up, and then need to scale back down again.
Even if you aren’t in a serverless-like environment, preemptable or spot instances can encounter similar issues with large decreases in workers, potentially triggering large recomputes. In this talk, we explore approaches for improving the scale-down experience on open source cluster managers, such as Yarn and Kubernetes-everything from how to schedule jobs to location of blocks and their impact (shuffle and otherwise).
Lightbend Lagom: Microservices Just Rightmircodotta
Microservices architecture are becoming a de-facto industry standard, but are you satisfied with the current state of the art? We are not, as we believe that building microservices today is more challenging than it should be. Lagom is here to take on this challenge. First, Lagom is opinionated and it will take some of the hard decisions for you, guiding you to produce microservices that adheres to the Reactive tenents. Second, Lagom was built from the ground up around you, the developer, to push your productivity to the next level. If you are familiar with the Play Framework's development environment, imagine that but tuned for building microservices; we are sure you are going to love it! Third, Lagom comes with batteries included for deploying in production: going from development to production could not be easier. In this session, you will get an introduction to the Lightbend Lagom framework. There will be code and live demos to show you in practice how it works and what you can do with it, making you fully equipped to build your next microservices with Lightbend Lagom!
Patrick Chanezon and Guillaume Laforge are presenting Google App Engine Java and Gaelyk, the lightweight groovy toolkit on top of the GAE SDK, at the Devoxx conference
LocationTech is an Eclipse Foundation industry working group for location aware technologies. This presentation introduces LocationTech, looks at what it means for our industry and the participating projects.
Libraries: JTS Topology Suite is the rocket science of GIS providing an implementation of Geometry. Mobile Map Tools provides a C++ foundation that is translated into Java and Javascript for maps on iOS, Andriod and WebGL. GeoMesa is a distributed key/value store based on Accumulo. Spatial4j integrates with JTS to provide Geometry on curved surface.
Process: GeoTrellis real-time distributed processing used scala, akka and spark. GeoJinni mixes spatial data/indexing with Hadoop.
Applications: GEOFF offers OpenLayers 3 as a SWT component. GeoGit distributed revision control for feature data. GeoScipt brings spatial data to Groovy, JavaScript, Python and Scala. uDig offers an eclipse based desktop GIS solution.
Attend this presentation if want to know what LocationTech is about, are interested in these projects or curious about what projects will be next.
SF Big Analytics 20191112: How to performance-tune Spark applications in larg...Chester Chen
Uber developed an new Spark ingestion system, Marmaray, for data ingestion from various sources. It’s designed to ingest billions of Kafka messages every 30 minutes. The amount of data handled by the pipeline is of the order hundreds of TBs. Omar details how to tackle such scale and insights into the optimizations techniques. Some key highlights are how to understand bottlenecks in Spark applications, to cache or not to cache your Spark DAG to avoid rereading your input data, how to effectively use accumulators to avoid unnecessary Spark actions, how to inspect your heap and nonheap memory usage across hundreds of executors, how you can change the layout of data to save long-term storage cost, how to effectively use serializers and compression to save network and disk traffic, and how to reduce amortize the cost of your application by multiplexing your jobs, different techniques for reducing memory footprint, runtime, and on-disk usage. CGI was able to significantly (~10%–40%) reduce memory footprint, runtime, and disk usage.
Speaker: Omkar Joshi (Uber)
Omkar Joshi is a senior software engineer on Uber’s Hadoop platform team, where he’s architecting Marmaray. Previously, he led object store and NFS solutions at Hedvig and was an initial contributor to Hadoop’s YARN scheduler.
Oleksandr gives a Lightning talk about GIS integration: Storing and displaying geospatial data using OpenGeo Suite.
I would tell about:
- GIS itself;
- open source software called OpenGeo Suite;
- Geo spatial data representation in Geoserver (shapefiles and postgresql databases with postgis extension);
- Scalable data import using rest api.
Finally i would present the way to serve stored data using Open Layers.
This topic might be useful for Ruby Devs, GIS enthusiasts that are looking for free and powerful tools and other Backend devs.
Running Airflow Workflows as ETL Processes on Hadoopclairvoyantllc
While working with Hadoop, you'll eventually encounter the need to schedule and run workflows to perform various operations like ingesting data or performing ETL. There are a number of tools available to assist you with this type of requirement and one such tool that we at Clairvoyant have been looking to use is Apache Airflow. Apache Airflow is an Apache Incubator project that allows you to programmatically create workflows through a python script. This provides a flexible and effective way to design your workflows with little code and setup. In this talk, we will discuss Apache Airflow and how we at Clairvoyant have utilized it for ETL pipelines on Hadoop.
2. The Big Idea
A library for processing of spatial data
Several implementations for scripting environments
Currently in Python, Groovy, JavaScript, Scala, Ruby
GeoTools under the hood
3. Why GeoScript
Easy to learn
Simple to use
Familiar spatial metaphors
Choice of a scripting language
Versatile products
Evolving, free and open source
5. Spatial Metaphors and API
Geometry Style
Feature Map
Workspace Projection
Filter Function
Layer Process
http://www. geoscript.org
6. Load, style and display data ...
# import libraries
from geoscript.layer import Shapefile
from geoscript.render import draw
from geoscript.style import *
# load data
solar = Shapefile("solar_dni_polygons_fla.shp")
# style and display it
solar.style = Stroke("blue")
draw(solar)
8. Examine data ...
# show structure of the data
solar.schema
# print out a single attribute for each record
for f in states.features():
print f["STATE"]
# aggregate all values of "DEC" attribute
list = [f.get("DEC") for f in solar.features()]
reduce(lambda a,b: a+b, list)
10. Build a Histogram ...
# import
from geoscript.plot import bar
# populate data for X axis
x = [1,2,3,4,5,6,7,8,9,10]
# populate data for Y axis
y = [i * i for i in x]
# draw histogram
bar.xy(zip(x,y)).show()
13. Run a process ...
# import
from geoscript.process import Process
# bind to a process and examine its inputs
p = Process.lookup("vec:Centroid")
p.inputs
# execute process
centroids = p.run(features=solar).get("result")
# draw result
draw(centroids)
16. OGC WPS input output
In a nutshell:
● WPS: Web Processing Service
● web service interface standard
● standardizes metadata, inputs and outputs of a (spatial) process
● defines how a client can execute the process
● implementation: some processing algorithm
OGC WPS [HTTP] Operations:
● GetCapabilities: returns service metadata
● DescribeProcess: returns description of a process, its inputs and outputs
● Execute: returns output of a process
17. WPS How-to
Step 1: Hack GeoScript from command line.
Step 2: Wrap your code in a Process.
Step 3: Copy to GeoServer.
It lives, it runs!
18. WPS Example
origin
http://suite.opengeo.org/docs/processing/scripting/processcreate.html
20. Pushing Ahead
GeoScript Rendering Transformations
● do processing from command line ● add advanced visualizations
● prototype and tweak your process ● author transformations using SLD
● use local and remote resources ● transform datasets in GeoServer
Process Chaining
● script complex processes
● wrap and deploy as a single service
21. Pushing Ahead
Web Mapping App Development Integration
● use OpenGeo Suite SDK ● invoke WPS from another system
● create a starter app automatically ● embed interactive maps into other apps
● customize, test and deploy