The document discusses pkgsrc usage on SmartOS. It provides background on SmartOS and illumos as open source derivatives of Solaris after Oracle made it closed source. It describes how pkgsrc is used in SmartOS zones, including features developed by Joyent like multi-architecture support, SMF integration, and an alternative "SmartOS is Not GNU/Linux" layout. It also covers pkgsrc development at Joyent and their wishlist.
The talk is about operating system virtualization technology known as OpenVZ. This is an effective way of partitioning a Linux machine into multiple isolated Linux containers. All containers are running on top of one single Linux kernel, which results in excellent density, performance and manageability. The talk gives an overall description of OpenVZ building blocks, such as namespaces, cgroups and various resource controllers. A few features, notably live migration and virtual swap, are described in greater details. Results of some performance measurements against VMware, Xen and KVM are given. Finally, we will provide a status update on merging bits and pieces of OpenVZ kernel to upstream Linux kernel, and share our plans for the future.
A quick introduction to OpenVZ, a virtualization platform for Linux that works like FreeBSD jails - it segments a system into different partitions, all running LInux. Each virtual system, container, can run different Linux distributions.
[DockerCon 2019] Hardening Docker daemon with Rootless modeAkihiro Suda
https://dockercon19.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=281879
Docker CE 19.03 is going to support "Rootless mode", which allows running the entire Docker daemon and its dependencies as a non-root user on the host, so as to protect the host from malicious containers in a simple but very strong way. Rootless mode is also attractive for users who cannot get `sudo` permission for installing Docker on shared computing machines. e.g. HPC users. In this talk, Akihiro Suda, the author of the Rootless mode (PR: moby#38050), will explain how users can get started with Rootless mode. He will also explain the implementation details of Rootless mode and planned enhancements such as LDAP integration.
The talk is about operating system virtualization technology known as OpenVZ. This is an effective way of partitioning a Linux machine into multiple isolated Linux containers. All containers are running on top of one single Linux kernel, which results in excellent density, performance and manageability. The talk gives an overall description of OpenVZ building blocks, such as namespaces, cgroups and various resource controllers. A few features, notably live migration and virtual swap, are described in greater details. Results of some performance measurements against VMware, Xen and KVM are given. Finally, we will provide a status update on merging bits and pieces of OpenVZ kernel to upstream Linux kernel, and share our plans for the future.
A quick introduction to OpenVZ, a virtualization platform for Linux that works like FreeBSD jails - it segments a system into different partitions, all running LInux. Each virtual system, container, can run different Linux distributions.
[DockerCon 2019] Hardening Docker daemon with Rootless modeAkihiro Suda
https://dockercon19.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=281879
Docker CE 19.03 is going to support "Rootless mode", which allows running the entire Docker daemon and its dependencies as a non-root user on the host, so as to protect the host from malicious containers in a simple but very strong way. Rootless mode is also attractive for users who cannot get `sudo` permission for installing Docker on shared computing machines. e.g. HPC users. In this talk, Akihiro Suda, the author of the Rootless mode (PR: moby#38050), will explain how users can get started with Rootless mode. He will also explain the implementation details of Rootless mode and planned enhancements such as LDAP integration.
OpenWrt is a Linux distribution for embedded systems that runs on many routers and networking devices today. In this session we'll talk about OpenWrt's origins, architecture and get down to building apps for the platform.
Along the way we will touch on some basic firmware concepts and at last present the final working OpenWrt router and its capabilities.
Anton Lerner, Architect at Sitaro, computer geek, developer and occasional maker.
Sitaro provides total cyber protection for small business and home networks. Sitaro prevents massive scale IoT cyber attacks.
Find out more information in the meetup event page - https://www.meetup.com/Tel-Aviv-Yafo-Linux-Kernel-Meetup/events/245319189/
OpenNebula Conf 2014 | Using Ceph to provide scalable storage for OpenNebula ...NETWAYS
Ceph is a open source distributed storage system which provides object, block and file interfaces. The Ceph block device interface (RBD) and object interface (RGW) are popular building blocks in private cloud deployments, and OpenNebula includes a datastore driver for Ceph.
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBITOpenNebula Project
You will learn what DRBD is, where it came from in its 15 years of existence. How it evolved into a software defined storage solution interesting for users of OpenNebula and why it is very well suited for hyperconverged deployment architectures. The presentation will contain IO performance results and (if time permits) a live demo.
Linux Container Brief for IEEE WG P2302Boden Russell
A brief into to Linux Containers presented to IEEE working group P2302 (InterCloud standards and portability). This deck covers:
- Definitions and motivations for containers
- Container technology stack
- Containers vs Hypervisor VMs
- Cgroups
- Namespaces
- Pivot root vs chroot
- Linux Container image basics
- Linux Container security topics
- Overview of Linux Container tooling functionality
- Thoughts on container portability and runtime configuration
- Container tooling in the industry
- Container gaps
- Sample use cases for traditional VMs
Overall, a bulk of this deck is covered in other material I have posted here. However there are a few new slides in this deck, most notability some thoughts on container portability and runtime config.
GlusterFS Cinder integration presented at GlusterNight Paris event @ Openstac...Deepak Shetty
This was a brief presentation talking about the current state of affairs on the GlusterFS Cinder Integration in the GlusterNight Paris event organised by Red Hat for GlusterFS community members, as part of the Openstack Paris Nov. 2014
Talk held by Javier Fontan at the CentOS Dojo in Paris, August 25th (http://wiki.centos.org/Events/Dojo/Paris2014)
In this talk we talk about OpenNebula from the perspsective of the CentOS, explaining tips and considerations for power users.
Gentoo Linux, or Why in the World You Should Compile EverythingDonnie Berkholz
Gentoo Linux is a special flavor of Linux that can be automatically optimized and customized for just about any application or need. Extreme performance, configurability and a top-notch user and developer community are all hallmarks of the Gentoo experience.
As a leader of Gentoo, I will provide an overview of how it works from a developer's and a user's point of view, and why you should be running it especially if you're:
- In need of an awesome development environment;
- Interested in learning what's inside the black box of Linux;
- OCD about having a perfectly configured setup; or
- Building an embedded, minimal system or a high-performance cluster.
If there's interest, I can also talk about future developments on the horizon for Gentoo, package management in general, etc.
In this talk we will discuss how to build and run containers without root privileges. As part of the discussion, we will introduce new programs like fuse-overlayfs and slirp4netns and explain how it is possible to do this using user namespaces. fuse-overlayfs allows to use the same storage model as "root" containers and use layered images. slirp4netns emulates a TCP/IP stack in userland and allows to use a network namespace from a container and let it access the outside world (with some limitations).
We will also introduce Usernetes, and how to run Kubernetes in an unprivileged user namespace
https://sched.co/Jcgg
OpenWrt is a Linux distribution for embedded systems that runs on many routers and networking devices today. In this session we'll talk about OpenWrt's origins, architecture and get down to building apps for the platform.
Along the way we will touch on some basic firmware concepts and at last present the final working OpenWrt router and its capabilities.
Anton Lerner, Architect at Sitaro, computer geek, developer and occasional maker.
Sitaro provides total cyber protection for small business and home networks. Sitaro prevents massive scale IoT cyber attacks.
Find out more information in the meetup event page - https://www.meetup.com/Tel-Aviv-Yafo-Linux-Kernel-Meetup/events/245319189/
OpenNebula Conf 2014 | Using Ceph to provide scalable storage for OpenNebula ...NETWAYS
Ceph is a open source distributed storage system which provides object, block and file interfaces. The Ceph block device interface (RBD) and object interface (RGW) are popular building blocks in private cloud deployments, and OpenNebula includes a datastore driver for Ceph.
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBITOpenNebula Project
You will learn what DRBD is, where it came from in its 15 years of existence. How it evolved into a software defined storage solution interesting for users of OpenNebula and why it is very well suited for hyperconverged deployment architectures. The presentation will contain IO performance results and (if time permits) a live demo.
Linux Container Brief for IEEE WG P2302Boden Russell
A brief into to Linux Containers presented to IEEE working group P2302 (InterCloud standards and portability). This deck covers:
- Definitions and motivations for containers
- Container technology stack
- Containers vs Hypervisor VMs
- Cgroups
- Namespaces
- Pivot root vs chroot
- Linux Container image basics
- Linux Container security topics
- Overview of Linux Container tooling functionality
- Thoughts on container portability and runtime configuration
- Container tooling in the industry
- Container gaps
- Sample use cases for traditional VMs
Overall, a bulk of this deck is covered in other material I have posted here. However there are a few new slides in this deck, most notability some thoughts on container portability and runtime config.
GlusterFS Cinder integration presented at GlusterNight Paris event @ Openstac...Deepak Shetty
This was a brief presentation talking about the current state of affairs on the GlusterFS Cinder Integration in the GlusterNight Paris event organised by Red Hat for GlusterFS community members, as part of the Openstack Paris Nov. 2014
Talk held by Javier Fontan at the CentOS Dojo in Paris, August 25th (http://wiki.centos.org/Events/Dojo/Paris2014)
In this talk we talk about OpenNebula from the perspsective of the CentOS, explaining tips and considerations for power users.
Gentoo Linux, or Why in the World You Should Compile EverythingDonnie Berkholz
Gentoo Linux is a special flavor of Linux that can be automatically optimized and customized for just about any application or need. Extreme performance, configurability and a top-notch user and developer community are all hallmarks of the Gentoo experience.
As a leader of Gentoo, I will provide an overview of how it works from a developer's and a user's point of view, and why you should be running it especially if you're:
- In need of an awesome development environment;
- Interested in learning what's inside the black box of Linux;
- OCD about having a perfectly configured setup; or
- Building an embedded, minimal system or a high-performance cluster.
If there's interest, I can also talk about future developments on the horizon for Gentoo, package management in general, etc.
In this talk we will discuss how to build and run containers without root privileges. As part of the discussion, we will introduce new programs like fuse-overlayfs and slirp4netns and explain how it is possible to do this using user namespaces. fuse-overlayfs allows to use the same storage model as "root" containers and use layered images. slirp4netns emulates a TCP/IP stack in userland and allows to use a network namespace from a container and let it access the outside world (with some limitations).
We will also introduce Usernetes, and how to run Kubernetes in an unprivileged user namespace
https://sched.co/Jcgg
Dieser Vortrag zeigt, welchen Herausforderungen im Hosting punkt.de in der jüngeren Vergangenheit gegenüber stand und welche Änderungen gegenüber unserer 2010 vorgestellten "NanoBSD"-Architektur wir seitdem umgesetzt haben. Der proServer hat auf der Seite des Hosting-Anbieters viele der erwünschten Eigenschaften eines "Private Cloud"-Produkts, stellt sich dem Kunden aber wie ein klassischer Root-Server dar. Für Anwendungen, bei denen eine solche Plattform gefordert ist, ein unschätzbarer Vorteil gegenüber reinen Container-Lösungen, die praktisch immer eine speziell angepasste Anwendungs-Architektur benötigen.
Running services in virtualized systems provides many benefits, but has often presented performance and flexibility drawbacks. This has become critical when managing large databases, where resource usage and performance are paramount. We will explore a case study in the use of Docker to roll out multiple database servers distributed across multiple physical servers.
Kubernetes supports several security mechanisms such as Seccomp, Apparmor, SELinux, and runAsUser for protecting the hosts from container-breakout attacks. However, these mechanisms are not sufficient for the security demand because Kubelet and CRI/OCI runtimes require the root privileges on the hosts, and these components are seriously bug-prone. The dependency on the root privileges has been also problematic for promoting Kubernetes to the HPC world, where users are often disallowed to install software as the root.
In this talk, Akihiro and Giuseppe will show the community’s ongoing work for making Kubernetes deployable and runnable as a non-root user, by using User Namespaces. The main topics of discussion will be UID/GID mapping, unprivileged Copy-on-Write filesystems, Usermode networking (Slirp), and Cgroups.
https://fosdem.org/2019/schedule/event/containers_k8s_rootless/
Docker is going to change the way services are deployed by encapsulating them, thus provide a robust and complete continous development/deployment workflow. But how is docker impacting the compute part of HPC? Containerization uses kernel functions (Namespaces/CGroups) to encapsulate the processes, while allowing a fain grained customization of the compute stack; sitting on top of a stripped down bare-metal OS that provides basic services.
This sessions aims to explore if docker already is the 'virtualization' technique the HPC community is waiting for.
Could a distributed MPI job across multiple containers placed on different physical nodes beat a natively started job?
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
Webinar Session - https://youtu.be/_5MfGMf8PG4
In this webinar, we share how the Container Attached Storage pattern makes performance tuning more tractable, by giving each workload its own storage system, thereby decreasing the variables needed to understand and tune performance.
We then introduce MayaStor, a breakthrough in the use of containers and Kubernetes as a data plane. MayaStor is the first containerized data engine available that delivers near the theoretical maximum performance of underlying systems. MayaStor performance scales with the underlying hardware and has been shown, for example, to deliver in excess of 10 million IOPS in a particular environment.
Introducing Container Technology to TSUBAME3.0 SupercomputerAkihiro Nomura
Invited Talk in ISC High Performance 2019 Focus Session "Containers for Acceleration and Accessibility in HPC and Cloud Ecosystems" https://2019.isc-program.com/presentation/?id=inv_sp183&sess=sess177
Introduction to Linux Kernel by Quontra SolutionsQUONTRASOLUTIONS
Course Duration: 30-35 hours Training + Assignments + Actual Project Based Case Studies
Training Materials: All attendees will receive,
Assignment after each module, Video recording of every session
Notes and study material for examples covered.
Access to the Training Blog & Repository of Materials
Pre-requisites:
Basic Computer Skills and knowledge of IT.
Training Highlights
* Focus on Hands on training.
* 30 hours of Assignments, Live Case Studies.
* Video Recordings of sessions provided.
* One Problem Statement discussed across the whole training program.
* Resume prep, Interview Questions provided.
WEBSITE: www.QuontraSolutions.com
Contact Info: Phone +1 404-900-9988(or) Email - info@quontrasolutions.com
his workshop will shed light on a modern solution to solve application portability, building, delivery, packaging, and system dependency issues. Containers especially Docker have seen accelerated adoption in the web, cloud and recently the enterprise. HPC environments are seeing something similar to the introduction of HPC containers Singularity and Shifter. They provide a good use case for solving software portability, not to mention ensure repeatability of results. Not to mention their ECO system provides for the better development, delivery, testing workflows that were alien to most of HPC environments. This workshop will cover the Theory and hands-on of containers and Its ecosystem. Introducing Docker and singularity containers; Docker as a general-purpose container for almost any app, Singularity as the particular container technology for HPC. The workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Engage 2020 - Kubernetes for HCL Connections Component Pack - Build or Buy?panagenda
HCL Connections V7 will be based on Kubernetes only! A parallel WebSphere environment won't be necessary any longer. Martin and Christoph collected the basics and differences in building a Kubernetes environment of your choice. They show you a comparison of an on-premises deployment versus a hosted cloud environment (Amazon EKS). After this session you have the basics to size and build a Kubernetes cluster for Component Pack, so you can start learning the new technology to take off with Connections V7 and become a Kubernaut.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Welocme to ViralQR, your best QR code generator.ViralQR
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
By Design, not by Accident - Agile Venture Bolzano 2024
pkgsrc on SmartOS
1. pkgsrc on SmartOS
Jonathan Perkin @jperkin
Software Engineer jperkin@joyent.com
Joyent jperkin@pkgsrc.org
Saturday, 23 March 13
2. Agenda
• Introduction
• History and background to SmartOS
• pkgsrc usage in SmartOS
• pkgsrc features we’ve developed
• pkgsrc development at Joyent
• Our pkgsrc wishlist
Saturday, 23 March 13
3. $ whoami!
• Started working on pkgsrc in 2001
• Used pkgsrc on Solaris internally at BBC
• Worked on build systems and release engineering for MySQL/Sun/Oracle
• Now work full-time on pkgsrc for Joyent
$ uname -a
SunOS www0-rth.thls.bbc.co.uk 5.7 Generic sun4m sparc SUNW,SPARCstation-20
$ uptime
12:28pm up 5000 day(s), 2 users, load average: 0.91, 0.89, 0.87
Saturday, 23 March 13
4. What is SmartOS - History
• Solaris 10 released by Sun in 2005
• ZFS, DTrace, Zones, SMF
• Open Source
• OpenSolaris continued the development towards Solaris 11
• IPS, Crossbow, Laptop/Desktop support, GNU environment
• However, Oracle made Solaris closed source and proprietary in 2010
• illumos was born to continue development of the last available OpenSolaris
code
• Replace closed-source parts needed to operate the system
• Continue feature development, working with other communities
Saturday, 23 March 13
5. What is SmartOS - illumos Distributions
• Diaspora of Solaris engineers from Oracle and community talent has to illumos
distributions focused on specific areas of interest
• Delphix continue DTrace and ZFS development for database virtualisation
products
• Nexenta and DEY focus on storage appliances and services
• OmniTI produce OmniOS, a general-purpose illumos distribution for servers
• OpenIndiana continues development of a desktop-oriented environment
• Joyent produce SmartOS with a specific focus on cloud computing
Saturday, 23 March 13
6. What is SmartOS - Joyent!
• Joyent operate a high performance public cloud. The key features of illumos
we require which differentiate it are:
• ZFS. Fast, scalable, robust storage
• DTrace. Observability and debugging across everything
• Zones. Low latency and highly customisable virtualisation
• KVM. Ability to run customer’s preferred OS (though we always
recommend Zones wherever possible)
Saturday, 23 March 13
7. What is SmartOS - Thin Hypervisor
• SmartOS is a USB/network-booted thin hypervisor
• Open source, available on GitHub
• OS resides on USB stick or PXE
• All disks are dedicated to zones
• The “Global Zone” is a ramdisk for zone administration, nothing else
• /usr is a read-only lofs mount, changes to /etc, /root, etc. not permanent
• Upgrades are trivial, replace platform directory and reboot
• Latest “First Customer Ship” build available every 2 weeks
• Clear separation between OS (platform) and packages (pkgsrc)
• PREFER_PKGSRC=yes (except for solaris-pam, terminfo)
• Zone images are fully independent of OS version
Saturday, 23 March 13
8. What is SmartOS - Zones
• All work is done inside Zones
• Zones are created from images, made up of a file system image (ZFS
snapshot) and metadata (JSON)
• imgadm and vmadm tools for management of images and virtual machines
• OS zones are “sparse” and mount /usr from GZ
• KVM systems are just images too, and run qemu process inside a thin zone
• Gain hosting features (throttling, resource management, etc.)
• Additional security
Saturday, 23 March 13
10. pkgsrc on SmartOS - Basics
• We track quarterly branches for each image version
• 2012Q2 => base*-1.8.x, 2012Q4 => base*-1.9.x, etc
• Separate images for 32-bit (ABI=32, base) and 64-bit (ABI=64, base64)
• Self-contained layout
• PREFIX=/opt/local
• PKG_SYSCONFDIR=/opt/local/etc
• PKG_DBDIR=/opt/local/pkg
• Various package sets available
• base for minimal install
• standard for base + common utilities
• percona, riak etc for specific purpose
Saturday, 23 March 13
11. pkgsrc development
• Our users demand additional pkgsrc features that we have developed:
• Multi-architecture binaries and libraries
• SMF support
• “SmartOS is Not GNU/Linux” distribution (PREFIX=/usr)
Saturday, 23 March 13
12. pkgsrc development - multiarch
• Separate 32-bit and 64-bit images are a pain:
• Confusing for users and customers who think it refers to the kernel
architecture
• Additional build and storage requirements
• Solaris has had multiarch support since Solaris 7 in 1998, old-time users
have come to expect it
Saturday, 23 March 13
13. pkgsrc development - multiarch #2
• High-level overview:
• For multiarch-enabled packages, perform each build stage (extract,
patch, ...) twice, once with ABI=32, once with ABI=64
• --bindir= and --libdir= set based on ABI
• Install both to the same DESTDIR, default ABI installed last
• PLIST expansion to cover multiple entries
• isaexec wrapper installed to handle binaries
• Create package
• Enable with MULTIARCH=yes in mk.conf
Saturday, 23 March 13
14. pkgsrc development - multiarch #3
• Infrastructure (mk/) changes:
• Each primary phase (e.g. ‘do-extract’) has a wrapper (e.g. ‘do-extract-
multi’) which calls the primary target for each ABI
• mk/plist/plist-multiarch.awk handles PLIST expansion
• OPSYS-specific {BIN,LIB}ARCHSUFFIX.{32,64} variables
• BINARCHSUFFIX.32= /i86
• BINARCHSUFFIX.64= /amd64
• pkgtools/isaexec package created, supports ABI environment variable
• pkg_install supports new @link directive for isaexec hardlinks
Saturday, 23 March 13
15. pkgsrc development - multiarch #4
• Converting a simple autoconf/cmake package requires:
• USE_MULTIARCH= bin lib
• Converting more complicated custom packages involves:
• USE_MULTIARCH= bin lib
• Change hardcoded lib to lib${LIBARCHSUFFIX}
• Change hardcoded bin to bin${BINARCHSUFFIX}
• Use MULTIARCH_DIRS.{bin,lib}
• Use MULTIARCH_SKIP_DIRS.{bin,lib}
• Complicated WRKSRC handling
• Developers, please stop writing your own build systems!
Saturday, 23 March 13
16. pkgsrc development - multiarch #5
• Current status
• Large number of packages converted (p5-* automatically handled)
• SmartOS “trunk” image uses multiarch by default, ~8,000 packages
• 504 files changed, 3525 insertions(+), 902 deletions(-)
• Developed in Joyent pkgsrc repository
• joyent/feature/multiarch/trunk -> joyent/release/trunk
• joyent/feature/multiarch/YYYYQQ -> joyent/release/YYYYQQ_multiarch
Saturday, 23 March 13
17. pkgsrc development - SMF
• Solaris 10 introduced the Service Management Framework as a replacement
for the legacy init.d system. We need to support it.
• svccfg to import a new manifest
• svcadm to enable/disable/restart
• svcs to display status
• Small number of infrastructure changes
• New mk/smf.mk
• 4 files changed, 131 insertions(+)
• Manifest and method files current held externally similar to LOCALPATCHES
• ${SMFBASE}/${PKGPATH}
• Developed in joyent/feature/smf/trunk branch
Saturday, 23 March 13
18. pkgsrc development - SNGL
• “SmartOS is Not GNU/Linux”, aka “snuggle”
• Some users not keen on /opt/local layout, we can fix that!
• Flexibility of zones allows /usr -> /system/usr, /lib -> /system/lib, etc.
• pkgsrc built with LOCALBASE=/usr PKG_SYSCONFDIR=/etc
• Symlinks in /usr for compatability, overridden by packages
• Works surprisingly well, with caveats
• Most builtin checks are not LOCALBASE=/usr clean
• Some packages (hi perl!) make assumptions about /usr
• Would be great to see other users of this, e.g. Linux
• Developed in joyent/feature/sngl/trunk branch
Saturday, 23 March 13
19. pkgsrc development - gccruntime
• SmartOS does not provide GCC runtime (libgcc, libstdc++ etc), needs to come
from pkgsrc
• Split runtime out to lang/gcc47-libs package
• Packages register full DEPENDS upon it, use GCC -specs override for RPATH
• USE_PKGSRC_GCC_RUNTIME=yes to enable
• Packages which USE_LIBTOOL handled automatically
• Packages which build their own shared libraries need special handling
• USE_GCC_RUNTIME=yes
• Not ideal, compiler.mk sourced early so needs to be set first
• Would like to integrate remaining USE_GCC_RUNTIME for 2013Q2
• Currently developed in joyent/feature/gccruntime/trunk
Saturday, 23 March 13
20. pkgsrc development - misc
• OpenJDK7
• Patches from SmartOS user “jesse_” to get latest OpenJDK working
• Mostly working openjdk7-current package, would like to discuss *BSD
status
• _PBULK_MULTI support for mysql* and percona
• Need a good naming scheme, p5-DBD-mysql51 / p5-DBD-percona55?
• Various small changes in joyent/feature/miscfix/trunk
Saturday, 23 March 13
21. pkgsrc development - git
• We develop everything with git / github
• https://github.com/joyent/pkgsrc (thanks joerg!)
• https://github.com/joyent/pkgsrc-wip
• https://github.com/joyent/pkgsrc-joyent
• Cheap and fast branching makes it all possible
• upstream/trunk->trunk, upstream/pkgsrc_*->pkgsrc_* (pull --rebase)
• trunk -> joyent/feature/*/trunk -> joyent/release/trunk
• pkgsrc_YYYYQQ -> joyent/feature/*/YYYYQQ -> joyent/release/YYYYQQ
• joyent/feature/*/YYYYQQ -> joyent/release/YYYYQQ_feature
• Currently 5 active features (gccruntime, miscfix, multiarch, smf, sngl)
• Short-lived branches (joyent/bulktest/*, pkggnudir, openssl1, etc)
Saturday, 23 March 13
22. pkgsrc development - pbulk
• We build lots of binary packages
• /opt/pbulk bootstrap + development tools
• pbulk cluster of 6 client VMs (8 CPU, 8GB each), 1 master, + NFS
• Can perform a full clean bulk build in under 2 days with MAKE_JOBS=8
• Builds done inside chroot for concurrency
• Some changes required (in joyent/feature/miscfix/trunk)
• Support creation/deletion of chroots + config file support
• SO_REUSEADDR for limited_list
• Enable rsync of RESTRICTED dependencies
• Add SPECIFIC_PKGS support for full pkgsrc + selected pkgsrc-wip
Saturday, 23 March 13
23. pkgsrc development - Jenkins
• We use Jenkins for triggering bulk builds
• Was in use by other parts of Joyent anyway
• GitHub plugin monitors for pushes in joyent/release/* branches
• Build triggered for every push
• Limited to one concurrent build per branch,
• Upstream builds triggered via Jenkins “cron”
• Tracked directly from jsonn/pkgsrc.git
• 4 builds per week (2 x 32-bit, 2 x 64-bit)
• Results sent directly to pkgsrc-bulk@netbsd.org
• Provides quite a nice interface, plus Jabber notification etc.
Saturday, 23 March 13
24. pkgsrc wishlist
• Things we really want at some point:
• Support for multiple repositories in pkgin
• Decent upgrade support / config file management
• Multi-packages (users require different PKG_OPTIONS)
• Additional pbulk-multi support (mysql, postgres, etc.)
• Things which would be nice to have:
• apt/dpkg backend support
• Better pkg_alternatives support
• pkgin package aliases (#1 confusion for users is scmgit, #2 is gcc47)
• Working (and updated) xorg server
Saturday, 23 March 13
25. Thank You!
• “All this is possible thanks to the unique way the BBC is funded”
• Thank you for all your development work and pkgsrc commits
• We couldn’t handle the pace of upgrades ourselves
• Your work is used further than perhaps you realise
• Let’s keep going!
• Regular binary builds for other platforms (OSX, Cygwin)
• Become the de-facto package manager on more platforms
• Users => Fixes => Developers => Users => ...
Saturday, 23 March 13
26. Thanks!
• www.joyent.com - high-performance VMs and private cloud infrastructure
• SmartDataCentre (SDC) powered, SmartOS + additional tooling
• REST APIs for managing VMs
• Cloud Analytics (DTrace) plus other goodies
• Use “FREEJOY” promo code for $125 free credit (2 months running a 1GB
SmartMachine)
• www.smartos.org - open source OS for running VMs
• Contribute your own images to http://datasets.at/ or publish yourself
• Questions?
Saturday, 23 March 13