Linux for development boardsLinux for development boards
Presented by: Igor PečovnikPresented by: Igor Pečovnik
Quick facts
 Community project
 Open source
 GPL 2.0
about me
 Founder, full time maintainer and
developer @Armbian
 Small business entrepreneur ever since
 Fields of competence: math,
electronics, computer and human science,
philosophy
about armbian
● Debian or Ubuntu for more than 70 boards
● More than 30 arm & arm64 kernels
● Unoffical ports for TV boxes
● Powerfull build tools
● Stable and beta packages repository
● Good reputation
about presentation
● Motives for start
● Historical overview
● Infrastructure
● Userspace
● Build engine
www.armbian.com 6
1. Motives for start
www.armbian.com 7
A board for a project
www.armbian.com 8
Motivation
● Disappointment
● Curiosity
● Finish the project
www.armbian.com 9
Project realisation
www.armbian.com 10
2. Historical overview
www.armbian.com 11
First
● Joined Cubieboards community
● Started messing with Cubietruck kernel
● Started working on a Bash build script
● Fell into a rapid development cycle
● Others started to join
www.armbian.com 12
Early steps
● Moved to Github
● New boards: Banana, Lime, Cubox
● Ubuntu Trusty
● Downloads from public servers
● Project hosted on @A20
www.armbian.com 14
Early problems and motives
● No real equipment
● No project name
● Little special knowledge
● Learning
● Helping
● Making
www.armbian.com 15
2014: Middle age
● armbian.com
● 17 new boards
● Jessie, Xenial: systemd
● Dedicated download server
www.armbian.com 16
Middle age
● Repository (aptly)
● Community forums (IPS)
● Tapatalk mobile friendly access
● Project hosting @Odroid XU4
www.armbian.com 17
Middle age problems
● Bad, false or nonexisting documentation
● Boards comes with its own low quality kernel & uboot
● Little to no support from chip or board makers
● Still too small group for the size of the problems
● A lot of users with little experience and high expectations
www.armbian.com 18
Steady growth in 2016
● 3D (MALI) and accelerated video decoding on Allwinner boards
● 24 new boards
● First ARM64 board
● eMMC boards start to show up
● dedicated build server
www.armbian.com 19
Present situation
● 26 new boards in 2017
● Torrent download service
● First notebook support
● Merging sources to github.com/armbian
● Adding Twitter @armbian
● Project hosting went to highend servers
www.armbian.com 20
Current problems
● Strong pressure on our support
● Users failing to use prepared documentation
● Users have unrealistic expectations
● Donations does not follow project expansion
● Big hardware diversity
www.armbian.com 21
Today and tomorrow
● we are admired
● defacto standard Linux on SBC
● project might create jobs
www.armbian.com 22
3. Infrastructure
www.armbian.com 23
Brainware
● Project developers. 5-10 people
● 3rd party:
– Linux-sunxi
– Board makers
– Random participant
● General knowlege community.
– Feedback
– Dedicated testings
– Demand and wishes
www.armbian.com 24
Software
Text editor
CLI and GUI Git tools
Ubuntu Xenial
3rd party utils
Self developed build toolchain
www.armbian.com 25
Services
● Website www.
● Forums forum.
● Toolchain git.
● Download server dl.
● Repository. Stable apt.
Unstable beta.
● Twitter @armbian
● IRC #armbian, #linux-sunxi @IgorPec
armbian.com
www.armbian.com 27
4. Userspace introduction
www.armbian.com 28
Main features
● Debian or Ubuntu
● CLI or XFCE based desktop with Chromium
● Serial console enabled by default
● Forced root password change
● Wireless enabled
www.armbian.com 29
Optimisations
● Reduced write frequency to save card and gain speed
● Unified boot with ramdisk and UUID
● DT overlays (Allwinner)
● Optimised DVFS
● Auto filesystem expand (ext4, btrfs)
● Headers and build essentials are installed by default
www.armbian.com 30
Low level tools and scripts
● armbianmonitor
● h3consumption
● armhwinfo
● firstrun
● armbian-config
www.armbian.com 31
armbian-config
www.armbian.com 32
Users problems
● Failed to read warnings and documentation
● Failed to understand powering issues (microUSB)
● Usage of old, low cost or fake SD cards
● Lack of basic Linux knowledge
● Unrealistic expectations
www.armbian.com 33
Build engine overview
● 24 source configs with up to 3 kernel branches for each SoC
● X64 Ubuntu Xenial with 11 external Linaro compilers
● Menu or parameter driven
compile
show a kernel
configuration
image
kernel
Development version (4.x)
Mainline (@kernel.org) (4.x) .deb
.img
Vendor provided / legacy (3.4.x - 4.4.x)
Ubuntu Xenial
Debian Stretch
Debian Jessie
image
type
Server
Desktop
Autoselecting
proper compilerAutoselecting
options
www.armbian.com 34
● apt-cacher
● Script parallelisation
● Compiler cache (CCACHE)
● Userspace remain cached
● Building is using memory
Build engine optimizations
www.armbian.com 35
build engine
external sources
SD images
core team
users communitymakers
Daily builds
testers
Release cycle
patching
compiling
serving
torrent
patching
www.armbian.com 36
Project legacy
1) Community
2) Knowledge
3) Tools
4) Universal system
5) Promoting OSS
www.armbian.com 37
Thank you for your attention!
@armbian
igor.pecovnik@gmail.com
www.armbian.com 38
Questions before the beer ? :)

Armbian balcoon

Editor's Notes

  • #2 Ladies and gentleman. Welcome to my presentation. 4 years ago. Mid 2013. I have a dream. I have a dream to setup my own email server. I decided to host it at home. I decided to run it on a cheap, small and silent machine. ARM based single board computers seems like a right choice. Raspberry Pi, A, at that time seems like an obvious choice. Not for me. It is badly designed and slow. I search further and I found a Cubitruck? It looks it also can run a Debian and on much better hardware: gigabit, 2G RAM, dual core, SATA, battery powered! Wow!!! * Of course it was not a perfect device but is a reason to be here today.
  • #3 Armbian is a community project.Our work is open source and its published under GPL v2.0. This tells a lot about the nature of the project - - Project is sponsored mostly by our private resources but we also recieve some help from board makers and users. - - I will be presenting the project from my own perspective. - * Let’s check who am I.
  • #4 1. … and I do provide consulting services in this field. 2. … operating mostly in IT and media field. 3. … Math and computer science formal others informal way. I like to read and I like to read at least 1000 books. Currently I am at around 300. Hamradio & tinkerer enthusiast I use computers since 80’, Linux since Slackware 1.0 2 kids age of 3 and 6, light sport and enjoy craft beer * Let’s check what Armbian is.
  • #5 … half of them receive full end user support, ¼ are WIP and ¼ are EOL … on our maintaince list. Once there will be one ARM kernel, but current situatin is like this. ... is maintained by community members. … which were the first step in this project … which is updated daily … is achieved because we tend to be brutally hones and help people to move on. We don’t need to produce marketing bullshit since we are non profit oriented. * You know now little about me and little about armbian. Let’s check what this presentation will be about?
  • #6 In the first part I will present basic ideas and actions which lead me to initiate this project. Than I will show how project develop throught the time. After that I will present what kind of infrastructure we need to run all this, why and where we are different than generic Debian and close the presentation with Build engine, which pull sources and is able to produce working images out of chaotic and diverse sources. Even in parallel. To understand early motives we must go back to 2013.
  • #7 I only wanted to run a Debian Server
  • #8 4 years ago. Early 2013. I bought a Cubietruck to host a server and quickly realised that kernel is too unstable to move my services on it. SDK was also in a terrible state and recompiling a kernel seemed to be a very hard task. It bought a brick. I forgot about using this board in production and start to digg in.
  • #9 Early motivation was disappointment and I need to compensate that. Curiosity to find out if there is anything I can do to improve the situation, even I was not poking with kernel for years. For a decade to be more precisely. To finish the project. On the right side you can see the actual plan what I wanted to achieve: web, smtp, imap, samba, CUPS, …and I did it. * A year later.
  • #10 What is in the box? - A20 based Orangepi - bus lines display as an practical usage - two 220V gpio driven power sockets - two moving sensors - external temperature sensor - fiber converter - audio amplifier - battery backup What I was doing in the mean time?
  • #11 I only wanted to run a Debian Server
  • #12 … which proved to be the best resource … some people were already doing somethin. I join. … to create value and make things simpler … things start to develop quickly … to my project of making a Debian build script * I am very consistent by nature and I had a lot of spare time. My Cubietruck image was well crafted, secure and stable. That brought attention.
  • #13 … as first general user request. I never used Github before.… I recieve them for free from Olimex and Solidrun. … which is different enought to provide it as option. ... like Mega or Google drive. … which proved to be enough for running web with forums for first two years of the project. Running Armbian of course. Wheezy. My motives were expanded over the initial ones and I start to identify some problems.
  • #14 You can picture how the situation looked like in 2013. You could not really use this hardware for anything more than playing with it. My motives were expanded over the initial ones and I start to identify some problems.
  • #15 … Nothing at all. I only had one old and slow AMD powered desktop computer, 19’’ screen and antique voltmeter. … what for? I never planed to go this far. … My knowledge was rusted. Motives … or recalling what I already knew once. … others feels good. … gives opportunity to express. * Nevertheless things quickly progressed. There was not just me.
  • #16 ... domain was bought in few minutes. … were granted support in 2014 … We were moving to new init system systemd and almost gave up. There was so many crazy troubles. … was a donation from one estonian guy, which approached to me: here is a server without limitation and here is a 200 EUR. He was an ordinary man. A Linux geek, which also pay me a visit later on and he still support us even we eat a lot of his bandwidth. This year we add a torrent downloading option to relieve his resources but repository is still there and it makes a strait line on router traffic monitor for about a week when we issue an update to main repostiory.* What else was done it that period?
  • #17 Quick check: A dedicated repository was established in mid 2014. For its creating we use Aptly. It’s also integrated into our build tools Invision community. High end forum software Reposive web and forum + Tapatalk access Odroid proved to be powerfull enough until last year * Let’s take a look what kind of problems were recognised in middle age
  • #18 We start to rely more and more on the documentation but it often proved to be bad quality and a lot of time was wasted on something that should save it. … and they hope community will do the rest. Most of makes relied solo on volonteers. Most of them know how to make a board but are terrible at software support. Fixing kernels for so many boards is a huge project and we are just few amateurs. Most users have no clue how much work is done that they are able to use hardware at present state and demand more. Without giving anything in return.
  • #19 video acceleration works on our legacy Ubuntu desktop builds: A10, A20, H3 and A64 Boards are keep comming We needed to expand userspace to arm64. It was also first upstream (Debian) version at that time Finally makers dumped NAND for eMMC which is closer to SSD on speed and reliability. It self built two 14 core XEON with 128Gb memory. It is running daily rebuilds of all kernel and servers as a development machine.
  • #20 Mostly 64bit boards, but none with more than 2G memory We have dedicated torrent seeders to insure top download speed all over the world Pinebook is very much like Pine64so and its suprisingly well made for the price. We merged our personal repositories under the project We started to use twitter for promoting general news Odroid XU4 is a great board, but project needs something more powerful.
  • #21 - huge number of users, free support - we cut it down with smart rearangement but still - this will never change - we want to relieve voloonters with employing few full timers and/or pay those who are present full time with something more than beer money - each new chip design requires more resources that we actually have
  • #22 Such project is long term or you better not start it. It’s hard to predict the future for few years in advance and I don’t trust or believe such predictions. We will just continue what we do. Let’s check current infrastrucuture.
  • #23 … is made from: - brainware - software - services
  • #24 Most of people involved in the core of the project are seniour developers with decades of experiences. Daily changes example: - solving bluetooth on Tinkerboard - porting R1 WiFi driver to modern Allwinner kernel - progress in modern video driver for H3-A64 - preparing Rock64 u-boot and two kernel options - tweaking boot scripts on Lapotato And so on.
  • #25 Gedit, Notepad++ GitKraken, SourceTree … which I will show more in details Native, virtualized, containerised SD card burning utilities
  • #26 Self hosted. Initially on Olimex Lime, later on Odroid XU4 and now on server grade. We use one of the most advanced forum software avaliable (IPS). Main toolchain is hosted on Github since early days. Main download server is located in Estonia while torrent infrastructure with dedicated mirrors is all over the world. Beta repository is updated once per day, stable once every two months or in case of emergency. We use Twitter for putting out news and you can usually find me on IRC.
  • #29 … stable base ... Thunderbird, LibreWrite, Hexchat … It’s a basic debug tool any you want to have it. … while user creation is optional … and ready to connect or host Access Point … Docker support on all kernels higher than 3.10
  • #30 … We use memory for logging, Chromium cache and profile. They are synced to SD card at shutdown or daily. We use filesystem garbage flush in 10 minutes interval.… to properly identify drives, minimise boot troubles and create standard way of booting. … for simple hardware configuration adjustements ... or »dynamic voltage and frequency scaling« to achieve best thermal / performance ratio (kernel) … expand SD media to it’s full size by default. … and you are ready to build a module * we also made few extra tools to make life easier.
  • #31 … is a base diagnostic tool when seeking support. It also installs web based monitoring, CPU miner and checks SD card for speed. Data can be uploaded to paste.bin with a switch. … provides various tweaks to bring down consumtion to insane low values. Limit CPU core count, DRAM speed, disabling video out, … … armhwinfo and firstrun applies special board settings at bootup or first run … is a central menu driven configuration utility for inexperienced users. * Let’s check it out briefly.
  • #32 - system: timezone, languages, adjust SSH daemon features, toggle desktop, RDP, adjusting the display resolution, enabling read only root filesystem (Ubuntu), install kernel headers - networking: edit IP, WiFi connect, enable Access Point with best mode autodetection, from G->AC - armbian: install Armbian to internal memory, USB, SATA, freeze Armbian upgrades, switch between different kernels if avaliable - software: monitoring tools and 3rd party software installer: from Samba to OMV, CUPS, ISP config = complete web server suite, Tv headend, Pi Hole and others. But even a good tool can’t fix all the problems.
  • #33 … warnings and documentations. … powering issues when using microUSB power supply. It’s a fail by design. Rpi masked this problem in a firmware. If there is too big voltage drop it throtles CPU down, while others who does not have this feature they simply restart. Tinkerboard, MiQi and Hummingboard v1 can easly draw above microUSB connector limits. … while don’t accept that SD card will eventually die. And that happens much faster than on some other flash media. I kill one brand new top quality SD card in less than one months while using it for torrent server :)
  • #34 … and currently up to 3 different userspaces: Jessie, Xenial and Stretch Cross compilation for two reasons. First one is speed. For example: 45 minutes on quad core H3 vs 2 minutes on my build server. Currently we need 11 different compilers because of source diversity. The basic menu driven workflow goes like this: We also implement a lot of optimisations
  • #35 If you already have one, you can configure to use that one Parallelisation is a must for making large number of different images. You need a lot of cores and a lot of memory and it will build much faster. Nothing special but end users usually forget about Packages are cached on yet another level. As a whole untouched userspace. If you have free memory, engine will use it
  • #36 = 4 month long … we engage external testers 3-4 weeks before release … there is a constant interaction and daily progress … all stable images are tested and only they are shared via torrent network. This is how far things are done. I hope it was interesting to listen and that you will get yourself one developer board, download Armbian and made some cool project on it.* Let’s focus on the project legacy at the end of this presentation.
  • #37 … is a central point. It is a reason of project existance and its main resource. Community shaped our tools, the way how system work and look like. Most donations also came from community, from end users. … we tend to answer just about every question that pops out. We made documentation to limit those questions down. This way a huge knowledge base for SBCs was created. … were designed to manage huge source diversity and make life easyer for people that just need to adjust their firmware for the project without beeing a hardcore hacker. In most cases they don’t need to recompile anything. We recompile all kernels daily. All you need to do is to send a pull request. … our system is pure Debian with tools and optimisations for those boards. And Ubuntu. For the sake of compatibility and diversity. … Armbian is a open source community project. My first one of this kind. From sharing time to sharing a great experience about that sharing :)
  • #38 Thank you Linux user group of Novi Sad for invitation and opportunity to present this speech. I hope you enjoy this late friday evening presentation and feel free to ping me during this congress. * If you have any questions, now is your turn!Thank you!