Email
Favorite
Favorited ×
Download
Embed
Private Content
Copy and paste this code into your blog or website Copy Customize Close
We have emailed the verification/download link to "".
Login to your email and click the link to download the file directly.

To request the link at a different email address, update it here. Close
Validation messages. Success message. Fail message.

Check your bulk/spam folders if you can't find our mail.

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

+ Follow

Device deployment

by Angelo van der Sijpt on Dec 07, 2010

633 views

Using Apache ACE as a distribution and management platform for a large--and growing-- number of embedded devices in the field. ...

More…

Using Apache ACE as a distribution and management platform for a large--and growing-- number of embedded devices in the field.

I used this presentation at Apachecon NA 2010.

I'm more about story and images than about text on slides, you can try to follow along here.

Less

2 Embeds 164

http://changelos.com	162
url_unknown	2

Statistics

Favorites: 0
Downloads: 0
Comments: 0
Embed Views: 164
Views on SlideShare: 469
Total Views: 633

\n
NOTE: I am not Karl Pauls; something came up at the last moment\n
\n
\nImage of transformer: http://www.flickr.com/photos/mrbill/2547622226/in/photostream/\nImage of phone: @m4rr5\n
\nImage of transformer: http://www.flickr.com/photos/mrbill/2547622226/in/photostream/\nImage of phone: @m4rr5\n
Who recognizes this? It’s a bridge rectifier.\nAnd what does it do? It converts AC to DC, but at loss of energy.\nMost devices you own actually don’t need AC at all (perhaps only your washing machine); everything else is happy with DC.\n
Moixa has a vision of putting a Home Energy System in your house, which provides your house with a DC network (USB 3.0 based). Now we have a smart hub in your home, you can control it using your mobile phone, and check your power usage.\n\nVideo: http://www.youtube.com/watch?v=IEENXnp6-Ng\n
So there we have it: for a greener future, we now have\na box,\nand and idea\nI have not shown it, but possible extensions for now:\n- solar panels,\n- small battery that charges during the night\nand who knows what else!\n\nBackground image: http://www.flickr.com/photos/kylebaker/4511795125/\n
So there we have it: for a greener future, we now have\na box,\nand and idea\nI have not shown it, but possible extensions for now:\n- solar panels,\n- small battery that charges during the night\nand who knows what else!\n\nBackground image: http://www.flickr.com/photos/kylebaker/4511795125/\n
So there we have it: for a greener future, we now have\na box,\nand and idea\nI have not shown it, but possible extensions for now:\n- solar panels,\n- small battery that charges during the night\nand who knows what else!\n\nBackground image: http://www.flickr.com/photos/kylebaker/4511795125/\n
So there we have it: for a greener future, we now have\na box,\nand and idea\nI have not shown it, but possible extensions for now:\n- solar panels,\n- small battery that charges during the night\nand who knows what else!\n\nBackground image: http://www.flickr.com/photos/kylebaker/4511795125/\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
From a deployment perspective, it looks like this:\n- You have LED lighting and other peripherals in your home,\n- You connect the HES to this,\n- are able to control it directly using a web interface, e.g. on your phone,\n- and we want to be able to sync this to the internet.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
The box contains\n- an embedded system based on an ARM processor, and\n- a piece of user software running linux\nThe server runs linux. Or actually, we don’t really care.\n\nTime goal: between 10m and 15m in.\n
Note: we have seen OSGi in the morning, so I only need to highlight the things that are really necessary to understand this context.\n\n
\n
\n\n\n
Working with hardware and device drivers is hard. Actually, we would just like to package these as a bundle!\n\nImages: http://www.flickr.com/photos/squeezyboy/3300595223/\nhttp://www.ikea.com/us/en/catalog/products/50081452\n
Working with hardware and device drivers is hard. Actually, we would just like to package these as a bundle!\n\nImages: http://www.flickr.com/photos/squeezyboy/3300595223/\nhttp://www.ikea.com/us/en/catalog/products/50081452\n
Some words about Bundle-NativeCode:\n- package native drivers in your bundle,\n- let the framework decide which one to load,\n- use it as regular JNI.\nThe framework can also dynamically reload your libraries, without need for stopping the JVM.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
For instance: we have a data store that represents the state of the device. The store and many peripherals are identical for server and client; only the role in synchronization is really different.\n\nSame thing applies to, for instance, the web ui.\n\nTime goal: Between 15 and 20m in.\n
\n
Pilot deployment: show a server and a single device. This didn’t last long; very soon, we needed to manage more than one device, and more to the point, we needed to manage them remotely.\n
Enter: Apache ACE\n
\n
Step one: deployment\n\n
Deployment has a time component\n
Deployment has a time component\n
Deployment has a time component\n
Why?\n- insight into who uses what (and reproduce any config locally)\n- component A is broken, what’s the damage/impact?\n- history of what happened to a target/system (“no, I did not change anything”)\n- integrate into build/test/QA cycle\n- basis for extensions such as: license mgmt, automated (re)install (the server burnt down/my mediabox broke -> minimum downtime)\n
This is the generic topology for ACE.\nin ACE we only store metadata\nthis avoids having IP in these repositories\nit also allows integration with different component repositories (OBR, Maven, Nexus, anything reachable via a URL)\nREST based interface\n
\n
- Organizing Artifacts\n- Mapping those to targets\n
Components are too fine grained to make sense to non-developers.\nTherefore we group them.\nWe have two levels of grouping: group and license\nAs an analogy, think of this as an IKEA catalog: individual components don’t make sense to end users, so we provide them with pre-defined configurations\nAll of this information is kept in the “store repository”\n\n
Second part of dependency management is actually keeping track of which licenses are associated with which targets.\nSometimes this information is already available in an external system, so we can simply import it: ACE has an API for that.\nThis information is kept in a separate store, called the “license repository”.\n
Both the store and license repository come together in the current ACE web UI.\nHere the user can:\n - import new components from an OBR,\n - create groups and licenses,\n - define associations between them.\nThe way this works is that, like with subversion, the user first retrieves the latest version of the repositories. Then he can start making changes, and once he is satisfied, he can store those changes on the server.\n\n
\n
Dependency management was all about making things manageable for users.\nDeployment is about getting the right components to the right targets.\nIt has a different repository, containing:\n - a list of targets\n - a list of versions per target\n - a list of components per version\n\nThe other part of deployment is the actual management agent, which is installed on the target and is in charge of actually installing and updating components.\n
Targets have a version (for this target), and that consists of components.\nVersions are immutable; this allows replication.\n\n
Back to topology: we have seen the server, client and touched on the component repository. What’s left: the management agent.\n\n
- The management agent consists of a number of building blocks. Distributed as separate jars, a ‘fat jar’, or even embedded in the framework.\n- Scheduler: when to connect to the server? We use a time-based scheduler, but this is pluggable.\n- The deployment task reaches out to the server:\n - it has to know who it is (identification, pluggable)\n - it has to know where to get to (discovery, pluggable)\n- The results of deployment are stored in the auditlog.\n- Finally, we cache our data, so we can e.g. quickly roll back to an older version.\n\nIf you now look at the management agent in ACE, it consists of the following components.\nI will expain how they work by describing the process of checking for updates and installing them.\nThis process starts with the scheduler.\nIt periodically triggers the deployment task.\nThis task then first asks the identification service for its own, unique name. Every target has to have some unique name, and this service actually allows you to implement different schemes for that.\nSecondly, it asks the discovery service for a URL to the server. Again this allows a flexible way of “finding the server”, the simplest implementation being one that returns a static URL.\nThen the server is actually contacted. The management agent asks which versions are actually available.\nThe server responds with a range of versions, as stored in the deployment repository.\nThe management agent knows its current version, so if the range includes a newer version, it will try to install it.\nAny life cycle changes that occur on the target (framework being restarted, bundles being started and stopped, updates being deployed) are recorded in the audit log which we will revisit a bit later. Finally, anything that gets downloaded can be stored in a cache that can be used to more quickly do things like rollbacks, install older versions, etc.\nSo that explains the different components in the management agent.\n
Let’s now in more detail at the actual update process.\nFor that we rely on a service defined in the OSGi compendium, so it’s actually a standard service, available at Apache Felix.\nDeployment Admin defines the notion of deployment packages, and in ACE we define such a deployment package for each target we have.\nA deployment package is a versioned set of artifacts or components.\nThese components can transactionally be installed or updated, so if anything goes wrong somewhere along the line then everything gets rolled back to its previous state.\nThere also is the notion of a fix package, which only contains the delta between two versions.\nYou can sign deployment packages to make them secure, or tamper proof.\nFinally, they are extensible through resource processors. This is actually a very cool feature, because it means that you can install any file type you want. You simply write a resource processor, which is just another bundle that gets deployed alongside the rest, and write some code to actually install and uninstall such a component. The resource processor obviously runs on the target, so you can use it for many different things. One such example, which is actually also part of the OSGi compendium, is the AutoConfig spec, which defines a new file type for configuration admin data, which gets installed in ConfigurationAdmin.\n\n
“From dependency to deployment”\n\nThe deployment repository is composed of the store- and license repository. You make all changes in the store- and license repositories, and generate the deployment repository based on that. We can now replicate the deployment repository (versions for targets are immutable, remember?)\n
\n
\n
Now let’s look at the last subsystem: feedback.\n- We want to know what happened during deployment; we only know whether a deployment has succeeded if we see it in the audit log.\n- The auditlog basically contains information on framework events (system start, bundle start/stop/update, etc).\n - this gives you detailed information\n- Basis for extensions\n
Time goal: between 30m and 35m in\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
Once we had a few systems, we put ACE on an Amazon machine, and started pushing software to\n- the web server\n- the few devices in the wild, and\n- our own development environment.\n
As soon as we have a few devices, it’s not only interesting to push software to the devices, but also configuration data. The feedback also becomes a very useful mechanism, since you don’t ‘see’ all devices anymore.\n
As soon as we have a few devices, it’s not only interesting to push software to the devices, but also configuration data. The feedback also becomes a very useful mechanism, since you don’t ‘see’ all devices anymore.\n
As soon as we have a few devices, it’s not only interesting to push software to the devices, but also configuration data. The feedback also becomes a very useful mechanism, since you don’t ‘see’ all devices anymore.\n
As soon as we have a few devices, it’s not only interesting to push software to the devices, but also configuration data. The feedback also becomes a very useful mechanism, since you don’t ‘see’ all devices anymore.\n
\n\n\n
\n
The bigger we get, it’s actually more of the same.\n\nNew features, like for instance battery control.\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
Thousands of devices?\n
\n