Large-scale, cross-platform synchronization using embedded python
•Download as ODP, PDF•
0 likes•1,287 views
A talk I gave at PyCon in 2006
![Large-scale, cross-platform synchronization using embedded python ,[object Object],[object Object]](https://image.slidesharecdn.com/pycon2006-fromoo-100221190539-phpapp02/85/Large-scale-cross-platform-synchronization-using-embedded-python-1-320.jpg)
![Wireless Generation ,[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
Similar to Large-scale, cross-platform synchronization using embedded python
Similar to Large-scale, cross-platform synchronization using embedded python (20)
Recently uploaded
Recently uploaded (20)
Large-scale, cross-platform synchronization using embedded python
- 3. Architecture
- 8. Design
Editor's Notes
- Hi, glad to be at pycon06 (post-lunch comment) For the next 20 minutes, I'm going to present our experience at wgen about putting python in the hands of the most demanding users: teachers. ... without them knowing. It's a short, non-technical talk and I think we have about 10 minutes at the end of the presentation for questions, and I'm of course available after that time-slot elapses. Let's get started...
- I'll take a few moments to describe the business to better understand our requirements. The name is somewhat misleading as we had hoped that classrooms would be equipped with wireless by now. When you go to the bank to talk to your banker, it's usually you, him/her and the computer. That does not work well with students. Hence we want small and cheap devices that work well around the classroom. No wireless so you need a computer for connectivity (at least for the next few years) That computer has to work reliably too, this is where python kicks in.
- That's the basic flow of information. Teachers produce and consume data in schools Which we manage (means code and data) Aggregate, analyze and build reports at various levels You can see where python fits; I'll focus in this talk on the client-side of the picture Now I'd like to explain a bit the title of this presentation
- Large scale because of the following. Listed users are active Each user has a PDA which we remotely manage Users might share sync stations, which run python All that is choreographed by a bunch of python servers that drive an oracle instance. So it's python talking to python...
- On a variety of hardware/software combination If you remember I said that teachers are demanding users: they have better to do than to click through error screens, repair broken programs or debug the usb connection to their device. So the software has to be relatively robust, on hardware that's more or less up-to-date.
- Picture this, our database has to synchronize with tens of thousands of small databases. Each small DB lives on a device, a device that was never really designed to handle a network connection on its own. So we rely on the sync station to perform all that synchronization work. It must keep the pda and the central db in sync, and make sure that the code on the pda is always up-to-date. For that we adopted a dialect of syncml. It started as syncml and evolved with our needs. It turns out that 2-phase commits are big need.
- Truth be told we started in c++ and that's not fun... Once we wrapped palm libraries in python, we were able to have a nice lean body of code that behaves pretty much consistently across platforms. Network code (proxy handling, etc.) and GUI are exactly the same (except for adapters) When your complex logic is the same across all platforms you can focus on testing and fixing it in one environment and simply work out kinks on the python-host interface. And you avoid your usual buffer-overflow bugs
- Python-wrapper around low-level device-interaction libraries (easy). Python can create/update/delete data on the device so more complex parts such as Caching, device data interpretation done in python, not hindered by trivial details. Given the network bandwidth/latencies the network stack is also implemented in python and performance not an issue
- Since the building blocks are sound and well-isolated: memory management xml parsing Network, etc. we can focus on building the necessary services to meet requirements. Example: auto-upgrades, logic packaged in .py tarballs, no cross-platform headaches.