Beginning Kindle Hackery

  • 5,369 views
Uploaded on

I gave a talk on Kindle hacking at Shibuya Perl Mongers in April 2009. It documents my first couple months messing around with the Kindle.

I gave a talk on Kindle hacking at Shibuya Perl Mongers in April 2009. It documents my first couple months messing around with the Kindle.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • So what I've learned from this is that before getting to the base system and actually coding I should try the simple path and just convert! What an amazing victory for you!
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
5,369
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
106
Comments
1
Likes
9

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. You can take the Hacker out of Perl...
  • 2. ...but you can’t take the Perl out of the hacker.
  • 3. Laziness, Impatience and Hubris on the Kindle
  • 4. This talk contains almost no Perl.
  • 5. ...and almost no Japanese.
  • 6. I’m very sorry.
  • 7. And no x86 ASM
  • 8. I’m not sorry.
  • 9. The Amazon Kindle
  • 10. eBook Reader
  • 11. eInk Screen
  • 12. 800x600; 16 levels of grey
  • 13. No backlight
  • 14. Looks great outside!
  • 15. Good things about the Kindle
  • 16. 3G for 1-click shopping
  • 17. 250,000+ books to buy
  • 18. (Most < USD10)
  • 19. Newspaper, Magazine, Blog subscriptions
  • 20. Delivered every morning
  • 21. $1-$10 each month
  • 22. Email → eBook conversion
  • 23. $0.10 for autodelivery
  • 24. $FREE if you copy the result by USB
  • 25. Free Web Browser
  • 26. Download .mobi, .prc, .azw, .txt for free
  • 27. “Experimental”
  • 28. Means: We might start charging money for this
  • 29. NetFront 3.5
  • 30. Basic CSS
  • 31. Javascript
  • 32. XmlHttpRequest
  • 33. Free Wikipedia
  • 34. Not “experimental”
  • 35. Text to speech.
  • 36. Minesweeper!
  • 37. Bad things about the Kindle
  • 38. DRM
  • 39. When you buy books, they are locked to the Kindle
  • 40. Amazon lawyers went after a tool to let you read non- Amazon DRMed ebooks
  • 41. (The same tool can help you remove the DRM from books Amazon sells you)
  • 42. Limited eBook formats
  • 43. I want to read PDFs
  • 44. I want to read ePubs
  • 45. I want to read Manga
  • 46. Actually, I don’t read manga
  • 47. But lots of my friends do
  • 48. Surveillance
  • 49. In the US, 3G includes GPS
  • 50. The Kindle has a GPS
  • 51. Device logs are sent to Amazon
  • 52. Including many user actions
  • 53. ...like the websites you visit
  • 54. ...and what books you read
  • 55. Appears to include GPS info
  • 56. Amazon knows where you are
  • 57. Lock-in
  • 58. Designed to work only with Amazon
  • 59. 3G use is free, but only while Amazon likes you
  • 60. Summary: The Kindle is an “appliance”
  • 61. You wouldn’t hack a book, would you?
  • 62. I got a Kindle to read books
  • 63. I didn’t plan to hack it
  • 64. I really wanted to read books in other formats
  • 65. I’m a sucker for sexy platforms
  • 66. And it was begging me
  • 67. It’s a new toy.
  • 68. I am going to hack it.
  • 69. Laziness
  • 70. http://igorsk.blogspot.com
  • 71. Hidden debug commands
  • 72. ;debugOn
  • 73. ‘help
  • 74. Hubris
  • 75. “I can’t possibly brick my Kindle with the keyboard, right?”
  • 76. So, I started typing commands.
  • 77. “‘usbNetwork” sounds good.
  • 78. ...nothing happened
  • 79. How about “‘usbQa’”?
  • 80. It turned off the WIFI
  • 81. ..and turned off USB Disk mode.
  • 82. Impatience
  • 83. I gave up
  • 84. Sometimes laziness wins
  • 85. An hour later, I rebooted my Macbook Air
  • 86. So I set up a DHCP server
  • 87. Nothing...
  • 88. sh-3.2# tcpdump -i en1 listening on en1, link-type EN10MB (Ethernet), capture size 96 bytes [ ...] 12:36:15.238229 arp who-has 192.168.15.200 tell 192.168.15.244
  • 89. Now my Kindle can tether through my Macbook
  • 90. I want to read other eBook formats - attempt #1
  • 91. The Kindle has a browser
  • 92. I have a web server
  • 93. Web based proxy
  • 94. Small perl app
  • 95. mobiperl
  • 96. Perl 4
  • 97. no strict;
  • 98. no warnings;
  • 99. global variables
  • 100. hmm. no ePub support
  • 101. Spent a weekend learning how ePub format works
  • 102. github.com/obra/ unsavory-epub-hacks
  • 103. Slow. Annoying. Requires a Server
  • 104. Servers are evil
  • 105. Hm.
  • 106. Back to the drawing board
  • 107. I want to read other eBook formats - attempt #2
  • 108. Let’s review what we know:
  • 109. What’s inside the Kindle?
  • 110. 800x600 eInk screen
  • 111. You know about the screen
  • 112. Freescale iMX31
  • 113. ARM1136JF-S
  • 114. (Includes FPU)
  • 115. + Multimedia stuff
  • 116. 2 GB Flash
  • 117. 128 MB RAM
  • 118. USB OTG + MicroUSB slot
  • 119. Audio hardware
  • 120. Keyboard
  • 121. The Kindle sounds like a computer, not a book
  • 122. It MUST use some GPL code...
  • 123. https://www.amazon.com/gp/help/ customer/display.html?inodeId=200203720
  • 124. gplrelease.tar.gz
  • 125. alsa-lib-1.0.13 gcc-4.1.2 module-init-tools-3.2.2_patch alsa-lib-1.0.13_patch glib-2.12.9 monit-4.9 alsa-utils-1.0.13 glibc-2.5 mtd-utils-1.0.0 alsa-utils-1.0.13_patch gst-plugins-base-0.10.17 picocom-1.4 base-files-3.0.14.ipk gst-plugins-base-0.10.6 powertop-1.10 base-passwd_3.5.9 gstreamer-0.10.17 procps-3.2.7 binutils-2.17.50.0.5 hotplug-2004_09_20 procps-3.2.7_patch bonnie++-1.03c ifupdown_0.6.8 readline-4.3 bootchart-0.9 iptables-1.3.3 syslog-ng-1.6.11 busybox-1.7.2 klibc-1.5 sysvinit-2.86 dosfstools-2.11 libol-0.3.18 taglib-1.5 e2fsprogs-1.38 linux-2.6.22-lab126 uboot-1.3.0-rc3 e2fsprogs-1.38_patch lrzsz-0.12.20 udev-112 fuse-2.7.1 lzo-1.08 util-linux-2.12r fuse-2.7.1_link module-init-tools-3.2.2
  • 126. Linux
  • 127. I can work with this
  • 128. But how do I get code onto it?
  • 129. My friend nmap tells me...
  • 130. The Kindle listens on a few ports.
  • 131. None of them love me at all
  • 132. I guess I’ll need to take matters into my own hands.
  • 133. Hey, the Kindle has busybox
  • 134. busybox has telnetd
  • 135. Maybe I just need to install /etc/rc5.d/S99telnetd
  • 136. More research from http://igorsk.blogspot.com
  • 137. Kindle 1 update extractor
  • 138. (Python script)
  • 139. But I want to make new updates...
  • 140. I reverse engineered the reverse engineering tool
  • 141. Updates are a short header, an MD5 and a tarball
  • 142. ...run through a trivial cipher
  • 143. they’re not encrypted
  • 144. The Kindle2 is a little different than the Kindle1
  • 145. It has different magic #s in the update header
  • 146. I waited for the first Kindle 2 “update.bin”
  • 147. I grabbed its header
  • 148. I built a new “update”
  • 149. It installed one file
  • 150. /etc/rc5.d/S99telnetd
  • 151. /bin/busybox telnetd -p 2323
  • 152. ...nope.
  • 153. I ran strings on the Kindle’s busybox
  • 154. No telnetd!
  • 155. Where can I get a busybox for the Kindle?
  • 156. What else has a a similar CPU?
  • 157. My gPhone!
  • 158. Lots of people built static busybox for the gPhone
  • 159. telnetd: take 2
  • 160. login:
  • 161. login: root Password: Login incorrect
  • 162. /bin/sh makes a better /bin/login
  • 163. 125-6-81-160:ß jesse$ telnet kindle 2323 Trying 192.168.15.244... Connected to kindle. Escape character is '^Ü'. / # cat /etc/motd ############################################### ## # NOTICE * NOTICE * NOTICE # ############################################### ## Rootfs is mounted read-only. Invoke mntroot rw to switch back to a writable rootfs. ############################################### ## /#
  • 164. What I found:
  • 165. Most of /sbin is written in sh
  • 166. Fun stuff in /proc
  • 167. /proc/config.gz
  • 168. # # Automatically generated make config: don't edit # Linux kernel version: 2.6.22.19 # Mon Mar 2 12:13:07 2009 # CONFIG_ARM=y CONFIG_SYS_SUPPORTS_APM_EMULATION=y # CONFIG_GENERIC_GPIO is not set CONFIG_GENERIC_TIME=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_MMU=y # CONFIG_NO_IOPORT is not set CONFIG_GENERIC_HARDIRQS=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_LOCKDEP_SUPPORT=y ...
  • 169. ...I could rebuild the Kernel
  • 170. /proc/filesystems
  • 171. nodev sysfs nodev rootfs nodev bdev nodev proc msdos nodev sockfs vfat nodev pipefs nodev nfs nodev nodev rpc_pipefs anon_inodefs nodev fuse nodev futexfs fuseblk nodev tmpfs nodev fusectl nodev inotifyfs nodev devpts ext3 nodev ramfs
  • 172. nodev sysfs nodev rootfs nodev bdev nodev proc msdos nodev sockfs vfat nodev pipefs nodev nfs nodev nodev rpc_pipefs anon_inodefs nodev fuse nodev futexfs fuseblk nodev tmpfs nodev fusectl nodev inotifyfs nodev devpts ext3 nodev ramfs
  • 173. I’m not restricted to 2GB
  • 174. It’s a Linux box
  • 175. I can cross-compile!
  • 176. http://www.codesourcery.com/ Prebuilt ARM toolchain
  • 177. It generates generic ARM machine code
  • 178. That’s ok, but not great
  • 179. I’ll cross compile Perl
  • 180. 1 day of frustration passes
  • 181. I won’t cross-compile Perl
  • 182. Crosscompiling Perl == Bad Joke
  • 183. I’ll cross-compile Python
  • 184. Same bad joke
  • 185. Maybe I need a native compiler for ARM
  • 186. Where do I get an ARM build farm?
  • 187. I have a gPhone
  • 188. It’s not a great build host
  • 189. I have an N810
  • 190. It’s not a great build host...
  • 191. ...but it has an important advantage
  • 192. apt-get install gcc
  • 193. N810: Linux 2.6; glibc 2.5
  • 194. N810 binaries run unmodified on the Kindle
  • 195. I built perl in an hour
  • 196. Sadly, I realized that Python is a better choice
  • 197. I also realized that building on the Kindle works better than on the N810.
  • 198. (Version skew in extra libraries makes things hard)
  • 199. I tried building gcc on the N810...
  • 200. Found Pengutronix / OSELAS.de
  • 201. It’s a compiler toolchain builder.
  • 202. I built my own crosscompilers for ARM1136JF-S - Linux 2.6 - glibc 2.5
  • 203. I used the cross compiler to compile gcc, glibc (for proper headers), binutils, shellutils, dropbear & screen
  • 204. I cross-compiled nfsmount
  • 205. I nfs-mounted a disk image with the compiler
  • 206. Then I started building more stuff
  • 207. I am a Perl Hacker
  • 208. I believe in the three virtues
  • 209. Lazyness
  • 210. Impatience
  • 211. Hubris
  • 212. Sometimes, Perl isn’t the Right Tool
  • 213. Calibre is the Killer App for ebook conversion and management
  • 214. Lazyness
  • 215. I like the best tools
  • 216. The best tools already exist
  • 217. Calibre has dozens of eBook format converters.
  • 218. Why reimplement them?
  • 219. Hubris
  • 220. “I can learn enough Python in a weekend to port this application to the Kindle”
  • 221. The downside
  • 222. Dependencies
  • 223. Who’s dealt with Python app dependencies?
  • 224. No CPAN.
  • 225. Everything you need is in the Standard Library.
  • 226. If it’s not in the Standard Library, it’s not worth using.
  • 227. Except when you need it.
  • 228. They have....
  • 229. “easy_install”
  • 230. It’s not so easy
  • 231. It is very perlish
  • 232. It does recursive web scraping to find tarballs on developers’ web sites.
  • 233. Most of the deps actually installed ok.
  • 234. I just ran the app over and over until it stopped erroring.
  • 235. God I miss Perl.
  • 236. And then we get to the big problem.
  • 237. Qt
  • 238. Calibre’s UI is in Qt
  • 239. ...so its backend uses Qt because it’s easy
  • 240. PyQt binds Qt to Python
  • 241. For Qt for Windows
  • 242. for Qt for Mac
  • 243. for Qt for X11
  • 244. No X11 on Kindle
  • 245. (Just a Framebuffer)
  • 246. QtEmbedded
  • 247. No problem!
  • 248. No PyQtEmbedded
  • 249. Finally got Calibre running...
  • 250. by hacking out components I don’t need.
  • 251. It was good enough to try to convert a trivial ebook.
  • 252. It took 12 hours...
  • 253. ...after I built swaputils and gave it 256MB of swap
  • 254. So what was it doing?
  • 255. HTML → Mobipocket converter
  • 256. With a full CSS engine
  • 257. It visits every DOM element...
  • 258. and computes CSS styles to convert them to trivial HTML 3.2...
  • 259. ...twice.
  • 260. Lazyness, Impatience, Hubris can all help here.
  • 261. Help me Larry-wan.
  • 262. Very few CSS rules really matter.
  • 263. The Kindle supports very little HTML.
  • 264. It mostly supports HTML 3.2...just no <pre>
  • 265. ...that’s the only thing the 12 hour CSS engine got us
  • 266. You can emulate <pre> with <tt> and &nbsp;
  • 267. if tag == 'pre': self.inside_pre = 1 tag = 'tt' if prefixname(elem.tag, nsrmap) == 'pre': buffer.write('<br/>n') self.inside_pre = 0 if self.inside_pre: text=text.replace(' ','&nbsp;') text=re.sub(r'(rn|r|n)', '<br/>n', text)
  • 268. Now it runs in 60 megs and about 10 minutes
  • 269. So, now I can run code.
  • 270. Still no UI access.
  • 271. I don’t really want to hack Java GUI code.
  • 272. And where could I plug my custom UI into the Kindle’s?
  • 273. I don’t want to break Amazon’s UI.
  • 274. Oh hey.
  • 275. There is an application I could replace with something custom...
  • 276. But really, I don’t want to.
  • 277. Sure, I could decompile.
  • 278. It’s obfuscated.
  • 279. It’d be annoying.
  • 280. If I built UI, I’d have to maintain a UI.
  • 281. And users can break a UI.
  • 282. No buttons = Less to screw up
  • 283. But I have this ebook converter.
  • 284. I do want to let users convert books.
  • 285. What to do?
  • 286. nodev sysfs nodev rootfs nodev bdev nodev proc msdos nodev sockfs vfat nodev pipefs nodev nfs nodev nodev rpc_pipefs anon_inodefs nodev fuse nodev futexfs fuseblk nodev tmpfs nodev fusectl nodev inotifyfs nodev devpts ext3 nodev ramfs
  • 287. nodev sysfs nodev rootfs nodev bdev nodev proc msdos nodev sockfs vfat nodev pipefs nodev nfs nodev nodev rpc_pipefs anon_inodefs nodev fuse nodev futexfs fuseblk nodev tmpfs nodev fusectl nodev inotifyfs nodev devpts ext3 nodev ramfs
  • 288. Inotify blocks on filesystem events.
  • 289. pyInotify lets me get at fs events easily.
  • 290. class InotifyListener (threading.Thread): global cv def run ( self ): global conversionQueue wm = WatchManager() # Watch Manager mask = IN_MOVED_TO | IN_CREATE # watched events p = PTmp() notifier = Notifier(wm, p) wdd = wm.add_watch('/mnt/us/documents', mask, rec=True) notifier.loop()
  • 291. It works great for downloads
  • 292. Copies over USB didn’t trigger inotify events.
  • 293. It’s probably something fuse-related.
  • 294. I went for the cheap hack.
  • 295. When you eject the Kindle, it generates a DBus event.
  • 296. class DbusWatcher (threading.Thread): global cv def run ( self ): global conversionQueue cmd='/usr/bin/dbus-monitor --system' pipe = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout while 1: line = pipe.readline() if any(line.find(i) != -1 for i in ['usbPlugOut', 'resuming']): for f in os.listdir('/mnt/us/documents'): maybe_enqueue_file('/mnt/us/documents/'+f)
  • 297. What’s next?
  • 298. Remember config.gz?
  • 299. I can build a new kernel
  • 300. ...and add back missing drivers
  • 301. USB Mass Storage Host
  • 302. USB WIFI?
  • 303. What isn’t next?
  • 304. Reverse engineering Java to extend the Kindle’s UI
  • 305. Python, and Shell, I’m happy to hack for a good cause.
  • 306. Java is another matter entirely.
  • 307. Thanks!
  • 308. I had a big finish planned.
  • 309. I was going to build and show off a manga converter.
  • 310. (for .cbz format books)
  • 311. So I downloaded a .cbz.
  • 312. ...and copied it to the Kindle...
  • 313. ...and I saw this...
  • 314. The best hacking is no hacking.
  • 315. Thanks!