Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Building an IP Network Camera

Frank Hunleth
Twitter: @fhunleth
Erlang Factory 2014
Agenda
●

Introducing Erlang to a project

●

The camera

●

Demo

●

Embedded plumbing and development

●

Performance

●...
Embedded Calibration Slide

Limited resource/
Specific purpose

General
purpose
Microcontrollers

RTOS

Embedded
Linux

An...
How to use Language X in an
Embedded Project
C/C++ application

Config scripts,
diagnostic, DSL

C/C++ framework

Operatin...
Fail

What makes Erlang/OTP interesting is its focus on
robust and scalable systems.
Try #3,4, or 5 - IP Camera
●

Build a non-trivial embedded device
–

–

Make sure that SW infrastructure supports producti...
Constraint
●

●

If a feature exists in both
Erlang and an embedded
Linux environment, use the
Erlang version
Examples
–

...
Agenda
●

Introducing Erlang to a project

●

The camera

●

Demo

●

Embedded plumbing and development

●

Performance

●...
Hardware Setup
10/100 Ethernet

Cape EEPROM

Imager
MT9V034
TI AM3359
Lens Holder
Debug Console

Beaglebone Black

GPS
Mic...
Hardware Components
AM3359
ARM Cortex-A8

Programmable
Real-time Unit
(PRU)

Data/10

I2C Master

Control - I2C

GPIO

Res...
High-level Software Components
C Ports

Erlang

Frame capture

Erlang/ALE

Erlang/ALE
GPIO

Imager reset

Erlang/ALE
I2C

...
Motion JPEG Streaming
GET /video HTTP/1.1
HTTP/1.1 200 OK
content-type: multipart/x-mixed-replace;
boundary=<boundary>
--<...
Streaming Code
handle(Req, _State) ->
Boundary = boundary(),
Headers =
[{<<"MIME-Version">>, <<"1.0">>},
{<<"content-type"...
Demo
Agenda
●

Introducing Erlang to a project

●

The camera

●

Demo

●

Embedded plumbing and development

●

Performance

●...
Nerves-Project Work Flow

Erlang/OTP
applications
rebar or
erlang.mk

BEAMs and
cross-compiled
binaries

Erlang/OTP
releas...
Raw SDCard Image
Master Boot Record
Bootloaders
Root Filesystem A

Linux kernel
Erlang
Libraries
Main application

Read-on...
relsync
●

●

●

Reprogramming SDCards gets old quickly!!!
relsync synchronizes the files in the generated Erlang/OTP
rele...
Initialization - Booting to Erlang
●

erlinit
–

Replacement for /sbin/init that starts an Erlang/OTP
release

–

Similar ...
Agenda
●

Introducing Erlang to a project

●

The camera

●

Demo

●

Embedded plumbing and development

●

Performance

●...
Performance - Throughput
●

Imager set to capture at 45 fps (22 ms/frame)

●

Browser stats
–
–

●

Average FPS ~43.5 fps ...
End-to-end Latency Measurements
Software Setup

UART
via erlang-serial

Latency
Measurement App
(Erlang, of course)

HTTP
via httpc
Received: 0.481 (~71 m...
Latency Results
●

Average frame latency 78 ms (best 66 ms,
worst 94 ms)

Display
time
updates

Exposure
time

Transfer
fr...
Boot Time (4.38 seconds)
DRAM Usage (/proc/meminfo)

●

Connect, stream for 5 seconds, disconnect, repeat

●

erlang:memory/0 reports 7.7 MB at ste...
Footprint - 18 MB RootFS

2.77 MB

3.5 MB

3.38 MB

1.9 MB

1.56 MB

1.9 MB

1.30 MB

.34

.29

.22

.2 .16 .16 .1 .1

7.3...
Conclusions
●

Will Erlang be too slow?
–

●

Will Erlang require too much DRAM and Flash memory?
–

●

No. Flash footprin...
Nerves-Project
●

All source code is open source
–
–

●

Mostly MIT licensed
Buildroot and build scripts are GPLv2

Upcomi...
Building an IP Network Camera

Frank Hunleth
Twitter: @fhunleth
Erlang Factory 2014
Upcoming SlideShare
Loading in …5
×

Building a Network IP Camera using Erlang

29,141 views

Published on

This is my Erlang Factory SF 2014 talk on using the Nerves project to make a streaming IP camera with the Beaglebone Black and a custom image sensor cape. The presentation provides throughput, latency, and memory usage measurements to motivate using Erlang in embedded Linux-based devices.

Published in: Technology

Building a Network IP Camera using Erlang

  1. 1. Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014
  2. 2. Agenda ● Introducing Erlang to a project ● The camera ● Demo ● Embedded plumbing and development ● Performance ● Conclusion
  3. 3. Embedded Calibration Slide Limited resource/ Specific purpose General purpose Microcontrollers RTOS Embedded Linux Android, iOS, Ubuntu, Windows 32-bit Microprocessors This talk is focused in this area of embedded
  4. 4. How to use Language X in an Embedded Project C/C++ application Config scripts, diagnostic, DSL C/C++ framework Operating System ● Low risk - can always implement in C first ● Worked in the past for TCL, Lua, and Javascript ● My first attempt to introduce Erlang to an organization so that it could be used for real on the next project
  5. 5. Fail What makes Erlang/OTP interesting is its focus on robust and scalable systems.
  6. 6. Try #3,4, or 5 - IP Camera ● Build a non-trivial embedded device – – Make sure that SW infrastructure supports production use cases – ● Didn't need to be a camera, but wanted a device with a hard real-time component and decent network usage Should benefit from in robustness or simplicity by using Erlang/OTP Questions – Will Erlang be too slow? – Will Erlang require too much DRAM and Flash memory? – Will developing Erlang in a cross-compiled environment be a pain? – Will I miss all of the libraries and frameworks available in C? – Will I have confidence in the Erlang/OTP platform?
  7. 7. Constraint ● ● If a feature exists in both Erlang and an embedded Linux environment, use the Erlang version Examples – No shell scripts – No SysV init, systemd, upstart (these provide process initialization and supervision)
  8. 8. Agenda ● Introducing Erlang to a project ● The camera ● Demo ● Embedded plumbing and development ● Performance ● Conclusion
  9. 9. Hardware Setup 10/100 Ethernet Cape EEPROM Imager MT9V034 TI AM3359 Lens Holder Debug Console Beaglebone Black GPS MicroSD
  10. 10. Hardware Components AM3359 ARM Cortex-A8 Programmable Real-time Unit (PRU) Data/10 I2C Master Control - I2C GPIO Reset Pixel Clock, HSync,VSync MT9V034 B&W Imager
  11. 11. High-level Software Components C Ports Erlang Frame capture Erlang/ALE Erlang/ALE GPIO Imager reset Erlang/ALE I2C Imager control Streaming Video Handler WebSockets Handler (Control) Frame 1 Camera Interface Cowboy Static Handler (index.html, JS) PRU Frame 2 Ready interrupt Frame capture Hard real-time PRU assembler JPEG encode libjpeg-turbo
  12. 12. Motion JPEG Streaming GET /video HTTP/1.1 HTTP/1.1 200 OK content-type: multipart/x-mixed-replace; boundary=<boundary> --<boundary> Content-Type: image/jpeg --<boundary> Content-Type: image/jpeg --<boundary> Content-Type: image/jpeg
  13. 13. Streaming Code handle(Req, _State) -> Boundary = boundary(), Headers = [{<<"MIME-Version">>, <<"1.0">>}, {<<"content-type">>, <<"multipart/x-mixed-replace; ", "boundary=", Boundary/binary>>}], {ok, Req2} = cowboy_req:chunked_reply(200, Headers, Req), send_first_picture(Req2), send_pictures(Req2). send_pictures(Req) -> Pic = troodon_cam:get_next_picture(), Msg = [multipart_header(Pic), Pic, delimiter()], ok = cowboy_req:chunk(Msg, Req), send_pictures(Req).
  14. 14. Demo
  15. 15. Agenda ● Introducing Erlang to a project ● The camera ● Demo ● Embedded plumbing and development ● Performance ● Conclusion
  16. 16. Nerves-Project Work Flow Erlang/OTP applications rebar or erlang.mk BEAMs and cross-compiled binaries Erlang/OTP release relsync relx fwtool + scripts nerves-sdk customizations to Buildroot Base Firmware Image Updated filesystem on target (Development) SDCard image and firmware update package
  17. 17. Raw SDCard Image Master Boot Record Bootloaders Root Filesystem A Linux kernel Erlang Libraries Main application Read-only Root Filesystem B Ping-pong location used when upgrading the firmware Read-only Application Data Partition Majority of nonvolatile memory for use by the application Read-write Raw images are only needed for initial code load and bulk device programming
  18. 18. relsync ● ● ● Reprogramming SDCards gets old quickly!!! relsync synchronizes the files in the generated Erlang/OTP release directory with corresponding ones on the target Like rsync except – – Reloads modules that changed – ● Communicates via the Erlang distribution protocol Runs scripts pre and post sync to stop and start Erlang/OTP applications (needed to update ports) Limitations – Target must have writable FS (currently using a union FS) – NIFs and linked-in port drivers can't be updated
  19. 19. Initialization - Booting to Erlang ● erlinit – Replacement for /sbin/init that starts an Erlang/OTP release – Similar to a release start script, but in C – Supports remounting root fs with unionfs for development – Configurable via Linux kernel command line
  20. 20. Agenda ● Introducing Erlang to a project ● The camera ● Demo ● Embedded plumbing and development ● Performance ● Conclusion
  21. 21. Performance - Throughput ● Imager set to capture at 45 fps (22 ms/frame) ● Browser stats – – ● Average FPS ~43.5 fps (23 ms/frame) About 1 frame dropped per second Profiling revealed JPEG encode time taking 22-23.5 ms – – ● AM3358 doesn't support HW encoding Did not investigate tuning JPEG Turbo library Take away: Erlang is not a bottleneck
  22. 22. End-to-end Latency Measurements
  23. 23. Software Setup UART via erlang-serial Latency Measurement App (Erlang, of course) HTTP via httpc Received: 0.481 (~71 ms) Timestamped Frames More info: https://github.com/fhunleth/cam_latency
  24. 24. Latency Results ● Average frame latency 78 ms (best 66 ms, worst 94 ms) Display time updates Exposure time Transfer from imager JPEG encode 1-11 ms ~10ms 20 ms Erlang / Cowboy/ HTTP TX 23 ms ● Easily accounted for latencies - 54-66 ms ● Receive HTTP stream Remaining latencies - 12-30 ms – Not unexpected; requires more instrumentation – Doubt that Erlang overhead is significant
  25. 25. Boot Time (4.38 seconds)
  26. 26. DRAM Usage (/proc/meminfo) ● Connect, stream for 5 seconds, disconnect, repeat ● erlang:memory/0 reports 7.7 MB at steady state
  27. 27. Footprint - 18 MB RootFS 2.77 MB 3.5 MB 3.38 MB 1.9 MB 1.56 MB 1.9 MB 1.30 MB .34 .29 .22 .2 .16 .16 .1 .1 7.39 MB ● Erlang/OTP release tools significantly trim footprint ● Usage on par with C++ frameworks like Qt
  28. 28. Conclusions ● Will Erlang be too slow? – ● Will Erlang require too much DRAM and Flash memory? – ● No. Flash footprint was on par with C/C++ frameworks. DRAM usage stable. Will developing Erlang in a cross-compiled environment be a pain? – ● No. Lack of HW compression is the bottleneck as it should have been. Erlang shell + relsync can be pretty nice Will I miss all of the libraries and frameworks available in C? – – ● Cowboy is far superior to anything I had used in C/C++ Still hit or miss when searching for Erlang libraries Will I have confidence in the Erlang/OTP platform? – Yes - platform feels very robust (no unexplained crashes) – Interface to C very easy – Rough edges look easily fixed with time and effort
  29. 29. Nerves-Project ● All source code is open source – – ● Mostly MIT licensed Buildroot and build scripts are GPLv2 Upcoming – – Documentation – Network configuration improvements – ● Hobby → Real products Elixir!! http://nerves-project.org
  30. 30. Building an IP Network Camera Frank Hunleth Twitter: @fhunleth Erlang Factory 2014

×