Android, Arduino, and the Headphone Jack

33,762 views

Published on

There are quite a few 3rd party devices that attach to a smartphone's 3.5 mm headphone jack, Square's Credit Card reader, being just one of them. In this session we will reveal the magic behind this cool innovative trick. Come to learn how it was done and how your mobile app too, could take advantage of the headphone jack.

Hijacking power and bandwidth from the mobile phone's audio interface. Come to see and learn how mobile app take advantage of a smartphone's headphone jack.

We will use "phone to phone" as well as Arduino to Android examples to demonstrate and explain this cool and innovative communication channel, and you may even pick up a couple DSP (Digital Signal Processing) basics along the way.

Published in: Technology
4 Comments
51 Likes
Statistics
Notes
No Downloads
Views
Total views
33,762
On SlideShare
0
From Embeds
0
Number of Embeds
2,457
Actions
Shares
0
Downloads
721
Comments
4
Likes
51
Embeds 0
No embeds

No notes for slide

Android, Arduino, and the Headphone Jack

  1. 1. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved. “ Can you hear me now ? ” Using a phone’s 3.5mm audio jack for data exchange
  2. 2. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  3. 3. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  4. 4. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  5. 5. Tip Ring Ring Sleeve Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  6. 6. build an embedded system that captures, encodes, and transmits sensor data the theory behind audio encoding / decoding build an Android app that receives, decode, and displays the sensor data Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  7. 7. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  8. 8. Android Ground iPhone Ground Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  9. 9. Transferring a Message • Encode the Message (e.g.ASCII or Morse Code) • Convert encoded message into audio signals [D➜A] • Send (Play) the audio signals • Receive (Listen/Record) audio signals • Interpret (Filter/Transform) audio signals [A➜D] • Decode digital signal into original message Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  10. 10. Amplitude Period (time required for one complete cycle) Frequency (number of cycles per second) Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  11. 11. FSK Frequency Shift Keying Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  12. 12. ASK Amplitude Shift Keying Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  13. 13. PSK Phase Shift Keying Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  14. 14. The Sender Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  15. 15. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  16. 16. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  17. 17. Arduino on the Web http://arduino.cc Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  18. 18. Getting Started with Arduino (Make: Projects) by Massimo Banzi, Co- founder of the Arduino project Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  19. 19. const int ledPin = 13; const int outPin = 11; const int freq = 2000; const int duration = 2000; const int pause = 1000; void setup() { pinMode(ledPin, OUTPUT); pinMode(outPin, OUTPUT); } void loop() { digitalWrite(ledPin, HIGH); tone(outPin, freq); delay(duration); noTone(outPin); digitalWrite(ledPin, LOW); delay(pause); } MainLoopSetupInitialization Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  20. 20. Demo Arduino + external Speaker plays 2000 Hz sound for 2 seconds Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  21. 21. Port 11 Gnd 10K 2K2K 1uF Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  22. 22. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  23. 23. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  24. 24. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  25. 25. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  26. 26. Transferring a Message • Encode the Message (e.g.ASCII or Morse Code) • Convert encoded message into audio signals [D➜A] • Send (Play) the audio signals • Receive (Listen/Record) audio signals • Interpret (Filter/Transform) audio signals [A➜D] • Decode digital signal into original message Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  27. 27. ASCII - Encoding H e l l o W o r l d ! 0x48 0x65 0x6C 0x6C 0x6F 0x20 0x57 0x6F 0x72 0x6C 0x64 0x21 Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  28. 28. frequency = ((value -k)*M + offset ) * frequency_resolution E.g.: ‘ A’ = 0x41 = 65 = value 1st char in range: 45 = k frequency bins per Symbol: 5 = M 0Hz .. 5200 Hz* unusable: 60 = offset 44100 Hz / 512 Samples = 86.13.. = frequency_resolution ((65 - 45)*5 + 60)*86.13 = 13780.8 Hz * related to specific hardware Mapping ASCII to Frequency Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  29. 29. Usable Frequency Range depends on the Hardware
  30. 30. const int ledPin = 13; // built-in LED const int outPin = 11; // PWM Port out const int inPin = 0; // Light Sensor const char STX = '?'; // Start Token const char ETX = '@'; // End Token const unsigned int SAMPLE_RATE = 44100; // Hz const int SAMPLES = 512 ; // Num of Samples (128,256,512,1024) const int REPEAT = 3; // prolong the signal const int OFFSET = 60; // 60 un-usable bin on the spectrum's low end const double FREQ_RES = (double) SAMPLE_RATE / SAMPLES; // 86.13 frq per bin const int DURATION = (int) REPEAT * (1000 / FREQ_RES); // about 35ms const int ENC_ETX = encodeAscii(ETX); // pre-calc for later use in loop unsigned int frq[6]; // global frequency array, changes every run 1 of 3 Initialization Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  31. 31. void setup() { pinMode( ledPin, OUTPUT ); pinMode( outPin, OUTPUT ); pinMode( inPin, INPUT ); Serial.begin( 9600 ); // debugging on } /** * Encode the message into an frequency array * Wrap the message into Start and End tags */ void encodeInt(int m) { char message[5]; itoa( m,message,10 ); frq[0] = encodeAscii( STX ); for ( int i=0; i<5; i++ ) { if ('0'==message[i]) { // replace '0' marker w/ ETX frq[i+1] = encodeAscii( ETX ); break; } frq[i+1] = encodeAscii( message[i] ); } } 2 of 3 Setup and Encoding a message Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  32. 32. unsigned int encodeAscii(char c) { return (unsigned int) ((5 * (c-45) + OFFSET) * FREQ_RES); } void loop() { const int brightness = analogRead(inPin); // reads value 0-1023 encodeInt(brightness); // into global frq[] digitalWrite(ledPin, HIGH); int i=0; do { tone(outPin, frq[i]); delay(DURATION); noTone(outPin); } while (frq[i++] != ENC_ETX); digitalWrite(ledPin, LOW); delay(500); Serial.println(brightness); } 3 of 3 Encoding a single char and main loop Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  33. 33. Demo Arduino + external Speaker + TRRS-Plug plays LightSensor value Message every .5 seconds Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  34. 34. The Receiver Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  35. 35. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  36. 36. • The total number of times the signal is sampled in one second is defined as the sampling frequency. Only 44.100 Hz is guaranteed on all Android Devices • Nyquist–Shannon sampling theorem: The sampling frequency should be at least twice the highest frequency contained in the signal. I.e. Signals from the Arduino board should be in the 0..22,050 Hz range. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  37. 37. At a 44,100 Hz sample rate, 512 samples are taken in 11.6 ms Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  38. 38. We took 512 samples in 11.6 ms But we need to know the frequency the Arduino played. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  39. 39. FFT (Fast Fourier Transform) FFT is an effective algorithm to convert signals from time domain to the frequency domain. We use the 512 samples in the audio buffer as input; and the FFT algorithm returns a complex array, allowing us to calculate the magnitude of 512 frequency ranges. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  40. 40. • Sampling Rate 44,100 Hz • Effective Frequency Range 0..22,050 Hz (...Nyquist–Shannon) • 512 Samples • Frequency Range is split into 256 ranges • The Frequency Resoultion 22,050/256 = 861.52 Hz We input 512 samples and as a return we know which of the 256 frequency ranges had the strongest signal. So we won’t know the exact frequency the Arduino played, but we know, what’s called the frequency bucket or frequency bin. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  41. 41. Text .. Light sensor value 0..1023 ProgressBar % Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  42. 42. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  43. 43. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  44. 44. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  45. 45. Demo Arduino + Android + external Speaker + TRRS-Plug LightSensor value encoded into Audio Signals Audio Signal decoded into Text and Progress-bar updated every .5 seconds Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  46. 46. Summary We built an embedded system that captured sensor data Encoded the data into ASCII and then into frequencies Generated tones for a predefined duration On an Android Phone, we received the Audio Signal via TRRS Cable Decoded the signal back into ASCII and then Numbers Displayed the Text and animated a progress bar every .5s Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved.
  47. 47. Copyright © 2011-2012, Wolf Paulus - http://wolfpaulus.com - A Tech Casita Production.All rights reserved. Thanks for coming http://wolfpaulus.com

×