Successfully reported this slideshow.
Your SlideShare is downloading. ×

Experimental Mobile Music - Pd/iOS Workshop

Ad

Experimental Mobile Music
Pd/iOS Workshop
Open Media Lab
School of Intermedia Art
China Academy of Art
May 23-31, 2013
Mas...

Ad

Introduction

Ad

akaMasayuki Akamatsu, Ph.D
● Media Artist
● Professor at IAMAS
● Fellow at some comapnies
Gocco., Soneru, Ma-tilde…
● Soun...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 100 Ad
1 of 100 Ad
Advertisement

More Related Content

Advertisement

Experimental Mobile Music - Pd/iOS Workshop

  1. 1. Experimental Mobile Music Pd/iOS Workshop Open Media Lab School of Intermedia Art China Academy of Art May 23-31, 2013 Masayuki Akamatsu / IAMAS
  2. 2. Introduction
  3. 3. akaMasayuki Akamatsu, Ph.D ● Media Artist ● Professor at IAMAS ● Fellow at some comapnies Gocco., Soneru, Ma-tilde… ● Sound, Visual, Network, Mobile… Real-time Performance Interactive Installation Mobile/Network Experience ● http://akamatsu.org/ 赤松 正行
  4. 4. akaIAMAS ● Institute of Advanced Media and Sciences ● Graduate school for new media arts 20 students x 2 years = 40 students 19 professors 24 hours / 364 days facilities Personal desks / Shared studios ● http://www.iamas.ac.jp/E/
  5. 5. akaincubator, 1999 ● Autonomous Distributed Cooperative System
  6. 6. akaTime Machine!, 2002 • Image analysis / Image Processing
  7. 7. akaEarthen Bodied Augur, 2007 ● Image/Audio Analysis and Processing
  8. 8. akaATOMxGALAXIES, 2007- ● Image Processing / Reversed Projection Mapping
  9. 9. akaGame Changer in 2007
  10. 10. akaComputer or Mobile ?
  11. 11. akaDinosaurs vs. Rats (Mammals)
  12. 12. akaMobile Information Environment ● Bodily Information Environment ● Sensory Information Environment ● Universal Information Environment ● World-wide Information Environment
  13. 13. akaMobile Apps
  14. 14. akaSekai Camera, 2008
  15. 15. akaSnowflakes, 2008
  16. 16. akaOkeanos Buoys, 2010
  17. 17. akaSync for Japan, 2011
  18. 18. akaUroboros Torch, 2012
  19. 19. akaMIDIWriter, 2012
  20. 20. ● Using App Store (on iOS) or iTunes (on OS X) ● Search Keywords “Masayuki Akamatsu” for my apps “ARART” for classic paintings “Bloom Card” for flowers akaTry Apps
  21. 21. akaWhat’s your favorite app ?
  22. 22. akaWhy create an app? ● Many excellent music apps ● Create your own music with your own idea with your own app Animoog by Moog Music Inc.
  23. 23. MobMuPlat:Basics
  24. 24. akaCreate a Music app with Xcode ● Xcode ● Objective-C ● Core Audio ● DSP algorithm ● Difficult!! Okeanos by Masayuki Akamatsu mini-composer by Karl Bartos and Masayuki Akamatsu
  25. 25. akaCreate a Music app with Pd ● MobMuPlat Pd + Editor No programming code Easy to develop Limited UI/Graphics Limited OS Access For Personal Use ● libpd / Pd for iOS Pd + Xcode Objective-C code Hard to develop Full UI/Graphics Full OS Access For App Store Distribution
  26. 26. akaQuestions: Do you have a skill of… ● Max ● Pd ● C/C++ ● iOS SDK
  27. 27. akaInstall MobMuPlat ● http://mobmuplat.com/ ● Install MobMuPlat on iOS device ● Install MobMuPlat distribution on Mac
  28. 28. akaMobMuPlat (iOS app)
  29. 29. akaMobMuPlat Workflow 1. Create an audio patch with Pd 2. Create a GUI with MobMuPlatEditor 3. Test them with PdWrapper.pd 4. Transfer them with iTunes 5. Play it with MobMuPlat (iOS) Mac iPhone
  30. 30. akaPd tips for Max users Run/Edit mode Audio On/Off Audio On/Off option+click Create an object
  31. 31. akaApp 1 - Drums Drums set by Pearl
  32. 32. akaStep 1: Read an Audio File ● Open Pd (on Mac) ● Open a new patch ● Create and connect objects ● Save the patch as “Drums.pd” into the same folder of the audio file ● Click the button You’ll see the waveform Object Message Button Array Object Patch Cord
  33. 33. akaStep 2: Play the Audio Buffer ● Create and connect objects ● Start DSP ● Click the button You’ll hear the sound
  34. 34. akaStep 3: Create a Communication ● Create and Connect Objects ● What’s “fromGUI” & “/claps”?
  35. 35. ● Open MobMuPlatEditor You’ll see a new document ● Open “Doc” tab ● Choose Screen Size ● Choose your Pd file ● Save the document as “Drums.mmp” akaStep 4: Create a GUI document 1 2 2
  36. 36. akaStep 5: Create a Button ● Open “Add” tab ● Click “button” ● Adjust the position Do not cover “i” button 2 3 Drag 1
  37. 37. akaStep 6: Set the Properties ● Open “Prop” tab ● Set the Address as “/claps” 1 2
  38. 38. akaStep 7: Test the GUI Pd ● Open “PdWrapper.pd” MobMuPlatEditor ● Open “Lock” tab ● Click the button You’ll hear the sound You’ll see the messages 1 2
  39. 39. akaStep 8: Transfer Files ● Connect iPhone to Mac via USB cable ● Open iTunes ● Choose your iPhone, Apps tab and MobMuPlat ● Put files into MobMuPlat Documents 1 2 3 4 Drag&Drop
  40. 40. akaStep 9: Hit your Drum ● Open MobMuPlat (on iPhone) ● Click “i” button ● Choose “Drums.mmp” ● Play!
  41. 41. akaExercises: Build your own Drums ● Add more buttons ● Use your own audio files ● Use a image Add a panel and choose a image file Make a button transparent ● Control the volume Play softly or loudly
  42. 42. ● Use “Tidy Up” in Edit menu in Pd ● .mmp & .pd are text files you can edit akaPd Editing Tips
  43. 43. MobMuPlat:Advanced
  44. 44. akaApp 2 - Theremin Léon Theremin and his invention “Theremin”
  45. 45. ● Receive a system message “fromSystem” and “/tilts” ● Convert values to pitch and volume Tilt X “-1.0~1.0” -> MIDI Note Number -> Frequency Tilt Y “-1.0~1.0” -> Volume ● Smoothing line ● Generate a sine waveform osc~ *~ dac~ akaTheremin Patch
  46. 46. akaTheremin GUI Tilt emulation Panel with a image
  47. 47. akaExercises: Build your own Synth ● Subtractive synthesis ● Additive synthesis ● Wavetable synthesis ● Frequency Modulation synthesis ● Phase Distortion synthesis ● Physical Modeling synthesis ● Sample-based synthesis MOOG Synthesizer
  48. 48. akaTutorial & Reference ● Programming Electronic Music in Pd http://www.pd-tutorial.com/english/index.html ● Pure Data Reference Card http://puredata.info/docs/manuals/pdrefcards
  49. 49. akaApp 3 - Delay Echoplex, a tape delay machine by Mike Battle
  50. 50. akaDelay Patch (incomplete) Input Volume Control Delay Feedback Input Volume Audio Input Audio Output
  51. 51. akaDelay GUI
  52. 52. ● Delay ● Reverb ● Distortion ● Overdrive ● Flanger ● Chorus ● Phaser ● Tremolo ● EQ ● Wah ● Pitch Shift akaExercises: Build your own Effector BOSS Compact Effectors
  53. 53. akaAudio I/O ● Headset Mic Earphone ● Breakout adapter ● Audio Interface ● Feedback is also fun!
  54. 54. akaApp 4 - Chorus The China National Chorus
  55. 55. akaChorus Patch (Standalone) Calculate Frequency Calculate Duration Produce Cycle Produce Envelop Generate Sine wave Modify Amplitude
  56. 56. akaChorus GUI
  57. 57. akaChorus Patch (Network) Send to Network Receive from Network Set GUI Values set value without output
  58. 58. akaStandalone vs. Network Standalone Network Your iPhone GUI Audio toNetwork fromNetwork Network Another iPhone GUI Audio toNetwork fromNetwork Your iPhone GUI Audio Another iPhone GUI Audio
  59. 59. ● Multicast (Broadcast) ● Port Number akaOSC (Open Sound Control)
  60. 60. ● Unison ● Harmonic ● Inharmonic ● Beating/Chorus ● Overtone ● Texture ● Additive Synthesis ● Granular Synthesis akaExercises: Build your own Harmony Harmonic (from Wikipedia)
  61. 61. akaNetwork(s) ● Natural Network Through the Air Physical Phenomenon ● Human Network Play Session Feelings / Rules ● Electronic Network LAN / The Internet Protocols
  62. 62. akaLimitations of MobMuPlat ● Pd-vanilla (not Pd-extended) ● Poor Audio functions ● Poor GUI elements ● Poor visual expressions ● Poor iOS accessibilities ● Can’t create a TRUE app ● But Fun and Easy to use! SpaceLab by Iglesia Intermedia
  63. 63. libpd/Pd for iOS
  64. 64. akalibpd ● http://libpd.cc ● Pd as Audio Engine + iOS SDK (Xcode)
  65. 65. akaInstall Xcode ● Open App Store (on Mac) ● Search “Xcode” ● Click “Free”
  66. 66. akaInstall pd-for-ios ● https://github.com/libpd/pd-for-ios ● Clone Repository Click “Clone in Mac” ● Update Submodules Follow “README.txt”
  67. 67. ● Open PdTest01.xcodeproj ● Choose Scheme ● Click “Run” akaRun Sample Projects 1 2
  68. 68. akaTrue App ;-) Test
  69. 69. akaTest Patch Receive a message from app (Objective-C code) Send a message to app (Objective-C code)
  70. 70. akaCreate a new Project ● Open Xcode ● Choose File > New > Project…
  71. 71. akaChoose a template 1 2
  72. 72. akaChoose options 1 2
  73. 73. akaSave the project 1 2
  74. 74. akaProject Window
  75. 75. akaCreate a User Interface 1 2 3 Interface Builder Drag&Drop Drag&Drop
  76. 76. akaTest the User Interface 1 2
  77. 77. akaAction and Outlet
  78. 78. akaCreate an Action control-drag 12 3 4 5 6 Code Editor
  79. 79. akaCreate an Outlet control-drag1 2 3
  80. 80. akaGenerated Codes #import <UIKit/UIKit.h> @interface TestViewController : UIViewController - (IBAction)changeFreq:(UISlider *)sender; @property (retain, nonatomic) IBOutlet UILabel *freqLabel; @end #import "TestViewController.h" @interface TestViewController () @end @implementation TestViewController - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. } - (void)didReceiveMemoryWarning { [super didReceiveMemoryWarning]; // Dispose of any resources that can be recreated. } - (IBAction)changeFreq:(UISlider *)sender { } - (void)dealloc { [_freqLabel release]; [super dealloc]; } @end TestViewController.h TestViewController.m
  81. 81. akaAdd Code and Run #import "TestViewController.h" @interface TestViewController () @end @implementation TestViewController - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. } - (void)didReceiveMemoryWarning { [super didReceiveMemoryWarning]; // Dispose of any resources that can be recreated. } - (IBAction)changeFreq:(UISlider *)sender { ! self.freqLabel.text = [NSString stringWithFormat:@"%f", sender.value]; } - (void)dealloc { [_freqLabel release]; [super dealloc]; } @end TestViewController.m
  82. 82. akaXcode Tips ● Snapshot
  83. 83. akaAdd libpd.xcodeproj Drag&Drop Close any other projects first because of workspaces problem
  84. 84. akaAdd Target Dependencies 1 2 3 5 4 7 6
  85. 85. akaAdd Libraries/Frameworks 2 3 4 Add libpd-ios.a AudioToolbox.framework and AVFoundation.framework 1
  86. 86. akaAdd Header Search Paths 1 4 Drag&Drop Double Click 3 Add libpd/objc and libpd/pure-data/src 2
  87. 87. akaAdd the Pd Patch 1Drag&Drop 2 3
  88. 88. akaAdd Codes to TestViewController.h #import <UIKit/UIKit.h> #import "PdBase.h" #import "PdAudioController.h" @interface TestViewController : UIViewController <PdReceiverDelegate> - (IBAction)changeFreq:(UISlider *)sender; @property (retain, nonatomic) IBOutlet UILabel *freqLabel; @property (strong, nonatomic) PdAudioController *audioController; @end
  89. 89. - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. ! // Create Audio Controller ! self.audioController = [[[PdAudioController alloc] init] autorelease]; ! [self.audioController configureAmbientWithSampleRate:44100 numberChannels:2 mixingEnabled:YES]; ! self.audioController.active = YES; ! ! // Open Pd Patch ! [PdBase openFile:@"Test.pd" path:[[NSBundle mainBundle] bundlePath]]; ! ! // Set PdBase Delegate ! PdBase.delegate = self;! ! // Subscribe messages from Pd ! [PdBase subscribe:@"freqInHz"]; } - (IBAction)changeFreq:(UISlider *)sender { ! // Send a message to Pd ! [PdBase sendFloat:sender.value toReceiver: @"freq" ]; } // Handle float messages from Pd - (void)receiveFloat:(float)received fromSource:(NSString *)source { ! if ([source isEqualToString:@"freqInHz"]) { ! ! self.freqLabel.text = [NSString stringWithFormat:@"%f Hz", received]; ! } } akaAdd Codes to TestViewController.m
  90. 90. akaRun the True App !
  91. 91. akaUser Interface - Code - Patch - (IBAction)changeFreq:(UISlider *)sender { ! [PdBase sendFloat:sender.value toReceiver: @"freq" ]; } - (void)receiveFloat:(float)received fromSource:(NSString *)source { ! if ([source isEqualToString:@"freqInHz"]) { ! ! self.freqLabel.text = [NSString stringWithFormat:@"%f", received]; ! } } Action Outlet Receive method (Delegate) Send method
  92. 92. akalibpd Project Summary ● Add libpd.xcodeproj ● Add Target Dependencies libpd-ios ● Add Libraries/Frameworks libpd-ios.a, AudioToolbox.framework, AVFoundation.framework ● Add Header Search Paths libpd/objc, libpd/pure-data/src ● Add Pd Patch ● Add Codes Create PdAudioController, Set it up and Make it active Open Pd patch Set PdBase delegate and Subscribe Pd messages Send messages to Pd patch and Receive messages from Pd patch
  93. 93. akaFurther References ● Pd for iOS wiki https://github.com/libpd/pd-for-ios/wiki/ios ● libpd Objective-C API https://github.com/libpd/libpd/wiki/objc ● iOSの教科書 http://iosbook.net/
  94. 94. Students Project
  95. 95. ● Solo or Group ● 3 minutes ● Theme/Motif/Feeling ● Audio Processing ● User Interface ● Composition ● Performance akaCreate your own music Kraftwerk / Pocket Calculator
  96. 96. akaWhy? (again) ● Many excellent music apps ● Create your own music with your own idea with your own app Animoog by Moog Music Inc.
  97. 97. akaMake a Documentation ● Format PDF Single Page Landscape ● Contents Title Name Description Images (Screen, Photo, etc.) Title: China My Love Name: Masayuki Akamatsu Description: “China My Love” is a simple app that transmits MIDI note messages by hitting keys on a keyboard. Playing an input phrase as a sequence is also possible. The tempo and the loop can be set. You can use either the on-screen software keyboard or a Bluetooth/Dock-connected hardware keyboard for iOS.
  98. 98. One more thing…
  99. 99. akaiDonation Unneeded iPhone, iPad, iPod touch, I am asking for a kind donation http://akamatsu.org/aka/ios/idonation/
  100. 100. 謝謝 Thanks to Prof. Dajuin Yao and students

×