SlideShare a Scribd company logo
Experimental Mobile Music
Pd/iOS Workshop
Open Media Lab
School of Intermedia Art
China Academy of Art
May 23-31, 2013
Masayuki Akamatsu / IAMAS
Introduction
akaMasayuki Akamatsu, Ph.D
● Media Artist
● Professor at IAMAS
● Fellow at some comapnies
Gocco., Soneru, Ma-tilde…
● Sound, Visual, Network, Mobile…
Real-time Performance
Interactive Installation
Mobile/Network Experience
● http://akamatsu.org/
赤松 正行
akaIAMAS
● Institute of Advanced Media and Sciences
● Graduate school for new media arts
20 students x 2 years = 40 students
19 professors
24 hours / 364 days facilities
Personal desks / Shared studios
● http://www.iamas.ac.jp/E/
akaincubator, 1999
● Autonomous Distributed Cooperative System
akaTime Machine!, 2002
• Image analysis / Image Processing
akaEarthen Bodied Augur, 2007
● Image/Audio Analysis and Processing
akaATOMxGALAXIES, 2007-
● Image Processing / Reversed Projection Mapping
akaGame Changer in 2007
akaComputer or Mobile ?
akaDinosaurs vs. Rats (Mammals)
akaMobile Information Environment
● Bodily Information Environment
● Sensory Information Environment
● Universal Information Environment
● World-wide Information Environment
akaMobile Apps
akaSekai Camera, 2008
akaSnowflakes, 2008
akaOkeanos Buoys, 2010
akaSync for Japan, 2011
akaUroboros Torch, 2012
akaMIDIWriter, 2012
● Using App Store (on iOS) or iTunes (on OS X)
● Search Keywords
“Masayuki Akamatsu” for my apps
“ARART” for classic paintings
“Bloom Card” for flowers
akaTry Apps
akaWhat’s your favorite app ?
akaWhy create an app?
● Many excellent music apps
● Create your own music
with your own idea
with your own app
Animoog by Moog Music Inc.
MobMuPlat:Basics
akaCreate a Music app with Xcode
● Xcode
● Objective-C
● Core Audio
● DSP algorithm
● Difficult!!
Okeanos by Masayuki Akamatsu
mini-composer by Karl Bartos and Masayuki Akamatsu
akaCreate a Music app with Pd
● MobMuPlat
Pd + Editor
No programming code
Easy to develop
Limited UI/Graphics
Limited OS Access
For Personal Use
● libpd / Pd for iOS
Pd + Xcode
Objective-C code
Hard to develop
Full UI/Graphics
Full OS Access
For App Store Distribution
akaQuestions: Do you have a skill of…
● Max
● Pd
● C/C++
● iOS SDK
akaInstall MobMuPlat
● http://mobmuplat.com/
● Install MobMuPlat on iOS device
● Install MobMuPlat distribution on Mac
akaMobMuPlat (iOS app)
akaMobMuPlat Workflow
1. Create an audio patch with Pd
2. Create a GUI with MobMuPlatEditor
3. Test them with PdWrapper.pd
4. Transfer them with iTunes
5. Play it with MobMuPlat (iOS)
Mac iPhone
akaPd tips for Max users
Run/Edit mode
Audio On/Off
Audio On/Off
option+click
Create an object
akaApp 1 - Drums
Drums set by Pearl
akaStep 1: Read an Audio File
● Open Pd (on Mac)
● Open a new patch
● Create and connect objects
● Save the patch as “Drums.pd”
into the same folder of the audio file
● Click the button
You’ll see the waveform
Object
Message
Button
Array
Object
Patch Cord
akaStep 2: Play the Audio Buffer
● Create and connect objects
● Start DSP
● Click the button
You’ll hear the sound
akaStep 3: Create a Communication
● Create and Connect Objects
● What’s “fromGUI” & “/claps”?
● Open MobMuPlatEditor
You’ll see a new document
● Open “Doc” tab
● Choose Screen Size
● Choose your Pd file
● Save the document
as “Drums.mmp”
akaStep 4: Create a GUI document
1
2
2
akaStep 5: Create a Button
● Open “Add” tab
● Click “button”
● Adjust the position
Do not cover “i” button
2
3 Drag
1
akaStep 6: Set the Properties
● Open “Prop” tab
● Set the Address
as “/claps”
1
2
akaStep 7: Test the GUI
Pd
● Open “PdWrapper.pd”
MobMuPlatEditor
● Open “Lock” tab
● Click the button
You’ll hear the sound
You’ll see the messages
1
2
akaStep 8: Transfer Files
● Connect iPhone to Mac via USB cable
● Open iTunes
● Choose your iPhone, Apps tab and MobMuPlat
● Put files into MobMuPlat Documents
1
2
3
4
Drag&Drop
akaStep 9: Hit your Drum
● Open MobMuPlat (on iPhone)
● Click “i” button
● Choose “Drums.mmp”
● Play!
akaExercises: Build your own Drums
● Add more buttons
● Use your own audio files
● Use a image
Add a panel and choose a image file
Make a button transparent
● Control the volume
Play softly or loudly
● Use “Tidy Up” in Edit menu in Pd
● .mmp & .pd are text files you can edit
akaPd Editing Tips
MobMuPlat:Advanced
akaApp 2 - Theremin
Léon Theremin and his invention “Theremin”
● Receive a system message
“fromSystem” and “/tilts”
● Convert values to pitch and volume
Tilt X “-1.0~1.0” -> MIDI Note Number -> Frequency
Tilt Y “-1.0~1.0” -> Volume
● Smoothing
line
● Generate a sine waveform
osc~
*~
dac~
akaTheremin Patch
akaTheremin GUI
Tilt emulation
Panel with a image
akaExercises: Build your own Synth
● Subtractive synthesis
● Additive synthesis
● Wavetable synthesis
● Frequency Modulation synthesis
● Phase Distortion synthesis
● Physical Modeling synthesis
● Sample-based synthesis
MOOG Synthesizer
akaTutorial & Reference
● Programming Electronic Music in Pd
http://www.pd-tutorial.com/english/index.html
● Pure Data Reference Card
http://puredata.info/docs/manuals/pdrefcards
akaApp 3 - Delay
Echoplex, a tape delay machine by Mike Battle
akaDelay Patch (incomplete)
Input Volume Control
Delay Feedback
Input Volume
Audio Input
Audio Output
akaDelay GUI
● Delay
● Reverb
● Distortion
● Overdrive
● Flanger
● Chorus
● Phaser
● Tremolo
● EQ
● Wah
● Pitch Shift
akaExercises: Build your own Effector
BOSS Compact Effectors
akaAudio I/O
● Headset Mic Earphone
● Breakout adapter
● Audio Interface
● Feedback is also fun!
akaApp 4 - Chorus
The China National Chorus
akaChorus Patch (Standalone)
Calculate Frequency Calculate Duration
Produce Cycle
Produce Envelop
Generate Sine wave
Modify Amplitude
akaChorus GUI
akaChorus Patch (Network)
Send to Network Receive from Network
Set GUI Values
set value without output
akaStandalone vs. Network
Standalone
Network
Your iPhone
GUI Audio
toNetwork fromNetwork
Network
Another iPhone
GUI Audio
toNetwork fromNetwork
Your iPhone
GUI Audio
Another iPhone
GUI Audio
● Multicast (Broadcast)
● Port Number
akaOSC (Open Sound Control)
● Unison
● Harmonic
● Inharmonic
● Beating/Chorus
● Overtone
● Texture
● Additive Synthesis
● Granular Synthesis
akaExercises: Build your own Harmony
Harmonic (from Wikipedia)
akaNetwork(s)
● Natural Network
Through the Air
Physical Phenomenon
● Human Network
Play Session
Feelings / Rules
● Electronic Network
LAN / The Internet
Protocols
akaLimitations of MobMuPlat
● Pd-vanilla (not Pd-extended)
● Poor Audio functions
● Poor GUI elements
● Poor visual expressions
● Poor iOS accessibilities
● Can’t create a TRUE app
● But Fun and Easy to use!
SpaceLab by Iglesia Intermedia
libpd/Pd for iOS
akalibpd
● http://libpd.cc
● Pd as Audio Engine + iOS SDK (Xcode)
akaInstall Xcode
● Open App Store (on Mac)
● Search “Xcode”
● Click “Free”
akaInstall pd-for-ios
● https://github.com/libpd/pd-for-ios
● Clone Repository
Click “Clone in Mac”
● Update Submodules
Follow “README.txt”
● Open PdTest01.xcodeproj
● Choose Scheme
● Click “Run”
akaRun Sample Projects
1
2
akaTrue App ;-) Test
akaTest Patch
Receive a message
from app (Objective-C code)
Send a message
to app (Objective-C code)
akaCreate a new Project
● Open Xcode
● Choose File > New > Project…
akaChoose a template
1
2
akaChoose options
1
2
akaSave the project
1
2
akaProject Window
akaCreate a User Interface
1
2
3
Interface Builder
Drag&Drop
Drag&Drop
akaTest the User Interface
1
2
akaAction and Outlet
akaCreate an Action
control-drag
12
3
4
5
6
Code Editor
akaCreate an Outlet
control-drag1 2
3
akaGenerated Codes
#import <UIKit/UIKit.h>
@interface TestViewController : UIViewController
- (IBAction)changeFreq:(UISlider *)sender;
@property (retain, nonatomic) IBOutlet UILabel *freqLabel;
@end
#import "TestViewController.h"
@interface TestViewController ()
@end
@implementation TestViewController
- (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view,
typically from a nib.
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
// Dispose of any resources that can be recreated.
}
- (IBAction)changeFreq:(UISlider *)sender {
}
- (void)dealloc {
[_freqLabel release];
[super dealloc];
}
@end
TestViewController.h TestViewController.m
akaAdd Code and Run
#import "TestViewController.h"
@interface TestViewController ()
@end
@implementation TestViewController
- (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view, typically from a nib.
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
// Dispose of any resources that can be recreated.
}
- (IBAction)changeFreq:(UISlider *)sender {
! self.freqLabel.text = [NSString stringWithFormat:@"%f", sender.value];
}
- (void)dealloc {
[_freqLabel release];
[super dealloc];
}
@end
TestViewController.m
akaXcode Tips
● Snapshot
akaAdd libpd.xcodeproj
Drag&Drop
Close any other projects first because of workspaces problem
akaAdd Target Dependencies
1
2
3
5
4
7
6
akaAdd Libraries/Frameworks
2
3
4
Add libpd-ios.a
AudioToolbox.framework
and AVFoundation.framework
1
akaAdd Header Search Paths
1
4
Drag&Drop
Double Click
3
Add libpd/objc
and libpd/pure-data/src
2
akaAdd the Pd Patch
1Drag&Drop
2
3
akaAdd Codes to TestViewController.h
#import <UIKit/UIKit.h>
#import "PdBase.h"
#import "PdAudioController.h"
@interface TestViewController : UIViewController <PdReceiverDelegate>
- (IBAction)changeFreq:(UISlider *)sender;
@property (retain, nonatomic) IBOutlet UILabel *freqLabel;
@property (strong, nonatomic) PdAudioController *audioController;
@end
- (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view, typically from a nib.
! // Create Audio Controller
! self.audioController = [[[PdAudioController alloc] init] autorelease];
! [self.audioController configureAmbientWithSampleRate:44100 numberChannels:2 mixingEnabled:YES];
! self.audioController.active = YES;
!
! // Open Pd Patch
! [PdBase openFile:@"Test.pd" path:[[NSBundle mainBundle] bundlePath]];
!
! // Set PdBase Delegate
! PdBase.delegate = self;!
! // Subscribe messages from Pd
! [PdBase subscribe:@"freqInHz"];
}
- (IBAction)changeFreq:(UISlider *)sender {
! // Send a message to Pd
! [PdBase sendFloat:sender.value toReceiver: @"freq" ];
}
// Handle float messages from Pd
- (void)receiveFloat:(float)received fromSource:(NSString *)source {
! if ([source isEqualToString:@"freqInHz"]) {
! ! self.freqLabel.text = [NSString stringWithFormat:@"%f Hz", received];
! }
}
akaAdd Codes to TestViewController.m
akaRun the True App !
akaUser Interface - Code - Patch
- (IBAction)changeFreq:(UISlider *)sender {
! [PdBase sendFloat:sender.value toReceiver: @"freq" ];
}
- (void)receiveFloat:(float)received fromSource:(NSString *)source {
! if ([source isEqualToString:@"freqInHz"]) {
! ! self.freqLabel.text = [NSString stringWithFormat:@"%f", received];
! }
}
Action
Outlet
Receive method
(Delegate)
Send method
akalibpd Project Summary
● Add libpd.xcodeproj
● Add Target Dependencies
libpd-ios
● Add Libraries/Frameworks
libpd-ios.a, AudioToolbox.framework, AVFoundation.framework
● Add Header Search Paths
libpd/objc, libpd/pure-data/src
● Add Pd Patch
● Add Codes
Create PdAudioController, Set it up and Make it active
Open Pd patch
Set PdBase delegate and Subscribe Pd messages
Send messages to Pd patch and Receive messages from Pd patch
akaFurther References
● Pd for iOS wiki
https://github.com/libpd/pd-for-ios/wiki/ios
● libpd Objective-C API
https://github.com/libpd/libpd/wiki/objc
● iOSの教科書
http://iosbook.net/
Students Project
● Solo or Group
● 3 minutes
● Theme/Motif/Feeling
● Audio Processing
● User Interface
● Composition
● Performance
akaCreate your own music
Kraftwerk / Pocket Calculator
akaWhy? (again)
● Many excellent music apps
● Create your own music
with your own idea
with your own app
Animoog by Moog Music Inc.
akaMake a Documentation
● Format
PDF
Single Page
Landscape
● Contents
Title
Name
Description
Images (Screen, Photo, etc.)
Title: China My Love
Name: Masayuki Akamatsu
Description: “China My Love” is a simple app that transmits MIDI note
messages by hitting keys on a keyboard. Playing an input phrase as a
sequence is also possible. The tempo and the loop can be set. You can
use either the on-screen software keyboard or a Bluetooth/Dock-connected
hardware keyboard for iOS.
One more thing…
akaiDonation
Unneeded iPhone, iPad, iPod touch,
I am asking for a kind donation
http://akamatsu.org/aka/ios/idonation/
謝謝
Thanks to Prof. Dajuin Yao and students

More Related Content

Similar to Experimental Mobile Music - Pd/iOS Workshop

Ubiquitous Content Symposium 2009
Ubiquitous Content Symposium 2009Ubiquitous Content Symposium 2009
Ubiquitous Content Symposium 2009
Shigeru Kobayashi
 
parellel computing
parellel computingparellel computing
parellel computing
katakdound
 
Grade Eight_Quarter 1_Week Four_Day 1.pptx
Grade Eight_Quarter 1_Week Four_Day 1.pptxGrade Eight_Quarter 1_Week Four_Day 1.pptx
Grade Eight_Quarter 1_Week Four_Day 1.pptx
RandyGaray
 
Ig2 task 1 work sheet connor martin
Ig2 task 1 work sheet connor martinIg2 task 1 work sheet connor martin
Ig2 task 1 work sheet connor martin
Kalen612
 
Arduino Development For Beginners
Arduino Development For BeginnersArduino Development For Beginners
Arduino Development For Beginners
FTS seminar
 
How to Use OpenMP on Native Activity
How to Use OpenMP on Native ActivityHow to Use OpenMP on Native Activity
Python-in-Embedded-systems.pptx
Python-in-Embedded-systems.pptxPython-in-Embedded-systems.pptx
Python-in-Embedded-systems.pptx
TuynLCh
 
SLA Captivate
SLA CaptivateSLA Captivate
SLA Captivate
Edward Metz
 
Michael williamsig2task1worksheet
Michael williamsig2task1worksheetMichael williamsig2task1worksheet
Michael williamsig2task1worksheet
Hooaax
 
Exploring the Internet of Things Using Ruby
Exploring the Internet of Things Using RubyExploring the Internet of Things Using Ruby
Exploring the Internet of Things Using Ruby
Mike Hagedorn
 
IG2 Task 1
IG2 Task 1 IG2 Task 1
IG2 Task 1
crookedpug
 
Livecoding with impromptu
Livecoding with impromptuLivecoding with impromptu
Livecoding with impromptu
Michele Pasin
 
Movie maker audacity - multimedia , summer camp 2019-2020
Movie maker   audacity - multimedia , summer camp 2019-2020Movie maker   audacity - multimedia , summer camp 2019-2020
Movie maker audacity - multimedia , summer camp 2019-2020
Osama Ghandour Geris
 
Absolute Beginners Guide to iPhone dev
Absolute Beginners Guide to iPhone devAbsolute Beginners Guide to iPhone dev
Absolute Beginners Guide to iPhone dev
Barry Ezell
 
Introduction to jupyter pandas and matplotlib
Introduction to jupyter pandas and matplotlibIntroduction to jupyter pandas and matplotlib
Introduction to jupyter pandas and matplotlib
tlsghalbi
 
Building an Open Source iOS app: lessons learned
Building an Open Source iOS app: lessons learnedBuilding an Open Source iOS app: lessons learned
Building an Open Source iOS app: lessons learned
Wojciech Koszek
 
Your first Elm program
Your first Elm programYour first Elm program
Your first Elm program
Per Lundholm
 
Bringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with ElectronBringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with Electron
Nir Noy
 
Sketching in Code
Sketching in CodeSketching in Code
Sketching in Code
Marcin Ignac
 
The Study For A Sound Engineering And Recording Class
The Study For A Sound Engineering And Recording ClassThe Study For A Sound Engineering And Recording Class
The Study For A Sound Engineering And Recording Class
Reggie621
 

Similar to Experimental Mobile Music - Pd/iOS Workshop (20)

Ubiquitous Content Symposium 2009
Ubiquitous Content Symposium 2009Ubiquitous Content Symposium 2009
Ubiquitous Content Symposium 2009
 
parellel computing
parellel computingparellel computing
parellel computing
 
Grade Eight_Quarter 1_Week Four_Day 1.pptx
Grade Eight_Quarter 1_Week Four_Day 1.pptxGrade Eight_Quarter 1_Week Four_Day 1.pptx
Grade Eight_Quarter 1_Week Four_Day 1.pptx
 
Ig2 task 1 work sheet connor martin
Ig2 task 1 work sheet connor martinIg2 task 1 work sheet connor martin
Ig2 task 1 work sheet connor martin
 
Arduino Development For Beginners
Arduino Development For BeginnersArduino Development For Beginners
Arduino Development For Beginners
 
How to Use OpenMP on Native Activity
How to Use OpenMP on Native ActivityHow to Use OpenMP on Native Activity
How to Use OpenMP on Native Activity
 
Python-in-Embedded-systems.pptx
Python-in-Embedded-systems.pptxPython-in-Embedded-systems.pptx
Python-in-Embedded-systems.pptx
 
SLA Captivate
SLA CaptivateSLA Captivate
SLA Captivate
 
Michael williamsig2task1worksheet
Michael williamsig2task1worksheetMichael williamsig2task1worksheet
Michael williamsig2task1worksheet
 
Exploring the Internet of Things Using Ruby
Exploring the Internet of Things Using RubyExploring the Internet of Things Using Ruby
Exploring the Internet of Things Using Ruby
 
IG2 Task 1
IG2 Task 1 IG2 Task 1
IG2 Task 1
 
Livecoding with impromptu
Livecoding with impromptuLivecoding with impromptu
Livecoding with impromptu
 
Movie maker audacity - multimedia , summer camp 2019-2020
Movie maker   audacity - multimedia , summer camp 2019-2020Movie maker   audacity - multimedia , summer camp 2019-2020
Movie maker audacity - multimedia , summer camp 2019-2020
 
Absolute Beginners Guide to iPhone dev
Absolute Beginners Guide to iPhone devAbsolute Beginners Guide to iPhone dev
Absolute Beginners Guide to iPhone dev
 
Introduction to jupyter pandas and matplotlib
Introduction to jupyter pandas and matplotlibIntroduction to jupyter pandas and matplotlib
Introduction to jupyter pandas and matplotlib
 
Building an Open Source iOS app: lessons learned
Building an Open Source iOS app: lessons learnedBuilding an Open Source iOS app: lessons learned
Building an Open Source iOS app: lessons learned
 
Your first Elm program
Your first Elm programYour first Elm program
Your first Elm program
 
Bringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with ElectronBringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with Electron
 
Sketching in Code
Sketching in CodeSketching in Code
Sketching in Code
 
The Study For A Sound Engineering And Recording Class
The Study For A Sound Engineering And Recording ClassThe Study For A Sound Engineering And Recording Class
The Study For A Sound Engineering And Recording Class
 

More from Masayuki Akamatsu

Artistic Imaging in the Age of Mobility
Artistic Imaging in the Age of MobilityArtistic Imaging in the Age of Mobility
Artistic Imaging in the Age of Mobility
Masayuki Akamatsu
 
Bicycle in the UK
 英国自転車調査報告
Bicycle in the UK
 英国自転車調査報告Bicycle in the UK
 英国自転車調査報告
Bicycle in the UK
 英国自転車調査報告
Masayuki Akamatsu
 
掌上のリアリティ AR+ART=ARART
掌上のリアリティ AR+ART=ARART掌上のリアリティ AR+ART=ARART
掌上のリアリティ AR+ART=ARART
Masayuki Akamatsu
 
Critical Cycling
Critical CyclingCritical Cycling
Critical Cycling
Masayuki Akamatsu
 
Bike Sharing System
Bike Sharing SystemBike Sharing System
Bike Sharing System
Masayuki Akamatsu
 
The Art of Life
The Art of LifeThe Art of Life
The Art of Life
Masayuki Akamatsu
 
Alternated Reality and Eternity
Alternated Reality and EternityAlternated Reality and Eternity
Alternated Reality and Eternity
Masayuki Akamatsu
 
チャリンフォメーション
チャリンフォメーションチャリンフォメーション
チャリンフォメーションMasayuki Akamatsu
 
ウロボロスのトーチ〜拡張現実と変容現実
ウロボロスのトーチ〜拡張現実と変容現実ウロボロスのトーチ〜拡張現実と変容現実
ウロボロスのトーチ〜拡張現実と変容現実Masayuki Akamatsu
 
新しい時空間における 表現研究プロジェクト〜akaプロジェクト
新しい時空間における 表現研究プロジェクト〜akaプロジェクト新しい時空間における 表現研究プロジェクト〜akaプロジェクト
新しい時空間における 表現研究プロジェクト〜akaプロジェクトMasayuki Akamatsu
 
スマートフォンITSがもたらす世界〜コンシューマの視点から
スマートフォンITSがもたらす世界〜コンシューマの視点からスマートフォンITSがもたらす世界〜コンシューマの視点から
スマートフォンITSがもたらす世界〜コンシューマの視点からMasayuki Akamatsu
 

More from Masayuki Akamatsu (12)

Artistic Imaging in the Age of Mobility
Artistic Imaging in the Age of MobilityArtistic Imaging in the Age of Mobility
Artistic Imaging in the Age of Mobility
 
Bicycle in the UK
 英国自転車調査報告
Bicycle in the UK
 英国自転車調査報告Bicycle in the UK
 英国自転車調査報告
Bicycle in the UK
 英国自転車調査報告
 
掌上のリアリティ AR+ART=ARART
掌上のリアリティ AR+ART=ARART掌上のリアリティ AR+ART=ARART
掌上のリアリティ AR+ART=ARART
 
Critical Cycling
Critical CyclingCritical Cycling
Critical Cycling
 
Bike Sharing System
Bike Sharing SystemBike Sharing System
Bike Sharing System
 
The Art of Life
The Art of LifeThe Art of Life
The Art of Life
 
Alternated Reality and Eternity
Alternated Reality and EternityAlternated Reality and Eternity
Alternated Reality and Eternity
 
チャリンフォメーション
チャリンフォメーションチャリンフォメーション
チャリンフォメーション
 
ウロボロスのトーチ〜拡張現実と変容現実
ウロボロスのトーチ〜拡張現実と変容現実ウロボロスのトーチ〜拡張現実と変容現実
ウロボロスのトーチ〜拡張現実と変容現実
 
ウロボロスのコーチ
ウロボロスのコーチウロボロスのコーチ
ウロボロスのコーチ
 
新しい時空間における 表現研究プロジェクト〜akaプロジェクト
新しい時空間における 表現研究プロジェクト〜akaプロジェクト新しい時空間における 表現研究プロジェクト〜akaプロジェクト
新しい時空間における 表現研究プロジェクト〜akaプロジェクト
 
スマートフォンITSがもたらす世界〜コンシューマの視点から
スマートフォンITSがもたらす世界〜コンシューマの視点からスマートフォンITSがもたらす世界〜コンシューマの視点から
スマートフォンITSがもたらす世界〜コンシューマの視点から
 

Recently uploaded

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 

Recently uploaded (20)

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 

Experimental Mobile Music - Pd/iOS Workshop

  • 1. Experimental Mobile Music Pd/iOS Workshop Open Media Lab School of Intermedia Art China Academy of Art May 23-31, 2013 Masayuki Akamatsu / IAMAS
  • 3. akaMasayuki Akamatsu, Ph.D ● Media Artist ● Professor at IAMAS ● Fellow at some comapnies Gocco., Soneru, Ma-tilde… ● Sound, Visual, Network, Mobile… Real-time Performance Interactive Installation Mobile/Network Experience ● http://akamatsu.org/ 赤松 正行
  • 4. akaIAMAS ● Institute of Advanced Media and Sciences ● Graduate school for new media arts 20 students x 2 years = 40 students 19 professors 24 hours / 364 days facilities Personal desks / Shared studios ● http://www.iamas.ac.jp/E/
  • 5. akaincubator, 1999 ● Autonomous Distributed Cooperative System
  • 6. akaTime Machine!, 2002 • Image analysis / Image Processing
  • 7. akaEarthen Bodied Augur, 2007 ● Image/Audio Analysis and Processing
  • 8. akaATOMxGALAXIES, 2007- ● Image Processing / Reversed Projection Mapping
  • 12. akaMobile Information Environment ● Bodily Information Environment ● Sensory Information Environment ● Universal Information Environment ● World-wide Information Environment
  • 20. ● Using App Store (on iOS) or iTunes (on OS X) ● Search Keywords “Masayuki Akamatsu” for my apps “ARART” for classic paintings “Bloom Card” for flowers akaTry Apps
  • 22. akaWhy create an app? ● Many excellent music apps ● Create your own music with your own idea with your own app Animoog by Moog Music Inc.
  • 24. akaCreate a Music app with Xcode ● Xcode ● Objective-C ● Core Audio ● DSP algorithm ● Difficult!! Okeanos by Masayuki Akamatsu mini-composer by Karl Bartos and Masayuki Akamatsu
  • 25. akaCreate a Music app with Pd ● MobMuPlat Pd + Editor No programming code Easy to develop Limited UI/Graphics Limited OS Access For Personal Use ● libpd / Pd for iOS Pd + Xcode Objective-C code Hard to develop Full UI/Graphics Full OS Access For App Store Distribution
  • 26. akaQuestions: Do you have a skill of… ● Max ● Pd ● C/C++ ● iOS SDK
  • 27. akaInstall MobMuPlat ● http://mobmuplat.com/ ● Install MobMuPlat on iOS device ● Install MobMuPlat distribution on Mac
  • 29. akaMobMuPlat Workflow 1. Create an audio patch with Pd 2. Create a GUI with MobMuPlatEditor 3. Test them with PdWrapper.pd 4. Transfer them with iTunes 5. Play it with MobMuPlat (iOS) Mac iPhone
  • 30. akaPd tips for Max users Run/Edit mode Audio On/Off Audio On/Off option+click Create an object
  • 31. akaApp 1 - Drums Drums set by Pearl
  • 32. akaStep 1: Read an Audio File ● Open Pd (on Mac) ● Open a new patch ● Create and connect objects ● Save the patch as “Drums.pd” into the same folder of the audio file ● Click the button You’ll see the waveform Object Message Button Array Object Patch Cord
  • 33. akaStep 2: Play the Audio Buffer ● Create and connect objects ● Start DSP ● Click the button You’ll hear the sound
  • 34. akaStep 3: Create a Communication ● Create and Connect Objects ● What’s “fromGUI” & “/claps”?
  • 35. ● Open MobMuPlatEditor You’ll see a new document ● Open “Doc” tab ● Choose Screen Size ● Choose your Pd file ● Save the document as “Drums.mmp” akaStep 4: Create a GUI document 1 2 2
  • 36. akaStep 5: Create a Button ● Open “Add” tab ● Click “button” ● Adjust the position Do not cover “i” button 2 3 Drag 1
  • 37. akaStep 6: Set the Properties ● Open “Prop” tab ● Set the Address as “/claps” 1 2
  • 38. akaStep 7: Test the GUI Pd ● Open “PdWrapper.pd” MobMuPlatEditor ● Open “Lock” tab ● Click the button You’ll hear the sound You’ll see the messages 1 2
  • 39. akaStep 8: Transfer Files ● Connect iPhone to Mac via USB cable ● Open iTunes ● Choose your iPhone, Apps tab and MobMuPlat ● Put files into MobMuPlat Documents 1 2 3 4 Drag&Drop
  • 40. akaStep 9: Hit your Drum ● Open MobMuPlat (on iPhone) ● Click “i” button ● Choose “Drums.mmp” ● Play!
  • 41. akaExercises: Build your own Drums ● Add more buttons ● Use your own audio files ● Use a image Add a panel and choose a image file Make a button transparent ● Control the volume Play softly or loudly
  • 42. ● Use “Tidy Up” in Edit menu in Pd ● .mmp & .pd are text files you can edit akaPd Editing Tips
  • 44. akaApp 2 - Theremin Léon Theremin and his invention “Theremin”
  • 45. ● Receive a system message “fromSystem” and “/tilts” ● Convert values to pitch and volume Tilt X “-1.0~1.0” -> MIDI Note Number -> Frequency Tilt Y “-1.0~1.0” -> Volume ● Smoothing line ● Generate a sine waveform osc~ *~ dac~ akaTheremin Patch
  • 47. akaExercises: Build your own Synth ● Subtractive synthesis ● Additive synthesis ● Wavetable synthesis ● Frequency Modulation synthesis ● Phase Distortion synthesis ● Physical Modeling synthesis ● Sample-based synthesis MOOG Synthesizer
  • 48. akaTutorial & Reference ● Programming Electronic Music in Pd http://www.pd-tutorial.com/english/index.html ● Pure Data Reference Card http://puredata.info/docs/manuals/pdrefcards
  • 49. akaApp 3 - Delay Echoplex, a tape delay machine by Mike Battle
  • 50. akaDelay Patch (incomplete) Input Volume Control Delay Feedback Input Volume Audio Input Audio Output
  • 52. ● Delay ● Reverb ● Distortion ● Overdrive ● Flanger ● Chorus ● Phaser ● Tremolo ● EQ ● Wah ● Pitch Shift akaExercises: Build your own Effector BOSS Compact Effectors
  • 53. akaAudio I/O ● Headset Mic Earphone ● Breakout adapter ● Audio Interface ● Feedback is also fun!
  • 54. akaApp 4 - Chorus The China National Chorus
  • 55. akaChorus Patch (Standalone) Calculate Frequency Calculate Duration Produce Cycle Produce Envelop Generate Sine wave Modify Amplitude
  • 57. akaChorus Patch (Network) Send to Network Receive from Network Set GUI Values set value without output
  • 58. akaStandalone vs. Network Standalone Network Your iPhone GUI Audio toNetwork fromNetwork Network Another iPhone GUI Audio toNetwork fromNetwork Your iPhone GUI Audio Another iPhone GUI Audio
  • 59. ● Multicast (Broadcast) ● Port Number akaOSC (Open Sound Control)
  • 60. ● Unison ● Harmonic ● Inharmonic ● Beating/Chorus ● Overtone ● Texture ● Additive Synthesis ● Granular Synthesis akaExercises: Build your own Harmony Harmonic (from Wikipedia)
  • 61. akaNetwork(s) ● Natural Network Through the Air Physical Phenomenon ● Human Network Play Session Feelings / Rules ● Electronic Network LAN / The Internet Protocols
  • 62. akaLimitations of MobMuPlat ● Pd-vanilla (not Pd-extended) ● Poor Audio functions ● Poor GUI elements ● Poor visual expressions ● Poor iOS accessibilities ● Can’t create a TRUE app ● But Fun and Easy to use! SpaceLab by Iglesia Intermedia
  • 64. akalibpd ● http://libpd.cc ● Pd as Audio Engine + iOS SDK (Xcode)
  • 65. akaInstall Xcode ● Open App Store (on Mac) ● Search “Xcode” ● Click “Free”
  • 66. akaInstall pd-for-ios ● https://github.com/libpd/pd-for-ios ● Clone Repository Click “Clone in Mac” ● Update Submodules Follow “README.txt”
  • 67. ● Open PdTest01.xcodeproj ● Choose Scheme ● Click “Run” akaRun Sample Projects 1 2
  • 69. akaTest Patch Receive a message from app (Objective-C code) Send a message to app (Objective-C code)
  • 70. akaCreate a new Project ● Open Xcode ● Choose File > New > Project…
  • 75. akaCreate a User Interface 1 2 3 Interface Builder Drag&Drop Drag&Drop
  • 76. akaTest the User Interface 1 2
  • 80. akaGenerated Codes #import <UIKit/UIKit.h> @interface TestViewController : UIViewController - (IBAction)changeFreq:(UISlider *)sender; @property (retain, nonatomic) IBOutlet UILabel *freqLabel; @end #import "TestViewController.h" @interface TestViewController () @end @implementation TestViewController - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. } - (void)didReceiveMemoryWarning { [super didReceiveMemoryWarning]; // Dispose of any resources that can be recreated. } - (IBAction)changeFreq:(UISlider *)sender { } - (void)dealloc { [_freqLabel release]; [super dealloc]; } @end TestViewController.h TestViewController.m
  • 81. akaAdd Code and Run #import "TestViewController.h" @interface TestViewController () @end @implementation TestViewController - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. } - (void)didReceiveMemoryWarning { [super didReceiveMemoryWarning]; // Dispose of any resources that can be recreated. } - (IBAction)changeFreq:(UISlider *)sender { ! self.freqLabel.text = [NSString stringWithFormat:@"%f", sender.value]; } - (void)dealloc { [_freqLabel release]; [super dealloc]; } @end TestViewController.m
  • 83. akaAdd libpd.xcodeproj Drag&Drop Close any other projects first because of workspaces problem
  • 86. akaAdd Header Search Paths 1 4 Drag&Drop Double Click 3 Add libpd/objc and libpd/pure-data/src 2
  • 87. akaAdd the Pd Patch 1Drag&Drop 2 3
  • 88. akaAdd Codes to TestViewController.h #import <UIKit/UIKit.h> #import "PdBase.h" #import "PdAudioController.h" @interface TestViewController : UIViewController <PdReceiverDelegate> - (IBAction)changeFreq:(UISlider *)sender; @property (retain, nonatomic) IBOutlet UILabel *freqLabel; @property (strong, nonatomic) PdAudioController *audioController; @end
  • 89. - (void)viewDidLoad { [super viewDidLoad]; ! // Do any additional setup after loading the view, typically from a nib. ! // Create Audio Controller ! self.audioController = [[[PdAudioController alloc] init] autorelease]; ! [self.audioController configureAmbientWithSampleRate:44100 numberChannels:2 mixingEnabled:YES]; ! self.audioController.active = YES; ! ! // Open Pd Patch ! [PdBase openFile:@"Test.pd" path:[[NSBundle mainBundle] bundlePath]]; ! ! // Set PdBase Delegate ! PdBase.delegate = self;! ! // Subscribe messages from Pd ! [PdBase subscribe:@"freqInHz"]; } - (IBAction)changeFreq:(UISlider *)sender { ! // Send a message to Pd ! [PdBase sendFloat:sender.value toReceiver: @"freq" ]; } // Handle float messages from Pd - (void)receiveFloat:(float)received fromSource:(NSString *)source { ! if ([source isEqualToString:@"freqInHz"]) { ! ! self.freqLabel.text = [NSString stringWithFormat:@"%f Hz", received]; ! } } akaAdd Codes to TestViewController.m
  • 91. akaUser Interface - Code - Patch - (IBAction)changeFreq:(UISlider *)sender { ! [PdBase sendFloat:sender.value toReceiver: @"freq" ]; } - (void)receiveFloat:(float)received fromSource:(NSString *)source { ! if ([source isEqualToString:@"freqInHz"]) { ! ! self.freqLabel.text = [NSString stringWithFormat:@"%f", received]; ! } } Action Outlet Receive method (Delegate) Send method
  • 92. akalibpd Project Summary ● Add libpd.xcodeproj ● Add Target Dependencies libpd-ios ● Add Libraries/Frameworks libpd-ios.a, AudioToolbox.framework, AVFoundation.framework ● Add Header Search Paths libpd/objc, libpd/pure-data/src ● Add Pd Patch ● Add Codes Create PdAudioController, Set it up and Make it active Open Pd patch Set PdBase delegate and Subscribe Pd messages Send messages to Pd patch and Receive messages from Pd patch
  • 93. akaFurther References ● Pd for iOS wiki https://github.com/libpd/pd-for-ios/wiki/ios ● libpd Objective-C API https://github.com/libpd/libpd/wiki/objc ● iOSの教科書 http://iosbook.net/
  • 95. ● Solo or Group ● 3 minutes ● Theme/Motif/Feeling ● Audio Processing ● User Interface ● Composition ● Performance akaCreate your own music Kraftwerk / Pocket Calculator
  • 96. akaWhy? (again) ● Many excellent music apps ● Create your own music with your own idea with your own app Animoog by Moog Music Inc.
  • 97. akaMake a Documentation ● Format PDF Single Page Landscape ● Contents Title Name Description Images (Screen, Photo, etc.) Title: China My Love Name: Masayuki Akamatsu Description: “China My Love” is a simple app that transmits MIDI note messages by hitting keys on a keyboard. Playing an input phrase as a sequence is also possible. The tempo and the loop can be set. You can use either the on-screen software keyboard or a Bluetooth/Dock-connected hardware keyboard for iOS.
  • 99. akaiDonation Unneeded iPhone, iPad, iPod touch, I am asking for a kind donation http://akamatsu.org/aka/ios/idonation/
  • 100. 謝謝 Thanks to Prof. Dajuin Yao and students