PROCESS WARP
distributed processing system
Yuji Ito
facebook : https://www.facebook.com/ito.yuuji
github : https://github.com/llamerada-jp
I talk about PROCESS WARP that is a distributed processing system, I'm developing.
PROCESS WARP : A new infrastructure
for distributed processing system. All
machines are equal. It doesn’t require a
special machine as a server.
Typical system : Formed by client/
server. It requires a server, if the server is
down, service will stop.
※ Icons by Crystal Project (LGPL).
Aim
Feature
• Applications run in virtual
space, that is made up by a
group of nodes.
• Makes it possible to use any
amount of computer resources
and control nodes by a single
process.
• Makes it possible to warp a
process to another node with
keep alive.
Advantage
Increase total processing power
according to number of joining
nodes.
• Decrease cost to maintain
service without relation to
number of nodes.
• Service provider can choose a
good topology for system
having a lot of nodes.
join
Advantage
It breaks the border between a
standalone application and a
cloud application.
• User can use stand alone
application similar to cloud
application on PROCESS
WARP.
• User can use application
offline, after a migration
process.
(Under active development)
Because a process image can migrate, the
user can use the application offline.
Share the process space among the
connected computers and freely pass the
running application between machines.
Architecture
Host : native environment like UNIX, or web-browser
CPU : Interpret LLVM-IR
instructions without using
native stack
Memory : Store memory
image partially per node
other host
VM
Process Virtual Machine : Dump and synchronise (or pass)
memory image; Control PCB (Process Control Block); Call API
Use native APIs
Synchronise memory image, Control PCB
I'm developing a special virtual machine to realise previous slide's system.
LLVM is a compiler infrastructure that is used in many environments.
Actual status
• You can run virtual machine on FreeBSD, Linux, OSX, and Web-
browser.
• And run sample C/C++ program on virtual machine.
• Multiple threads can be distributed across several machines.
• Virtual machines have minimal APIs like “printf”.
• I published source code with MIT license and anyone can try it on
web-browser.
• http://www.processwarp.org/
• https://github.com/processwarp
Demo
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
char* ret = "a";
static void *thread_func(void *vptr_args) {
int i;
for (i = 0; i < 20000; i++) {
printf(" b [%d]n", i);
}
return (void*)ret;
}
int main(void) {
int i;
pthread_t thread;
void* retval;
if (pthread_create
(&thread, NULL, thread_func, NULL)
!= 0) {
return EXIT_FAILURE;
}
for (i = 0; i < 10000; i++) {
printf("a [%d]n", i);
}
if (pthread_join(thread, &retval) != 0) {
return EXIT_FAILURE;
}
printf("ret %016llx %16llxn",
(unsigned long long)ret,
(unsigned long long)retval);
return EXIT_SUCCESS;
}
This program is a simple multi thread
application. This program makes two
threads and increments a counter on each
thread.
Next milestones
• Remove server (currently needs control server) and implement
peer to peer transfer.
• Make test, document, and more APIs to run existing applications.
• Develop I/O functions including storage and socket layers.
Thank you!

PROCESS WARP

  • 1.
    PROCESS WARP distributed processingsystem Yuji Ito facebook : https://www.facebook.com/ito.yuuji github : https://github.com/llamerada-jp I talk about PROCESS WARP that is a distributed processing system, I'm developing.
  • 2.
    PROCESS WARP :A new infrastructure for distributed processing system. All machines are equal. It doesn’t require a special machine as a server. Typical system : Formed by client/ server. It requires a server, if the server is down, service will stop. ※ Icons by Crystal Project (LGPL). Aim
  • 3.
    Feature • Applications runin virtual space, that is made up by a group of nodes. • Makes it possible to use any amount of computer resources and control nodes by a single process. • Makes it possible to warp a process to another node with keep alive.
  • 4.
    Advantage Increase total processingpower according to number of joining nodes. • Decrease cost to maintain service without relation to number of nodes. • Service provider can choose a good topology for system having a lot of nodes. join
  • 5.
    Advantage It breaks theborder between a standalone application and a cloud application. • User can use stand alone application similar to cloud application on PROCESS WARP. • User can use application offline, after a migration process. (Under active development) Because a process image can migrate, the user can use the application offline. Share the process space among the connected computers and freely pass the running application between machines.
  • 6.
    Architecture Host : nativeenvironment like UNIX, or web-browser CPU : Interpret LLVM-IR instructions without using native stack Memory : Store memory image partially per node other host VM Process Virtual Machine : Dump and synchronise (or pass) memory image; Control PCB (Process Control Block); Call API Use native APIs Synchronise memory image, Control PCB I'm developing a special virtual machine to realise previous slide's system. LLVM is a compiler infrastructure that is used in many environments.
  • 7.
    Actual status • Youcan run virtual machine on FreeBSD, Linux, OSX, and Web- browser. • And run sample C/C++ program on virtual machine. • Multiple threads can be distributed across several machines. • Virtual machines have minimal APIs like “printf”. • I published source code with MIT license and anyone can try it on web-browser. • http://www.processwarp.org/ • https://github.com/processwarp
  • 8.
    Demo #include <stdio.h> #include <stdlib.h> #include<pthread.h> char* ret = "a"; static void *thread_func(void *vptr_args) { int i; for (i = 0; i < 20000; i++) { printf(" b [%d]n", i); } return (void*)ret; } int main(void) { int i; pthread_t thread; void* retval; if (pthread_create (&thread, NULL, thread_func, NULL) != 0) { return EXIT_FAILURE; } for (i = 0; i < 10000; i++) { printf("a [%d]n", i); } if (pthread_join(thread, &retval) != 0) { return EXIT_FAILURE; } printf("ret %016llx %16llxn", (unsigned long long)ret, (unsigned long long)retval); return EXIT_SUCCESS; } This program is a simple multi thread application. This program makes two threads and increments a counter on each thread.
  • 9.
    Next milestones • Removeserver (currently needs control server) and implement peer to peer transfer. • Make test, document, and more APIs to run existing applications. • Develop I/O functions including storage and socket layers.
  • 10.