Node.js Macht Frei
Timur Shemsedinov
Kiev Polytechnic Institute
https://github.com/tshemsedinov
Metarhia
Timur Shemsedinov
Kiev Polytechnic Institute
https://github.com/tshemsedinov
Experimental
Alternative
Technological
Stack
#1 The Problem
Web is terrible...
Everyday you
struggle for
every step
#2 Alternatives
Same task for desktop or
mobile takes less time...
...but not easy distributed,
needs installation, updates
and so platform-dependent
#3 Metarhia Concept
What if ...
Remove everything
except JavaScript
#4 Concept and Purpose
We need applied software
for following fields:
• Business
• Economics
• Enterprise
• Education
• Medicine
• Governance
• Social Net
• Communication
• Trade
• TV
etc.
#5 Concept and Purpose
We need applications
to be at the same time:
• Distributed (multiserver, multiclient)
• Highload (billions of concurrent users)
• Interactive (bi-directional in real time)
• Secure (both anonymity and reliability)
• Flexible (continuous self-updates)
#6 What we have as of now?
Now we have:
• Locally installed Applications
for Windows, Mac, Linux
using Qt, WinForms, WPF, AWT, SWING...
• Mobile Applications
for iOS, Android, etc...
• Web Applications
GUI and b-logic should be implemented
many times for different platforms
#7 Problems of Archaic Web
• HTTP is not designed for applications
• HTML is not designed for rich UI or GUI
• DOM is not designed for dynamic changes
• AJAX is not designed for interactivity
• REST is not designed for statefull apps
Also Internet is full of trash:
Search engines contains fakes, SEO and spam
DNS are full of domaining and cybersquatting
Data and content are mixed with GUI
#8 Archaic Web
• HTTP is designed a long time ago
• HTML is designed for hypertext
• DOM is designed for static pages
But we can’t change everything
at once. We need both:
• Compatibility
• New solutions
#9 Temporary Solutions
• HTTP: gzip, sprites, keep-alive,
CDN, use HTTPS, HTTP/2
• DOM: use Angular, React.js, etc.
(to speedup rendering
and application development)
• WebSockets: emulates TCP
sockets over HTTP (overhead)
#10 All problems together
• Platforms for applied software are
fragmented
• Web technologies are archaic and
obsoleted
• Protocols have great overhead and
few features
• Frameworks and tools are not stable
and unified and changing all the time
• We have great security risks
#11 Conclusion
• Applications become
obsolete quickly
• We have a problems in
application integration
• No unification, less code reuse,
fragmentation and extra work
#12 What we need?
We need following for applied software:
• Server-side runtime
• Client-side runtime with Renderer
• Interprocess communication with:
• RPC, MQ, streaming and introspection
• S/C, S/S, C/C (P2P) interactions
• DBMS designed to be globally
distributed
• New naming and identification
conventions
#13 Introduction to Metarhia
Metarhia is a new Technological Stack
• Experimental
(just to demonstrate fresh vision)
• Alternative
(should not support compatibility)
• Open Source
(free and no vendor dependent)
#14 Metarhia components
• JSTP
JavaScript Transfer Protocol
• Impress
Application Server
• Global Storage
distributed DBMS
• Console
new generation browser
#15 JavaScript Transfer Protocol
JSTP is a family of data formats and the
libraries and implementations for
different languages and platforms based
on several simple assumptions:
• Simple and well-known format
• Interactive communications
• Reactive data processing
• Real-time bidirectional interactions
• Focused on optimization and highload
#16 JSTP Ideas
1. We can take data serialization format,
already available for multiple platforms
This format is not even JSON
it is JavaScript itself
2. Use metadata everywhere: for schema
model, for data representation
minimization, for flexibility
#17 Metarhia and JSTP Ideas
Minimization of data transformations
Single data format for:
• Persistent storage in database,
• Representation in server memory
• Serialization and protocol
• Representation in client memory
• Intersystem data exchange
#18 JSTP Family
Record Serialization
{ name: 'Marcus Aurelius', birth: '1990-02-15' }
Object Serialization
{ name: 'Marcus Aurelius', age: function() {...} }
Record Metadata
{ name: 'string', age: '[number]' }
Record Data
[ 'Marcus Aurelius', 'AE127095' ]
JavaScript Transfer Protocol
{ call: [17, 'api'], method: ['Marcus ...', 26] }
#19 Compare: XML
<oilPump name=”PT004” displacement=”constant” value=”63” control=
”automatic” status=”working”>
<flowMeter substance=”fluid” recording=”off” role=”master”
period=”00:30” dataOutput=”Parent.FT002.Verification”/>
<outlet pressure=”180” status=working”/>
<failureSensors>
<row><module>Seatings</module><indication>none</indication>
<status>OK</status></row>
<row><module>Flap01</module><indication>open</indication>
<status>OK</status></row>
<row><module>Flap02</module><indication>closed</indication>
<status>overload</status></row>
<row><module>Joint</module><indication>detach</indication>
<status>OK</status></row>
</failureSensors>
</oilPump>
#20 Compare: CLEAR
1: PT004:OilPump Displacement[constant] Value[63]
Control[automatic] Status[working]
2: #FlowMeter Substance[fluid] Recording[off] Role[master]
Period[00:30] DataOutput[Parent.FT002.Verification]
2: #Outlet Pressure[180] Status[working]
2: #FailureSensors:Table [Module,Indication,Status]
3: [Seatings,none,OK]
3: [Flap01,open,OK]
3: [Flap02,closed,overload]
3: [Joint,detach,OK]
#21 Compare with JSON
{ “name”: “PT004”, “type”: “OilPump”, “displacement”: “constant”,
“value”: 63, “control”: “automatic”, “status”:”working”,
“flowMeter”: { “substance”: “fluid”, “recording”: “off”,
“role”: “master”, “period”: “00:30”,
“dataOutput”: “Parent.FT002.Verification”,
},
“outlet”: { “pressure”: 180, “status”: “working”,
“failureSensors”: [ [“Module”, ”Indication”, ”Status”] ,
[“Seatings”, ”none”, ”OK”], [“Flap01”, ”open”, ”OK”],
[“Flap02”, ”closed”, ”overload”], [“Joint”, ”detach”, ”OK”]
]
}
}
#22 Compare: JSTP Object
{ name: “PT004”, type: “OilPump”, displacement: “constant”,
value: 63, control: “automatic”, status:”working”,
flowMeter: { substance: “fluid”, recording: “off”, role: “master”,
period: “00:30”, dataOutput: “Parent.FT002.Verification”,
},
outlet: { pressure: 180, status: “working”,
failureSensors: [ [“Module”,”Indication”,”Status”] ,
[“Seatings”,”none”,”OK”], [“Flap01”,”open”,”OK”],
[“Flap02”,”closed”,”overload”], [“Joint”,”detach”,”OK”]
]
}
}
#23 Compare: JSTP Data
[“PT004”,“OilPump”,“constant”,63,“automatic”,”working”,
[“fluid”,“off”,“master”,“00:30”,“Parent.FT002.Verification”],
[180,“working”,[[“Module”,”Indication”,”Status”],
[“Seatings”,”none”,”OK”],[“Flap01”,”open”,”OK”],
[“Flap02”,”closed”,”overload”], [“Joint”,”detach”,”OK”]]]]
#24 Simple parser for Node.js
// Implementation
api.jstp.parse = (s) => {
var sandbox = api.vm.createContext({});
var js = api.vm.createScript('(' + s + ')');
return js.runInNewContext(sandbox);
};
// Usage
api.fs.readFile('./person.record', (e, s) => {
var person = api.jstp.parse(s);
console.dir(person);
});
#25 JSTP Example
{
name: ['Marcus', 'Aurelius'].join(' '),
passport: 'AE' + '127095',
birth: { date: new Date('1990-02-15'), place: 'Rome' },
age: function() {
var difference = new Date() - birth.date;
return Math.floor(difference / 31536000000);
},
address: {
country: 'Ukraine',
city: 'Kiev',
zip: '03056',
street: 'Pobedy',
building: '37',
floor: '1',
room: '158'
}
}
#26 Implementation & SDK
SDK includes mote then 10 languages
• JavaScript for Browser
• JavaScript for Node.js
• and Impress Application Server
• C and C++ for STL and Qt
• Swift and Objective-C for iOS
• Java for Android and JavaEE
• C# for .NET
• Python, Haskell, PHP, GoLang
etc.
#27 JSTP features
JSTP supports:
1. Interactive communication
2. RPC (Remote Procedure Calls)
3. MQ and Event (Message Queuing)
4. Bi-directional calls and event
5. Establish long-time connection
6. Both statefull and staleless approach
7. Types of interaction:
Client-Server, Server-Server, Client-Client
#28 Comparison HTTP, HTTP/2, JSTP
HTTP HTTP/2 JSTP
Async - + +
Sync + - +
Statefull - +/- +
Stateless + + +
Mult. Interf. - - +
Mult. packets - + +
Mult. ports - - +
HDR comp. - + +
#29 Comparison HTTP, HTTP/2, JSTP
HTTP HTTP/2 JSTP
Human text + - +
Binary - + +
Combine Req. - + +
Divide Req. - - +
Prioritization - + +
Encription +/- + +
With no encr. + + -
AJAX + + -
Cancel Req. - + +
#30 Comparison HTTP, HTTP/2, JSTP
HTTP HTTP/2 JSTP
RPC - - -
MQ - +/- -
State Sync. - - +
Health - - +
Cache push - + +
Binary stream + ++ ++
C/S + + +
S/S - - +
C/C - - +
#31 Comparison HTTP, HTTP/2, JSTP
HTTP HTTP/2 JSTP
For ex. Web + + -
HTTP cmptbl. + + -
DNS + + -
URL + + -
Anonymity 2 area 2 area unlim.
Less load - + ++
Fewer con. - + ++
Fewer req. - + +
#32 JSTP Packet types
call Request packet
callback Reply packet
event Message packet
state State synchronization packet
stream Binary data transfer packet
handshake Connection initialization
health Remote side status monitoring
extensible
#33 Metarhia Console Browser
What is it?
• DSL for GUI
• Native Renderer for each platform
• Built-in DBMS
• Built-in JSTP Client
• V8 JavaScript Runtime
#34 Metarhia Console Browser
Main ideas:
• Navigate Databases
• Everything is interactive
• Use metadata to change behavior
• Scaffolding User Interfaces
• Component approach for UI Controls
• Separate Content and Rendering
• Work online and offline
• Use TLS encryption
#35 Console Implementations
We are working on
rendering implementation for:
• For Desktops
with Linux, Mac, Windows etc.
using Qt (C++)
• For iOS (Swift, Objective-C)
• For Android (Java)
• For archaic Web (JavaScript)
#36 GlobalStorage DBMS
What is it?
It is hybrid DBMS API, now we have just In-
memory DBMSD and no own persistent
storage implementation but we can use this
API to store persistent data in MongoDB,
PostgreSQL, Oracle, etc.
Distributed, Built-in, In-memory DBMS,
JSTP Compatible
#37 GlobalStorage Ideas
Main ideas:
• To be built-in for Client-side (Console)
• To be built-in for Application server
• Minimize Interprocess communication
• Minimize data repacking and transform
• Use metadata and metamodel
• Work online and offline with sync.
#38 Metarhia
Github https://github.com/metarhia
• JSTP
https://github.com/metarhia/JSTP
• Impress
https://github.com/metarhia/Impress
• Global Storage
https://github.com/metarhia/GlobalStorage
• Console
https://github.com/metarhia/Console
Metarhia
Timur Shemsedinov
Kiev Polytechnic Institute
https://github.com/tshemsedinov
Experimental
Alternative
Technological
Stack
Questions?
Thank you!

Metarhia: Node.js Macht Frei

  • 1.
    Node.js Macht Frei TimurShemsedinov Kiev Polytechnic Institute https://github.com/tshemsedinov
  • 2.
    Metarhia Timur Shemsedinov Kiev PolytechnicInstitute https://github.com/tshemsedinov Experimental Alternative Technological Stack
  • 3.
    #1 The Problem Webis terrible... Everyday you struggle for every step
  • 4.
    #2 Alternatives Same taskfor desktop or mobile takes less time... ...but not easy distributed, needs installation, updates and so platform-dependent
  • 5.
    #3 Metarhia Concept Whatif ... Remove everything except JavaScript
  • 6.
    #4 Concept andPurpose We need applied software for following fields: • Business • Economics • Enterprise • Education • Medicine • Governance • Social Net • Communication • Trade • TV etc.
  • 7.
    #5 Concept andPurpose We need applications to be at the same time: • Distributed (multiserver, multiclient) • Highload (billions of concurrent users) • Interactive (bi-directional in real time) • Secure (both anonymity and reliability) • Flexible (continuous self-updates)
  • 8.
    #6 What wehave as of now? Now we have: • Locally installed Applications for Windows, Mac, Linux using Qt, WinForms, WPF, AWT, SWING... • Mobile Applications for iOS, Android, etc... • Web Applications GUI and b-logic should be implemented many times for different platforms
  • 9.
    #7 Problems ofArchaic Web • HTTP is not designed for applications • HTML is not designed for rich UI or GUI • DOM is not designed for dynamic changes • AJAX is not designed for interactivity • REST is not designed for statefull apps Also Internet is full of trash: Search engines contains fakes, SEO and spam DNS are full of domaining and cybersquatting Data and content are mixed with GUI
  • 10.
    #8 Archaic Web •HTTP is designed a long time ago • HTML is designed for hypertext • DOM is designed for static pages But we can’t change everything at once. We need both: • Compatibility • New solutions
  • 11.
    #9 Temporary Solutions •HTTP: gzip, sprites, keep-alive, CDN, use HTTPS, HTTP/2 • DOM: use Angular, React.js, etc. (to speedup rendering and application development) • WebSockets: emulates TCP sockets over HTTP (overhead)
  • 12.
    #10 All problemstogether • Platforms for applied software are fragmented • Web technologies are archaic and obsoleted • Protocols have great overhead and few features • Frameworks and tools are not stable and unified and changing all the time • We have great security risks
  • 13.
    #11 Conclusion • Applicationsbecome obsolete quickly • We have a problems in application integration • No unification, less code reuse, fragmentation and extra work
  • 14.
    #12 What weneed? We need following for applied software: • Server-side runtime • Client-side runtime with Renderer • Interprocess communication with: • RPC, MQ, streaming and introspection • S/C, S/S, C/C (P2P) interactions • DBMS designed to be globally distributed • New naming and identification conventions
  • 15.
    #13 Introduction toMetarhia Metarhia is a new Technological Stack • Experimental (just to demonstrate fresh vision) • Alternative (should not support compatibility) • Open Source (free and no vendor dependent)
  • 16.
    #14 Metarhia components •JSTP JavaScript Transfer Protocol • Impress Application Server • Global Storage distributed DBMS • Console new generation browser
  • 17.
    #15 JavaScript TransferProtocol JSTP is a family of data formats and the libraries and implementations for different languages and platforms based on several simple assumptions: • Simple and well-known format • Interactive communications • Reactive data processing • Real-time bidirectional interactions • Focused on optimization and highload
  • 18.
    #16 JSTP Ideas 1.We can take data serialization format, already available for multiple platforms This format is not even JSON it is JavaScript itself 2. Use metadata everywhere: for schema model, for data representation minimization, for flexibility
  • 19.
    #17 Metarhia andJSTP Ideas Minimization of data transformations Single data format for: • Persistent storage in database, • Representation in server memory • Serialization and protocol • Representation in client memory • Intersystem data exchange
  • 20.
    #18 JSTP Family RecordSerialization { name: 'Marcus Aurelius', birth: '1990-02-15' } Object Serialization { name: 'Marcus Aurelius', age: function() {...} } Record Metadata { name: 'string', age: '[number]' } Record Data [ 'Marcus Aurelius', 'AE127095' ] JavaScript Transfer Protocol { call: [17, 'api'], method: ['Marcus ...', 26] }
  • 21.
    #19 Compare: XML <oilPumpname=”PT004” displacement=”constant” value=”63” control= ”automatic” status=”working”> <flowMeter substance=”fluid” recording=”off” role=”master” period=”00:30” dataOutput=”Parent.FT002.Verification”/> <outlet pressure=”180” status=working”/> <failureSensors> <row><module>Seatings</module><indication>none</indication> <status>OK</status></row> <row><module>Flap01</module><indication>open</indication> <status>OK</status></row> <row><module>Flap02</module><indication>closed</indication> <status>overload</status></row> <row><module>Joint</module><indication>detach</indication> <status>OK</status></row> </failureSensors> </oilPump>
  • 22.
    #20 Compare: CLEAR 1:PT004:OilPump Displacement[constant] Value[63] Control[automatic] Status[working] 2: #FlowMeter Substance[fluid] Recording[off] Role[master] Period[00:30] DataOutput[Parent.FT002.Verification] 2: #Outlet Pressure[180] Status[working] 2: #FailureSensors:Table [Module,Indication,Status] 3: [Seatings,none,OK] 3: [Flap01,open,OK] 3: [Flap02,closed,overload] 3: [Joint,detach,OK]
  • 23.
    #21 Compare withJSON { “name”: “PT004”, “type”: “OilPump”, “displacement”: “constant”, “value”: 63, “control”: “automatic”, “status”:”working”, “flowMeter”: { “substance”: “fluid”, “recording”: “off”, “role”: “master”, “period”: “00:30”, “dataOutput”: “Parent.FT002.Verification”, }, “outlet”: { “pressure”: 180, “status”: “working”, “failureSensors”: [ [“Module”, ”Indication”, ”Status”] , [“Seatings”, ”none”, ”OK”], [“Flap01”, ”open”, ”OK”], [“Flap02”, ”closed”, ”overload”], [“Joint”, ”detach”, ”OK”] ] } }
  • 24.
    #22 Compare: JSTPObject { name: “PT004”, type: “OilPump”, displacement: “constant”, value: 63, control: “automatic”, status:”working”, flowMeter: { substance: “fluid”, recording: “off”, role: “master”, period: “00:30”, dataOutput: “Parent.FT002.Verification”, }, outlet: { pressure: 180, status: “working”, failureSensors: [ [“Module”,”Indication”,”Status”] , [“Seatings”,”none”,”OK”], [“Flap01”,”open”,”OK”], [“Flap02”,”closed”,”overload”], [“Joint”,”detach”,”OK”] ] } }
  • 25.
    #23 Compare: JSTPData [“PT004”,“OilPump”,“constant”,63,“automatic”,”working”, [“fluid”,“off”,“master”,“00:30”,“Parent.FT002.Verification”], [180,“working”,[[“Module”,”Indication”,”Status”], [“Seatings”,”none”,”OK”],[“Flap01”,”open”,”OK”], [“Flap02”,”closed”,”overload”], [“Joint”,”detach”,”OK”]]]]
  • 26.
    #24 Simple parserfor Node.js // Implementation api.jstp.parse = (s) => { var sandbox = api.vm.createContext({}); var js = api.vm.createScript('(' + s + ')'); return js.runInNewContext(sandbox); }; // Usage api.fs.readFile('./person.record', (e, s) => { var person = api.jstp.parse(s); console.dir(person); });
  • 27.
    #25 JSTP Example { name:['Marcus', 'Aurelius'].join(' '), passport: 'AE' + '127095', birth: { date: new Date('1990-02-15'), place: 'Rome' }, age: function() { var difference = new Date() - birth.date; return Math.floor(difference / 31536000000); }, address: { country: 'Ukraine', city: 'Kiev', zip: '03056', street: 'Pobedy', building: '37', floor: '1', room: '158' } }
  • 28.
    #26 Implementation &SDK SDK includes mote then 10 languages • JavaScript for Browser • JavaScript for Node.js • and Impress Application Server • C and C++ for STL and Qt • Swift and Objective-C for iOS • Java for Android and JavaEE • C# for .NET • Python, Haskell, PHP, GoLang etc.
  • 29.
    #27 JSTP features JSTPsupports: 1. Interactive communication 2. RPC (Remote Procedure Calls) 3. MQ and Event (Message Queuing) 4. Bi-directional calls and event 5. Establish long-time connection 6. Both statefull and staleless approach 7. Types of interaction: Client-Server, Server-Server, Client-Client
  • 30.
    #28 Comparison HTTP,HTTP/2, JSTP HTTP HTTP/2 JSTP Async - + + Sync + - + Statefull - +/- + Stateless + + + Mult. Interf. - - + Mult. packets - + + Mult. ports - - + HDR comp. - + +
  • 31.
    #29 Comparison HTTP,HTTP/2, JSTP HTTP HTTP/2 JSTP Human text + - + Binary - + + Combine Req. - + + Divide Req. - - + Prioritization - + + Encription +/- + + With no encr. + + - AJAX + + - Cancel Req. - + +
  • 32.
    #30 Comparison HTTP,HTTP/2, JSTP HTTP HTTP/2 JSTP RPC - - - MQ - +/- - State Sync. - - + Health - - + Cache push - + + Binary stream + ++ ++ C/S + + + S/S - - + C/C - - +
  • 33.
    #31 Comparison HTTP,HTTP/2, JSTP HTTP HTTP/2 JSTP For ex. Web + + - HTTP cmptbl. + + - DNS + + - URL + + - Anonymity 2 area 2 area unlim. Less load - + ++ Fewer con. - + ++ Fewer req. - + +
  • 34.
    #32 JSTP Packettypes call Request packet callback Reply packet event Message packet state State synchronization packet stream Binary data transfer packet handshake Connection initialization health Remote side status monitoring extensible
  • 35.
    #33 Metarhia ConsoleBrowser What is it? • DSL for GUI • Native Renderer for each platform • Built-in DBMS • Built-in JSTP Client • V8 JavaScript Runtime
  • 36.
    #34 Metarhia ConsoleBrowser Main ideas: • Navigate Databases • Everything is interactive • Use metadata to change behavior • Scaffolding User Interfaces • Component approach for UI Controls • Separate Content and Rendering • Work online and offline • Use TLS encryption
  • 37.
    #35 Console Implementations Weare working on rendering implementation for: • For Desktops with Linux, Mac, Windows etc. using Qt (C++) • For iOS (Swift, Objective-C) • For Android (Java) • For archaic Web (JavaScript)
  • 38.
    #36 GlobalStorage DBMS Whatis it? It is hybrid DBMS API, now we have just In- memory DBMSD and no own persistent storage implementation but we can use this API to store persistent data in MongoDB, PostgreSQL, Oracle, etc. Distributed, Built-in, In-memory DBMS, JSTP Compatible
  • 39.
    #37 GlobalStorage Ideas Mainideas: • To be built-in for Client-side (Console) • To be built-in for Application server • Minimize Interprocess communication • Minimize data repacking and transform • Use metadata and metamodel • Work online and offline with sync.
  • 40.
    #38 Metarhia Github https://github.com/metarhia •JSTP https://github.com/metarhia/JSTP • Impress https://github.com/metarhia/Impress • Global Storage https://github.com/metarhia/GlobalStorage • Console https://github.com/metarhia/Console
  • 41.
    Metarhia Timur Shemsedinov Kiev PolytechnicInstitute https://github.com/tshemsedinov Experimental Alternative Technological Stack Questions? Thank you!