Adventures in rest3d
github.com/amd/rest3d
Rémi Arnaud
Maxime Helen
AMD
TIMELINE 10+ years
(Dec) 2004
SCEA releases COLLADA v1.1.0
2006
Khronos releases COLLADA v1.42007
COLLADA DOM released
200...
SigGraph 2004
• First public presentation of COLLADA
• http://www.collada.org/public_forum/files/C
OLLADASigGraphTechTalkW...
2004 design goals
Client / Server
Database Server
clientServer
3D API
Server Client
Shaders
COLLADA
XML DB
core
cache
Various
Media
Local
Co...
Use cases
• Asset repository
– Query, download, upload
– asset store, archivals
• Content delivery
– Web browser
– Mobile ...
Web Server / Web client
Database Server
Web Browser / AppHttp Server
HttpRequest
COLLADA
XML DB
Server Client
XML
DB
Node....
The REST architectural style
• REpresentational State Transfer (REST) is a way to create, read,
update or delete informati...
Definitions
• Resource: A single instance of an object.
• Collection: A collection of Resources.
• Endpoint: An URL on a S...
REST HTTP verbs
Most used:
• GET
– Retrieve a specific Resource from the Server, or other idempotent queries. Cacheable,
s...
REST HTTP verbs
Barely used:
• HEAD
– Returns http header fields without any response body. May invalidate cache.
• OPTION...
nouns
• GET /something-ID : returns something
• PUT /something-ID : send something to be
stored at something-ID
• POST : s...
http status codes
• 1xx – experimental
• 2xx – Successful [*]
• 3xx – Redirection
• 4xx – Client Error
• 5xx – Server Erro...
Typical status codes
• 200 OK – [GET]
– The Client requested data from the Server, and the Server found it for them (Idemp...
Use case: Asset repository
• Sketchup warehouse
3dwarehouse.sketchup.com
• 3dvia warehouse
www.3dvia.com
• TurboSquid mark...
Sketchup warehouse main
Sketchup warehouse model
Sketchup warehouse download
https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=d5d13872-32c2-4fb2-8b54-5291c3...
Asset vs Resource & Collection
• A 3D asset contains multiple Resources
– Geometry, images, animation, shaders, media …
– ...
response header
Accept-Ranges:bytes
Access-Control-Allow-Origin:*
Age:35
Cache-Control:max-age=2592000
Connection:keep-ali...
MIME types
Name MIME Type File
Extension
More Details
COLLADA model/vnd.collada+xml .dae IANA: COLLADA
Google Earth -
Zipp...
3DVIA download
3DVIA .dae download
Status Code:302 Found
Cache-Control:no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Con...
3DVIA .dae (Zip) download
Cache-Control:no-cache, must-revalidate
Connection:Keep-Alive
Content-
Disposition:attachment;fi...
3DVIA .3dxml download
3DVIA .3dxml download
Accept-Ranges:bytes
Cache-Control:must-revalidate
Connection:Keep-Alive
Content-Disposition:filename...
rest3d - assets
• 3d assets are Collections
– They can be downloaded as zip archives
– Some have a specific organization, ...
Rest3d – warehouse proxy
GET /rest3d/info/warehouse Get top level
collections
GET /rest3d/search/warehouse Search for
mode...
rest3d API design
• /rest3d prefix
• /rest3d/info/.. -> returns information (metadata)
• /rest3d/data/.. -> returns data (...
http://127.0.0.1:8000/rest3d/info/warehouse/?uuid=c
_f8fc62afb9653d03cbe9bc85eececab4
{
"database": "warehouse",
"mimetype...
http://127.0.0.1:8000/rest3d/search/warehouse/fisher
%20fine%20art%20library
{
"database": "warehouse",
"mimetype": "model...
rest3d gui – Collections
http://127.0.0.1:8000/rest3d/search/warehouse/fisher
%20fine%20art%20library
{
"database": "warehouse",
"mimetype": "model...
http://127.0.0.1:8000/rest3d/info/warehouse/?uuid=m
_71187c14aab1b33bdc0d3b8ac51c2a32_k2
{
"database": "warehouse",
"mimet...
rest3d ui – asset composition
content preview
• Web server can provide content preview
– Send javascript and content
– rest API does not need to define ...
rest3d application
• 3D viewer
– COLLADA loader
– glTF loader
– rest3d webGL viewer
• ‘workstation’ style UI
– Split windo...
http://127.0.0.1:8000/viewer/viewer6.html
rest3d implementation
sketchup 3dvia xml db
rest3d server
(nodejs)
rest3d drivers
• Provided drivers for:
– warehouse/ – proxy to sketchup warehouse
– 3dvia/ – proxy to 3dvia warehouse
– tm...
Authentication
(e.g. asset store, private cloud..)
• 3dvia example. Requires login.
– Do not store login information on re...
Proxy authentication
Web browser
(client)
POST /rest3d/login/3dvia
User: username
Passwd: password
Status Code 2xx or 4xx
...
Proxy request principle
Web browser
(client)
GET /rest3d/info/3dvia/..
cookie: sessionID
Data is available
https rest3d se...
Third party authentification
(e.g. Google oauth2)
Web browser
(client)
POST /rest3d/login/db
POST /google/login
Interactiv...
DB access example
Web browser
(client)
GET /rest3d/info/db/..
cookie: sessionID
Data is available
https rest3d server
(nod...
search (warehouse)
search (3dvia)
rest3d search API ? (tbd)
• Simple: query string in title / description
– /rest3d/search/database?q=xxxx
• Complex: Xquery...
Use case: content delivery
• E.g. html page requests individual resources
– Paging: Content is downloaded as needed
• Cont...
CDN
CDN service provides multiple servers at several
locations, faster service to end user
1. rest3d request redirect to C...
Live update / Manifest
• Manifest contains list of all resources and their
(CDN) URL
• Application use Manifest to transla...
url vs uuid
http://127.0.0.1:8000/rest3d/info/db
{
"database":"eXist",
"name":"/",
"mimetype":"model/rest3d-collection+jso...
Caching
• Basic internet cache
– Cache is based on entire URL, including parameters
• http://server.com/test.htm?p=1 ≠ htt...
Format conversion
• IMHO - ok to use file extension as format conversion request
– Thinking about:
• http://127.0.0.1:8000...
Jobs (WIP)
• Need to manage server load, or distribute work
• Create a return (immediately) a jobid, do not wait
for job t...
Use case: Content Creation
• Import/Export
– Tools can import/export/browser using rest3d API
• Versioning
• Local changes...
Downloading::compression
• Server will compress before sending data
– Why storing .gziped files then?
• Asset repository -...
Uploading
• Web browser cannot access hard drive
– For security. User has to provide files
• Open file. Drag & Drop files
...
Versioning (WIP)
• One uuid = single version, single format?
• Current implementation:
– Collection(asset) list resources ...
rest3d versioning
http://127.0.0.1:8000/rest3d/info/db/tmp/models/Chicken.dae
Collection
Path
Indirection (versioning)
{"name":"Chicken.dae","mimetype":"model/collada+xml","size":2694152,"created":
{"date":1406848884...
Patches -
• WYSIWYG changes
– Final content get patched inside game engine
– Patch files are create, and automatically app...
Use case: user content
Use case: user content (sell)
To be continued !
Thank you for your attention
Questions and comments are welcomed
contact info: remi@acm.org
Source code:...
Upcoming SlideShare
Loading in …5
×

rest3d Web3D 2014

1,965 views
1,882 views

Published on

rest3d presentation form Web3D 2014 (Vancouver)

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,965
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • There is no API provided, but since those are accessible through a web browser, the are using URI, so we can reverse engineer those URI as the ‘unofficial API’, and parse the results (either html or json).
  • https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=d5d13872-32c2-4fb2-8b54-5291c38ac3a9&fn=FurnessLibrary.kmz
  • .daz anybody?
  • Returns a rest3d-collection -> which contains a kmz asset
  • Returns a rest3d-collection -> which contains a kmz asset
  • Assets are rest3d-collections, with an ‘assets’ list
  • Dotted lines signify separate computers – interface through network
  • rest3d Web3D 2014

    1. 1. Adventures in rest3d github.com/amd/rest3d Rémi Arnaud Maxime Helen AMD
    2. 2. TIMELINE 10+ years (Dec) 2004 SCEA releases COLLADA v1.1.0 2006 Khronos releases COLLADA v1.42007 COLLADA DOM released 2008 Khronos releases COLLADA 1.4.1 and COLLADA 1.5 2011 Khronos releases WebGL 1.0 2011 Open source COLLADA conformance test 2013 Khronos work on glTF github.com KhronosGroup/glTF/ 2013 COLLADA 1.5 ISO standard github.com /amd/rest3d rest3d discussion rest3d.org SigGraph 2004 BOF (Aug) 2003 COLLADA discussion
    3. 3. SigGraph 2004 • First public presentation of COLLADA • http://www.collada.org/public_forum/files/C OLLADASigGraphTechTalkWebQuality.pdf
    4. 4. 2004 design goals
    5. 5. Client / Server Database Server clientServer 3D API Server Client Shaders COLLADA XML DB core cache Various Media Local Content UI Script ?
    6. 6. Use cases • Asset repository – Query, download, upload – asset store, archivals • Content delivery – Web browser – Mobile / native apps • Content Creation – Local modeling tools, local content – Exporter, version control – Local modifications, WYSIWYG – Collaboration • User Content – Game levels – In game content
    7. 7. Web Server / Web client Database Server Web Browser / AppHttp Server HttpRequest COLLADA XML DB Server Client XML DB Node.js COLLADA XML DB Xquery, Xslt … rest3d Html / CSS Javascript WebGL API GLES Shaders cache Local Content
    8. 8. The REST architectural style • REpresentational State Transfer (REST) is a way to create, read, update or delete information on a server. • The term representational state transfer was introduced and defined in 2000 by Roy Fielding in his doctoral dissertation. • REST API goal is to be stateless, scalable and simple, focusing on the (software) components • REST over HTTP relies upon Uniform Resource Identifier and HTTP protocol • The REST architectural style is applied to the development of Web services – Facebook, Twitter, Google, GitHub, Amazon, Netflix… provide API – A large and growing industry of applications are using those API. There are billions of API calls a day.
    9. 9. Definitions • Resource: A single instance of an object. • Collection: A collection of Resources. • Endpoint: An URL on a Server which represents either a Resource or an entire Collection • Idempotent: Side-effect free, can happen multiple times with same result.
    10. 10. REST HTTP verbs Most used: • GET – Retrieve a specific Resource from the Server, or other idempotent queries. Cacheable, safe. • POST – Everything else, with side effects (Create, Update ..). Not cacheable, unless the response includes explicit caching information. • DELETE – Remove a Resource from the Server. Idempotent?? Not-cacheable Also available: • PUT – Create or update a new Resource on the Server. Not cacheable • PATCH – Update a resource. Not cacheable, unless the response includes explicit caching information.
    11. 11. REST HTTP verbs Barely used: • HEAD – Returns http header fields without any response body. May invalidate cache. • OPTIONS – Retrieve information about what is allowed to do with the Resource. Should be used to self document the API. (WADL?). Non cacheable Not used • TRACE – Echo the request as seen by the Server – debugging purposes • CONNECT – Proxy and tunelling
    12. 12. nouns • GET /something-ID : returns something • PUT /something-ID : send something to be stored at something-ID • POST : send something: returns something-ID • DELETE /something-ID • PATCH /something-ID : send data, to update something at something-ID
    13. 13. http status codes • 1xx – experimental • 2xx – Successful [*] • 3xx – Redirection • 4xx – Client Error • 5xx – Server Error [*] – ?? suppress_response_codes=true (e.g. twitter)
    14. 14. Typical status codes • 200 OK – [GET] – The Client requested data from the Server, and the Server found it for them (Idempotent) • 201 CREATED – [POST/PUT/PATCH] – The Client gave the Server data, and the Server created a resource • 204 NO CONTENT – [DELETE] – The Client asked the Server to delete a Resource, and the Server deleted it • 400 INVALID REQUEST – [POST/PUT/PATCH] – The Client gave bad data to the Server, and the Server did nothing with it (Idempotent) • 403 - Forbidden – The Client request was understood but denied by the server. (idempotent) • 404 NOT FOUND – The Client referenced an inexistent Resource or Collection, and the Server did nothing (Idempotent) – 410 GONE: resource no longer available • 500 INTERNAL SERVER ERROR – The Server encountered an error, and the Client has no knowledge if the request was successful • 501 NOT IMPLEMENTED – The Client request is valid, but the server does not provide the service. (Idempotent)
    15. 15. Use case: Asset repository • Sketchup warehouse 3dwarehouse.sketchup.com • 3dvia warehouse www.3dvia.com • TurboSquid marketplace www.turbosquid.com • Many more.. tf3dm.com, daz3d.com, poserworld.com, orbolt.com, … No API provided so far – human html browsing only.
    16. 16. Sketchup warehouse main
    17. 17. Sketchup warehouse model
    18. 18. Sketchup warehouse download https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=d5d13872-32c2-4fb2-8b54-5291c38ac3a9&fn=FurnessLibrary.kmz
    19. 19. Asset vs Resource & Collection • A 3D asset contains multiple Resources – Geometry, images, animation, shaders, media … – A 3D asset is a Collection, not a Resource • A 3D asset may be combined into a single Resource – e.g. Google .kmz :: a zip file contains .dae and images – e.g. Open XML formats :: a zip file contains XML and images, audio, video…
    20. 20. response header Accept-Ranges:bytes Access-Control-Allow-Origin:* Age:35 Cache-Control:max-age=2592000 Connection:keep-alive Content-Disposition:attachment; filename=FurnessLibrary.kmz Content-Length:7276392 Content-Type:application/vnd.google-earth.kmz Date:Sat, 05 Jul 2014 01:26:26 GMT Expires:Mon, 04 Aug 2014 01:26:26 GMT Last-Modified:Sat, 05 Jul 2014 01:26:26 UTC Last-Modified:Sat, 05 Jul 2014 01:26:26 GMT Pragma:cache Server:Restlet-Framework/2.1.7 Via:1.1 854f95df95dd97f4a6eb1132740f1b0e.cloudfront.net (CloudFront) X-Amz-Cf-Id:Wjceyv_4T_rwOZhw-hZbPbsDGNpKU2QMWnyDogUJCgICJWD_VPxy5w== X-Cache:Hit from cloudfront https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=d5d13872-32c2-4fb2-8b54-5291c38ac3a9&fn=FurnessLibrary.kmz
    21. 21. MIME types Name MIME Type File Extension More Details COLLADA model/vnd.collada+xml .dae IANA: COLLADA Google Earth - Zipped KML application/vnd.google- earth.kmz .kmz IANA: Google Earth JSON (generic) application/json .json RFC7158 Portable Network Graphics image/png .png RFC 2083 X3D (XML) model/x3d+xml .x3d (pending IANA) XML (generic) application/xml .xml RFC Zip archive application/zip .zip
    22. 22. 3DVIA download
    23. 23. 3DVIA .dae download Status Code:302 Found Cache-Control:no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Connection:Keep-Alive Content-Encoding:gzip Content-Length:20 Content-Type:text/html; charset=UTF-8 Date:Sat, 05 Jul 2014 23:27:51 GMT Expires:Thu, 19 Nov 1981 08:52:00 GMT Keep-Alive:timeout=5, max=100 Location:/3dsearch/Content/F1AA50E7F9CBDDEF.zip?Downloader=2&Pass= Pragma:no-cache Server:Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8e mod_jk/1.2.25 PHP/5.3.6 Vary:Accept-Encoding X-Powered-By:PHP/5.3.6 X-UA-Compatible:IE=edge, Chrome=1 http://www.3dvia.com/download.php?media_id=F1AA50E7F9CBDDEF& ext=zip&file=%2F3dsearch%2FContent%2FF1AA50E7F9CBDDEF.zip%3FDownloader%3D2%26Pass%3D
    24. 24. 3DVIA .dae (Zip) download Cache-Control:no-cache, must-revalidate Connection:Keep-Alive Content- Disposition:attachment;filename="B_ldm_FussenNeusch wansteinCastle_T3.dae.zip" Content-Type:application/zip;charset=UTF-8 Date:Sat, 05 Jul 2014 23:00:27 GMT Expires:Sat, 05 Jul 2014 23:00:27 GMT Keep-Alive:timeout=5, max=99 Server:Apache-Coyote/1.1 Transfer-Encoding:chunked http://www.3dvia.com/3dsearch/Content/F1AA50E7F9CBDDEF.zip?Downloader=2&Pass=
    25. 25. 3DVIA .3dxml download
    26. 26. 3DVIA .3dxml download Accept-Ranges:bytes Cache-Control:must-revalidate Connection:Keep-Alive Content-Disposition:filename="Neuschwanstein Castle in Bavaria Germany.3dxml" Content-Length:1120364 Content-Type:application/x-3dxmlplayer;charset=UTF-8 Date:Sat, 05 Jul 2014 23:35:23 GMT ETag:F1AA50E7F9CBDDEF.3dxml-11186c-4df9a197 Expires:Sat, 05 Jul 2014 23:35:23 GMT Keep-Alive:timeout=5, max=100 Last-Modified:Thu, 16 Jun 2011 06:24:45 GMT Server:Apache-Coyote/1.1 http://www.3dvia.com/3dsearch/Content/F1AA50E7F9CBDDEF.3dxml !!
    27. 27. rest3d - assets • 3d assets are Collections – They can be downloaded as zip archives – Some have a specific organization, with defined mime-type and file extension (kmz, daz, …) – Asset need a specific ‘scene’ entry point – Allow for downloading specific resource (e.g. x.zip/image/i.png) • rest3d should allow exploration of assets Resources – Differentiate APIs for returning asset data vs info • Server should store resources separately – Pack on demand + cache – Allow for content update (new image patch..) – Potentially save space on server – sharing resources • rest3d Collections contain Resources and/or other Collections.
    28. 28. Rest3d – warehouse proxy GET /rest3d/info/warehouse Get top level collections GET /rest3d/search/warehouse Search for models with string in name GET /rest3d/info/warehouse/?uuid= Get Collection or Asset composition GET /rest3d/data/warehouse/?uuid= Get an Asset (kmz or zip) or Resource (not yet implemented)
    29. 29. rest3d API design • /rest3d prefix • /rest3d/info/.. -> returns information (metadata) • /rest3d/data/.. -> returns data (e.g. kmz, zip, png) • /rest3d/search/.. -> returns matching set • /rest3d/X/database -> database selection • /rest3d/X/database?uuid= -> Universally unique identifier for specific Resource or Asset or Collection GET /rest3d/what/where/item
    30. 30. http://127.0.0.1:8000/rest3d/info/warehouse/?uuid=c _f8fc62afb9653d03cbe9bc85eececab4 { "database": "warehouse", "mimetype": "model/rest3d-collection+json", "page": { … }, "uuid": "c_f8fc62afb9653d03cbe9bc85eececab4", "children": { "Sketchyphysics ragdoll -- normal": { "uuid": "m_f4c7e11f3d2f35181d30c9736bb0eb9f_k2", "mimetype": "application/vnd.google-earth.kmz", … }, "Sketchyphysics ragdoll -- man": { "uuid": "m_f13c1e22e236d110a30b36db25fb7e21_k2", "mimetype": "application/vnd.google-earth.kmz", … }, "Terrain": { "uuid": "m_f0e5ff51877e125e788bfa6b4507970a_k2", "mimetype": "application/vnd.google-earth.kmz", … }, …, }
    31. 31. http://127.0.0.1:8000/rest3d/search/warehouse/fisher %20fine%20art%20library { "database": "warehouse", "mimetype": "model/rest3d-collection+json", "page": { … }, "search": "fisher%20fine%20art%20library", "children": { "Fisher Fine Arts Library": { "uuid": "m_71187c14aab1b33bdc0d3b8ac51c2a32_k2", "mimetype": "application/vnd.google-earth.kmz", "iconUri": "https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=1837478f-4bb9-4e91-81d5-83a6587356fb&fn", "modelUri": "https://3dwarehouse.sketchup.com/model.html?id=71187c14aab1b33bdc0d3b8ac51c2a32", "previewUri": "https://3dwarehouse.sketchup.com/embed.html?entityId=71187c14aab1b33bdc0d3b8ac51c2a32", "description": "Furness's library at Penn is a large Gothic structure of red brick, brownstone, and terra-cotta. With its separation of the reading room from the book stacks, the building was considered highly innovative in its day. Light was admitted to the book stacks through a sloping glass roof and down through translucent glass floors. Furness studied in the atelier of Richard Morris Hunt who later became a mentor for Louis Sullivan. Model by Noel Nemcik.", "created": { "date": 1176163749000, "user": "1006372608444182407313338" }, "latitude": 39.9516751764, "longitude": -75.1927958996 } } }
    32. 32. rest3d gui – Collections
    33. 33. http://127.0.0.1:8000/rest3d/search/warehouse/fisher %20fine%20art%20library { "database": "warehouse", "mimetype": "model/rest3d-collection+json", "page": { … }, "search": "fisher%20fine%20art%20library", "children": { "Fisher Fine Arts Library": { "uuid": "m_71187c14aab1b33bdc0d3b8ac51c2a32_k2", "mimetype": "application/vnd.google-earth.kmz", "iconUri": "https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=1837478f-4bb9-4e91-81d5-83a6587356fb&fn", "modelUri": "https://3dwarehouse.sketchup.com/model.html?id=71187c14aab1b33bdc0d3b8ac51c2a32", "previewUri": "https://3dwarehouse.sketchup.com/embed.html?entityId=71187c14aab1b33bdc0d3b8ac51c2a32", "description": "Furness's library at Penn is a large Gothic structure of red brick, brownstone, and terra-cotta. With its separation of the reading room from the book stacks, the building was considered highly innovative in its day. Light was admitted to the book stacks through a sloping glass roof and down through translucent glass floors. Furness studied in the atelier of Richard Morris Hunt who later became a mentor for Louis Sullivan. Model by Noel Nemcik.", "created": { "date": 1176163749000, "user": "1006372608444182407313338" }, "latitude": 39.9516751764, "longitude": -75.1927958996 } } }
    34. 34. http://127.0.0.1:8000/rest3d/info/warehouse/?uuid=m _71187c14aab1b33bdc0d3b8ac51c2a32_k2 { "database": "warehouse", "mimetype": "model/rest3d-collection+json", "uuid": "m_71187c14aab1b33bdc0d3b8ac51c2a32_k2", "children": {}, "assets": { "doc.kml": "doc.kml", "images/texture0.jpg": "texture0.jpg", "images/texture0.png": "texture0.png", "images/texture1.jpg": "texture1.jpg", "images/texture1.png": "texture1.png", "images/texture10.jpg": "texture10.jpg", "images/texture11.jpg": "texture11.jpg", "images/texture12.jpg": "texture12.jpg", "images/texture13.jpg": "texture13.jpg", "images/texture59.jpg": "texture59.jpg", … snip snip … "images/texture6.jpg": "texture6.jpg", "images/texture60.jpg": "texture60.jpg", "images/texture61.jpg": "texture61.jpg", "images/texture7.jpg": "texture7.jpg", "images/texture8.jpg": "texture8.jpg", "images/texture9.jpg": "texture9.jpg", "models/model.dae": "model.dae", "textures.txt": "textures.txt" } } ---------------- resources
    35. 35. rest3d ui – asset composition
    36. 36. content preview • Web server can provide content preview – Send javascript and content – rest API does not need to define content format • Example: Pre-calculated set of images https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId=fa 6c02bc-9047-4c65-b06e-7e669140c74c • Example webGL viewer https://3dwarehouse.sketchup.com/embed.html?entityId=uc6f7b2c8- 6775-4ef9-9ca9-4a7d2b18ee0f – Note: This is a skj file ! https://3dwarehouse.sketchup.com/3dw/getpubliccontent?contentId =310c7591-b763-4dc3-8ace-5e1bc955cb5e&fn • rest3d example embedded viewer – http://127.0.0.1:8000/viewer/easy- viewer.html?file=/models/cat/20_cat_smooth_bake_channel.json&ba ckground=328a15
    37. 37. rest3d application • 3D viewer – COLLADA loader – glTF loader – rest3d webGL viewer • ‘workstation’ style UI – Split window in viewports, resize, move – ‘applets’ running inside tabs – Based on jquery-ui, jquery-layout • Tree views – Scene – Database exploration – …
    38. 38. http://127.0.0.1:8000/viewer/viewer6.html
    39. 39. rest3d implementation sketchup 3dvia xml db rest3d server (nodejs)
    40. 40. rest3d drivers • Provided drivers for: – warehouse/ – proxy to sketchup warehouse – 3dvia/ – proxy to 3dvia warehouse – tmp/ – temporary space available to user • Stored as files on harddirve – db/ - XML database • Currently using eXist-db http://exist-db.org/ • Could use other databases e.g baseX http://basex.org/ • Please contribute .. add more ‘drivers’ !
    41. 41. Authentication (e.g. asset store, private cloud..) • 3dvia example. Requires login. – Do not store login information on rest3d server – Login information is per database server – Protocol specific to each database server • Authentication is not a rest3d API – But it needs to be part of the implementation – https is mandatory for sending password
    42. 42. Proxy authentication Web browser (client) POST /rest3d/login/3dvia User: username Passwd: password Status Code 2xx or 4xx use cookie or http headers https rest3d server (nodejs) POST www.3dvia.com/login signing[user_id]: signing[user_pwd]: Save cookies cookies['www.3dvia.com']['/']. PHPSESSID cookies['3dvia.com']['/']['3DVI A_SESSION'] Return rest3d sessionID (cookie or custom http header) https 3dvia Check credentials Return cookies Login/password is not stored on rest3d server 3dvia cookies are not available to the client. - Web browser will not send 3dvia.com cookies to rest3d server (security) - Web browser maintain sessionID with rest3d server
    43. 43. Proxy request principle Web browser (client) GET /rest3d/info/3dvia/.. cookie: sessionID Data is available https rest3d server (nodejs) 3dvia drive retrieve 3dvia cookies from sessionID store GET www.3dvia.com/… + 3dvia cookies return data https 3dvia Check credentials Return data rest3d server has a cache – local file system When possible - data is public & available as-is on – rest3d server sends a 302
    44. 44. Third party authentification (e.g. Google oauth2) Web browser (client) POST /rest3d/login/db POST /google/login Interactive session connected Google Cookie is set http/rest3d?openid.xx Reload! https rest3d server (nodejs) 302 www.google.com/login ReturnURL =/rest3d/auth/ret http://Return URL Map googleID with SessionID 302 Final redirect (/viewer/) https google Login page, set cookies 302 ReturnURL With openid.xx params Check credentials Oauth does not work in iFrame potentially lost local data from web browser
    45. 45. DB access example Web browser (client) GET /rest3d/info/db/.. cookie: sessionID Data is available https rest3d server (nodejs) Retrieve GoogleID from sessionID GET www.xml.com with GoogleID (or other protocols) return data XML Database Check credentials Return data
    46. 46. search (warehouse)
    47. 47. search (3dvia)
    48. 48. rest3d search API ? (tbd) • Simple: query string in title / description – /rest3d/search/database?q=xxxx • Complex: Xquery ? – List of available fields for search ? – Search within a .dae file – e.g. return <node> with specific id collection('/db/assets')//node[@id = $q]
    49. 49. Use case: content delivery • E.g. html page requests individual resources – Paging: Content is downloaded as needed • Content formatting specific to target platform • Scalability and Caching are very important • Live update – Update server without disturbing current users
    50. 50. CDN CDN service provides multiple servers at several locations, faster service to end user 1. rest3d request redirect to CDN service or provides links to resources within the CDN 2. if CDN has content, it delivers. Otherwise CDN makes request to rest3d to get content. -> CDN caching works best when no requests are made to rest3d server (not even a 304), and no redirects are done to begin with
    51. 51. Live update / Manifest • Manifest contains list of all resources and their (CDN) URL • Application use Manifest to translate resource to URL • Live update: – Server creates all new assets, and new manifest – New clients get the new manifests, current clients still use old manifest – Option: Once all old clients have disconnected, delete old assets/manifests • rest3D API – url vs uuid
    52. 52. url vs uuid http://127.0.0.1:8000/rest3d/info/db { "database":"eXist", "name":"/", "mimetype":"model/rest3d-collection+json", "created":{ "date":1406848785182, "user":0 }, "uuid":"287083e0-1909-11e4-8f0a-b1e5aa401bc4", "children":{ }, "assets":{ "tmp/images/texture0.jpg":"639337b0-1909-11e4-8f0a-b1e5aa401bc4", "tmp/images/texture0.png":"63b565b0-1909-11e4-8f0a-b1e5aa401bc4", "tmp/textures.txt":"63be8d70-1909-11e4-8f0a-b1e5aa401bc4", "tmp/models/Chicken.dae":"63c74000-1909-11e4-8f0a-b1e5aa401bc4", "tmp/doc.kml":"63d17930-1909-11e4-8f0a-b1e5aa401bc4" } } Those two URLs return the same resource: http://127.0.0.1:8000/rest3d/info/db/tmp/images/texture0.jpg http://127.0.0.1:8000/rest3d/info/db?uuid=639337b0-1909-11e4-8f0a-b1e5aa401bc4
    53. 53. Caching • Basic internet cache – Cache is based on entire URL, including parameters • http://server.com/test.htm?p=1 ≠ http://server.com/test.htm?p=2 – Client check expiration date – If past due, ask again for resource – Get 304 from server -> what you have still valid • Update expiration date – Get new resource • Best if for assets to never expire – No communication to server at all – Ok to have lots of resources – Manifest -> create all new assets when needed
    54. 54. Format conversion • IMHO - ok to use file extension as format conversion request – Thinking about: • http://127.0.0.1:8000/rest3d/info/db/tmp/images/texture0.jpg • http://127.0.0.1:8000/rest3d/info/db/tmp/images/texture0.png • http://127.0.0.1:8000/rest3d/info/db/tmp/images/texture0.jpg?size=1024x1024 – For reference: Amazon size code style: • http:/…/uuid.COUNTRY_CODE.SIZE_CODE.jpg • Resource uuid -> one format, one version • Asset uuid -> one format (e.g. collada zipped package (.daz), google earth package (.kmz) • work in progress: dae2gltf in the server – [POST] rest3d/convert/tmp/ convert .dae to glTF Returns result into tmp database Input from url (external or rest3d uuid) Auto-recognize input (kmz, zipped dae ..)
    55. 55. Jobs (WIP) • Need to manage server load, or distribute work • Create a return (immediately) a jobid, do not wait for job to complete before responding to request • GET /rest3d/jobs – Get jobs list • GET /rest3d/jobs/status/jobID • POST /rest3d/jobs/suspend/jobID • POST /rest3d/jobs/resume/jobID • POST /rest3d/jobs/kill/jobID POST vs GET ?
    56. 56. Use case: Content Creation • Import/Export – Tools can import/export/browser using rest3d API • Versioning • Local changes – WYSIWYG – patches • User/Teams – Projects – Settings
    57. 57. Downloading::compression • Server will compress before sending data – Why storing .gziped files then? • Asset repository -> need packaging. Why not tar? • Content Delivery -> web app need to unzip/unpack in JavaScript – Slower than native code. – Patch html to load images/css/scripts – Reducing the number of queries could be the right trade-off
    58. 58. Uploading • Web browser cannot access hard drive – For security. User has to provide files • Open file. Drag & Drop files • There is no option to automatically gzip files from browser to server. – Can zip content in JavaScript. May still be the right thing to do. • rest3d API Upload (multiple) files [POST] /rest3d/db/collection-path <form enctype="multipart/form-data" method="POST"> <input type="file” multiple="multiple" /> </form> Upload from url [POST] /rest3d/db/collection-path <form enctype="multipart/form-data" method="POST"> <input id=”url" value=”http://…" /> </form> Create Collection [POST] /rest3d/db/collection-path <form enctype="multipart/form-data" method="POST"> <input type="file" name="upload" multiple="multiple" /> </form>
    59. 59. Versioning (WIP) • One uuid = single version, single format? • Current implementation: – Collection(asset) list resources (url) with proxy uuid – Proxy uuid contains list of uuid with version info • Better implementation? – Collection list resources with current version uuid – Need way to browse versions from current version – Similar if not identical to git architecture • note: sha1 used for uuid. If new content is same as previous content, sha1 will be identical. Save space ! • http://git-scm.com/book/en/Git-Internals
    60. 60. rest3d versioning http://127.0.0.1:8000/rest3d/info/db/tmp/models/Chicken.dae Collection Path
    61. 61. Indirection (versioning) {"name":"Chicken.dae","mimetype":"model/collada+xml","size":2694152,"created": {"date":1406848884309,"user":"TK7JwRquJhSsNmR5ef67Fx0FnolvmhGqLO71cTK3"} ,"uuid":"63861850-1909-11e4-8f0a-b1e5aa401bc4"}
    62. 62. Patches - • WYSIWYG changes – Final content get patched inside game engine – Patch files are create, and automatically applied to final content when loaded next time – 1 patch per resource – Patches can be stored locally, or on server • Server side patch application – Reverse content pipeline? • Seems very hard if not impossible – Apply patch after (on-demand) conversion • Local or server based – Patch source content • Server apply patch • Then apply on-demand content conversion • Or apply patch to source content • Repeatable patching – Need UUID • Most exporters do not provide consistent IDs 
    63. 63. Use case: user content
    64. 64. Use case: user content (sell)
    65. 65. To be continued ! Thank you for your attention Questions and comments are welcomed contact info: remi@acm.org Source code: github.com/amd/rest3d

    ×