Database Driven3D Content Management        Systems      By Tim Child      3DMashUp
Outline•   Biography•   3D Content Applications•   3D Content Categories•   3D Geometry•   3D Samples•   3D UDT’s & Functi...
Biography• Tim Child• 35 years experience of software development• Formerly   –   VP Oracle Corporation   –   VP BEA Syste...
3D CMS Issues•   Ad Hoc•   File/SCM based•   Large number for file formats•   Files dispersed across the users network•   ...
3D Content Applications• Augmented/Virtual Reality• Medical        10x growth in 5yr - “Baptist Hospital”• AEC / BIM      ...
3D Content             Catagories “Vector Graphics” “Raster Graphics”• Geometry                         • Samples  – Mathe...
Representing             3D Geometry• Point            • Polygon• Line                   • Mesh• Triangle
Representing 3D Samples• Pixel/Voxel     • Quantized Intensity I (x,y,z,t)• Point Cloud• Voxel Array
PostgreSQL              Advanced Capabilities•   Functions     –   Methods executing with in the server     –   Consider t...
System Overview    DCC                  Plug-                                          File   Tools                       ...
System Architecture
3D UDTs & Functions Current 3D Schema • 75+ Functions • 15+ UDT’s • 20 + Tables
Recap• Native SQL representation of 3D objects• Supports  – Vector Graphics  – Raster Graphics• 3D Content Aware  – Vector...
IMPORTANT                      3D QUERIESDBMS Goal:“Provide sufficient data to rapidly render the scene with acceptable fi...
Camera View          Simplify Camera          Field of View to               a Cone
Spatial Cone Query
Spatial IndexingSpatial Indexing allows scalable queries from > 100M Objects
View Coherent   Queries
Mesh SizeStanford Bunny                              XYZ Dragon• 70K Triangles                             • 7.2M Triangle...
Mesh Spatial                             ProcessingEach mesh subdivided into1024 x 1024 x 1024 cells                      ...
Spatially Clustered           Mesh Storage                                             8 KB DBMS Pages                Quer...
LOD Processing            On loading a Mesh            Compute and Store              Reduced LOD             Representati...
Triangle Decimation                Edge              CollapsingCost based algorithm       Collapsing Cost          based o...
LOD Queries                          Document                            Table  Query path                                ...
PostgreSQL           Rules and Triggers• Select, Update, Delete Rules      Do ALSO Command, Command, …          – Extends ...
Recap• Camera SQL Queries   –   Selects only what’s visible• View coherent queries   – Optimizes scene data retrieval for ...
Demo
GPGPUACCELERATION
GPGPU                   Goals• OpenCL a PostgreSQL Procedural Language  – OpenCL Kernels execute as SQL UDF’s• Type Mappin...
WEB 2.0INTEGRATION
Web 2.0 Goals• Provide a Rich Web 2.0 UI• Integration with Web 2.0 Apps• Programmability• Tracking & Analytics
Rich UI• HTML 5 –2D Canvas –WebGL –Drag & Drop –….• Others? –Silverlight for IE9
Web 2.0 Integration•   Linking•   Embedding•   Social Bookmarking•   Tagging•   Drag & Drop•   Annotations•   Mash-Ups
Programmable• REST API’s Supports  – Browsers & other Web 2.0 Apps• SQL Queries Supports  – LAMP, RoR, POJ, ASP.Net, …• XM...
Tracking & AnalyticsWeb Browsers           Who’s looking at what?                                  HTTP                Con...
3D MashUp                                  Service                                                                      We...
CONTENTMANAGEMENT
CMS Goals• Track Revisions & Changes• Compare documents• Publish Documents• Annotation• Metadata Queries
Content Versioning             All Database Rows include Version ID        Versioning at the File, Section, and Record Lev...
Contents Diff               Objects in the database for the 3D content                     are signed CryptographicallyRev...
PERFORMANCE
Performance Challenges– Visual Scenes   • 20K – 5M Polygons– Near Real-Time Queries   • Queries 15m – 30ms end user respon...
Further Development                    Areas• Performance Investigations  – What are the important queries?  – Where are t...
Summary• Why a 3D in a Database?   –   Shared   –   Multi-user   –   Scalable   –   Reliable   –   Extensible   –   Secure...
Q&A
Upcoming SlideShare
Loading in...5
×

Using postgre sql for 3d cms

1,512

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,512
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Using postgre sql for 3d cms

  1. 1. Database Driven3D Content Management Systems By Tim Child 3DMashUp
  2. 2. Outline• Biography• 3D Content Applications• 3D Content Categories• 3D Geometry• 3D Samples• 3D UDT’s & Functions• System Architecture• 3D Queries• Demo• GPGPU Acceleration• Web 2.0 Integration• Performance Challenges• Further Development• Summary• Q & A Session
  3. 3. Biography• Tim Child• 35 years experience of software development• Formerly – VP Oracle Corporation – VP BEA Systems Inc. – VP Informix – Leader at Autodesk, Navteq, Intuit, …• 30 + years experience in 3D, CAD, GIS and DBMS• Built >10 Spatial DBMS Applications
  4. 4. 3D CMS Issues• Ad Hoc• File/SCM based• Large number for file formats• Files dispersed across the users network• Hard to Sync content between files• Poor Web 2.0 Integration• Need of Real-Time Display
  5. 5. 3D Content Applications• Augmented/Virtual Reality• Medical 10x growth in 5yr - “Baptist Hospital”• AEC / BIM “Arup” predicts 70% growth in BIM data/yr• GIS / Earth Sciences /Environmental Sciences• CAD / CAM• Games / 3D Video
  6. 6. 3D Content Catagories “Vector Graphics” “Raster Graphics”• Geometry • Samples – Mathematical Representations – Measurements • Location X,Y,Z – Topological Rules • Time t • Channels – R,G,B – Many varieties of Objects – Intensity levels – Discreet/Quantized – Many File Formats – Noisy – Need Calibration – Scales N * N * N
  7. 7. Representing 3D Geometry• Point • Polygon• Line • Mesh• Triangle
  8. 8. Representing 3D Samples• Pixel/Voxel • Quantized Intensity I (x,y,z,t)• Point Cloud• Voxel Array
  9. 9. PostgreSQL Advanced Capabilities• Functions – Methods executing with in the server – Consider them a Delegate Methods • Take the method to the data …• Languages – PGSQL – C (UDF) – Java – Others, …• Types – Complex types and Arrays – Used Defined Types (UDT) – Examples include • XML, JSON, PostGIS• Indices – BTree 1 Dimensional – RTree Multi Dimensional – Inverted Index Text searches – GIST Roll your own• User Defined Aggregate (UDA) – Supports Sum, Min, Max, Average, …
  10. 10. System Overview DCC Plug- File Tools WebDav In Storage (COLLADA) Existing CMS/SCM Web Browser HTTP View Server DBMS (WebGL) Server Web Pages 3D SQL Table MashUp Storage Service Web / File Crawler Authentication 3D Service Analytics Service File Active File LDAPStorage Directory Storage
  11. 11. System Architecture
  12. 12. 3D UDTs & Functions Current 3D Schema • 75+ Functions • 15+ UDT’s • 20 + Tables
  13. 13. Recap• Native SQL representation of 3D objects• Supports – Vector Graphics – Raster Graphics• 3D Content Aware – Vector math – 4 x4 Homogenous coordinate transformations – Polygon to Polygon Intersections• Image processing
  14. 14. IMPORTANT 3D QUERIESDBMS Goal:“Provide sufficient data to rapidly render the scene with acceptable fidelity”
  15. 15. Camera View Simplify Camera Field of View to a Cone
  16. 16. Spatial Cone Query
  17. 17. Spatial IndexingSpatial Indexing allows scalable queries from > 100M Objects
  18. 18. View Coherent Queries
  19. 19. Mesh SizeStanford Bunny XYZ Dragon• 70K Triangles • 7.2M Triangles• 39K Vertices • 3.6M Vertices No point in using 7M triangles to render 64 x 64 pixels
  20. 20. Mesh Spatial ProcessingEach mesh subdivided into1024 x 1024 x 1024 cells Each triangle centroid is sorted on a Z Coordinate 0 - 1023 Morton Z Coordinates Each 10 bit coordinate Is “bit interleaved” (Z9,Y9,X9, … Z0,Y0,X0)
  21. 21. Spatially Clustered Mesh Storage 8 KB DBMS Pages Queries Each page Contains 50 – 100 PostgreSQL Mesh Triangles Server Spatially ClusteredSpatially clustered meshes optimizes DBMS access!
  22. 22. LOD Processing On loading a Mesh Compute and Store Reduced LOD Representations
  23. 23. Triangle Decimation Edge CollapsingCost based algorithm Collapsing Cost based on Area and Angle Between
  24. 24. LOD Queries Document Table Query path Query path for forDetailed Objects Coarse Objects Mesh Table Triangle LOD Determined by Query Triangle Cluster Based on Object Size Cluster LOD 0 and Query Distance LOD N
  25. 25. PostgreSQL Rules and Triggers• Select, Update, Delete Rules Do ALSO Command, Command, … – Extends SQL operations Do INSTEAD Command, Command, … – Replaces SQL operations• Insert, Update, Delete Trigger Events – Before Event Command Or – After Event Command SQL Views and Rules System hides implementation complexity
  26. 26. Recap• Camera SQL Queries – Selects only what’s visible• View coherent queries – Optimizes scene data retrieval for camera motions• 2 Level Spatial Indexing – RTree Supports large number of (>100M) objects – Z-Ordering improves finer access for large meshes• Spatially Clustered Storage – Optimizes partial mesh access• LOD processing – Optimizes speed versus size for large meshes• PostgreSQL – UDT’s and UDF encapsulate functionality in server – Rules System hides implementation complexity
  27. 27. Demo
  28. 28. GPGPUACCELERATION
  29. 29. GPGPU Goals• OpenCL a PostgreSQL Procedural Language – OpenCL Kernels execute as SQL UDF’s• Type Mapping, support common data types – Vectors, Arrays, Images• Sorting – Used GPGPU based sorts in SQL queries
  30. 30. WEB 2.0INTEGRATION
  31. 31. Web 2.0 Goals• Provide a Rich Web 2.0 UI• Integration with Web 2.0 Apps• Programmability• Tracking & Analytics
  32. 32. Rich UI• HTML 5 –2D Canvas –WebGL –Drag & Drop –….• Others? –Silverlight for IE9
  33. 33. Web 2.0 Integration• Linking• Embedding• Social Bookmarking• Tagging• Drag & Drop• Annotations• Mash-Ups
  34. 34. Programmable• REST API’s Supports – Browsers & other Web 2.0 Apps• SQL Queries Supports – LAMP, RoR, POJ, ASP.Net, …• XML Input / Output support – Document Exchange and Import / Export• JSON Output provides – Easy integration with JavaScript
  35. 35. Tracking & AnalyticsWeb Browsers Who’s looking at what? HTTP Content Server Queries HTTP Request / Response Log Queries Logging DBMS Service View Web Analytics Analytics Analytics Report Reports Service Queries
  36. 36. 3D MashUp Service Web Site MashUp A Meta Data URI URI HTTP Look-Up Request Request / Response ContentBrowser HTTP Filter URI Request / Transform HTTP Request Response Clip Request / Merge Response 3D MashUp Engine Web Site Acts as Proxy between Browser and Web Sites B Filtering, Transforming, Clipping and Merging Content Responses
  37. 37. CONTENTMANAGEMENT
  38. 38. CMS Goals• Track Revisions & Changes• Compare documents• Publish Documents• Annotation• Metadata Queries
  39. 39. Content Versioning All Database Rows include Version ID Versioning at the File, Section, and Record LevelsRev 0 Rev 1 Rev 2 Rev 3 Rev 4 Rev 1.0 Rev 1.1
  40. 40. Contents Diff Objects in the database for the 3D content are signed CryptographicallyRevision N Revision M Diff SHA1 SHA1 Diff SHA1 SHA1 Diff SHA1 SHA1 Diff SHA1 SHA1
  41. 41. PERFORMANCE
  42. 42. Performance Challenges– Visual Scenes • 20K – 5M Polygons– Near Real-Time Queries • Queries 15m – 30ms end user response– How much can I retrieve in 16.7 ms? • DB Size < RAM • DB Size >= Ram Size && <= 1 T B • DB Size > 1 T B
  43. 43. Further Development Areas• Performance Investigations – What are the important queries? – Where are the bottlenecks?• Picking Operations – Using a Cylinder to pick from a view• Editing Operations – Updating Geometry• OpenCl as Procedural Language – Allows “cleaner” OpenCl integration
  44. 44. Summary• Why a 3D in a Database? – Shared – Multi-user – Scalable – Reliable – Extensible – Secure and well define Security Model – Rich Tools Set• Why choose PostgreSQL for 3D – Advanced capabilities – Open and Opensource – Versatile – Simple to use – Robust Developer Community – Leverages existing skill sets
  45. 45. Q&A

×