InfoGrid Core Ideas
by InfoGrid.org on Jul 06, 2009
- 10,825 views
Statistics
- Likes
- 8
- Downloads
- 51
- Comments
- 2
- Embed Views
- Views on SlideShare
- 10,747
- Total Views
- 10,825
Pierre Bellet, Creative Technologist at 909c, conseil & production social media
4 months ago
tinoviena
10 months ago
rickylee86
1 year ago
Osman Blake, EU Lead, Creative Technology at SapientNitro
2 years ago
rrahulhansuka
2 years ago
gzugic
3 years ago
Régis Gaidot, Co-Founder / CTO / Lead Developer at Boosket
3 years ago
katherine goodier, cko at L-3
3 years ago
InfoGrid Core Ideas
1–2 of 2 previous next
Once you are comfortable with these ideas, much of InfoGrid just falls into place.
You can develop perfectly fine InfoGrid applications with applying just the first two ideas. But the more of these core ideas you apply, the more value you get out of InfoGrid.
In this slide deck, we’ll discuss those core ideas on a very high level. You can find more detailed information at infogrid.org.
All information in InfoGrid is represented as objects, called MeshObjects. Each circle in the diagram shows one example MeshObject.
Each MeshObject has a unique identifier, which typically is a URL. The diagram shows some MeshObject URLs in the example.com domain.
You can access any MeshObject in an InfoGrid application by typing its URL into the browser, or do the equivalent from your application code using an InfoGrid API. Of course, you do need to have the correct access rights to access the MeshObject; but assuming you do, any MeshObject in your application can be read or updated right from the browser at the MeshObject’s URL. That’s not only very easy to understand for users, because it feels very natural on the web, but it is also great for debugging InfoGrid applications because you can “see” all your data right from the browser. It also means that without further work from the developer, “everything” in your InfoGrid application can be bookmarked, hyperlinked, shared, rated, blogged, twittered, backtracked and all the other functionality in many new social web applications. It comes for free with InfoGrid for any piece of application data.
MeshObjects can relate to each other. The lines between the circles in the diagram represent their relationships. By relating MeshObjects to each other, you can construct arbitrarily complex molecules of information from atomic MeshObjects.
Creating an object first, and then blessing it with a type is very different from most programming environments or databases that you might be familiar with. For example, in a typical Java program you specify the class of the object at the time of construction, and the object can never change its class for as long as it lives. In SQL, you have to specify the table into which to insert the new row, and moving the row to another table while keeping relationships intact and without application code changes is so impossible to be almost ridiculous.
InfoGrid takes a different approach that turns out to be very consistent with how the web works: the information at any given URL on the web can change its type many times. (Think of any website URL that you have visited many times over some years – it was always the same URL, but the content might have changed dramatically over time.) A MeshObject is, conceptually, the object at that URL. So it makes sense that MeshObjects can change their type without being deleted first.
InfoGrid goes even beyond that. Any MeshObject can be blessed with more than one type at a time, so a MeshObject could be a Customer and an Employee at the same time. It could lose its Employee type if the person represented by the MeshObject quits the company, but still remains a Customer. In the history of information modeling, this is definitely unusual. But it can be very convenient and helps, as a side effect, to make InfoGrid information models simpler than their traditional counterparts.
You can think of blessing and unblessing with multiple types as similar to tagging on a site like delicious. Except, of course, that InfoGrid types are real types that define properties and the like, and that are enforced at run-time. So they go far beyond tagging, but nevertheless there is some similarity.
In InfoGrid, types define which properties an object carries. So once you bless a MeshObject with a type, it might start carrying a few properties initialized with default values. Which properties those are depends on how you defined the type. For example, a Customer type might have a name, an address, and a property that indicates whether the customer is late in payments or in good standing. After you have blessed a MeshObject with such a Customer type, you then can change the corresponding properties as you like. (Presuming, of course, that you have the access rights to do that.)
Types are defined in what we call InfoGrid models. You could also call them semantic models, objects models, schemas and the like. That would be correct, too. InfoGrid ships with a number of models, and you can define as many as you like yourself.
Just like for MeshObjects, any relationship can be blessed with more than one type, although blessing it with one is the most common case. Blessed relationships are typically directed, so you can distinguish, say, whether Person A is the daughter of Person B, or the other way around.
As you would expect, the InfoGrid run-time makes sure that objects and relationships can only be blessed in ways that are valid according to the rules defined in the model. For example, you can only bless a relationship with Customer_Places_Order if the source object is indeed a Customer, and the destination object an Order. The same is true for properties; for example, if a property is defined as required, you cannot assign a null value.
So far so good?
To repeat: what tables are to relational databases, the MeshObject Graph is to InfoGrid.
We believe that as an application developer, you should not have to worry about storing and retrieving MeshObjects or relationships from external storage such as a hard disk. In fact, your application logic should be completely independent of where your application data is stored, how it is serialized and so forth. There is no reason to hard-code storage decisions into application code as it is usually done in traditional applications that have SQL sprinkled all over. Imagine how much rewriting you’d have to do to port that application to grid storage like Hadoop or Amazon S3. Why? The application logic still is the same application logic!
In InfoGrid, choosing a persistence alternative is essentially a configuration step, not a programming step. Let’s see how this works:
We mentioned earlier that all MeshObjects in InfoGrid live in a MeshBase. In programming terms, MeshBase is an interface, not an implementation. InfoGrid provides a number of different implementations of MeshBase, some of which use different means to persist the MeshObjects.
The InfoGrid project also has developed code that stores MeshObjects in Amazon’s S3 and in Hadoop. No application logic changes are necessary, which proves this point.
If you don’t need persistence for your application, there is an in-memory-only implementation of MeshBase, again with the same API. This is sometimes very handy for automated test setups. And because everything is pluggable in InfoGrid, you can write your own, too.
InfoGrid takes care of reading and writing MeshObjects to and from persistent storage at the right time, aided by an in-memory cache that most MeshBases use for performance reasons. For example, if you have a hold on one MeshObject and you traverse one of its relationships to another MeshObject, InfoGrid will make sure that the found MeshObject gets restored from disk, so it is in your application’s address space just in time you want to access it. Or, if you make changes to a MeshObject or its relationships, InfoGrid will make sure they get written back to the Store. If you don’t use a MeshObject for some time, it will be automatically garbage collected from the cache.
To summarize: you can write your application logic independently from how you decide to store the data. Which persistence alternative you choose for your application can be configured after you built the application.
If the user types a URL to a MeshObject in their browser as discussed above, what determines what shows up on screen? In traditional web applications, URLs identify verbs (such as CGI scripts or servlets), which determine what is rendered on screen.
If InfoGrid uses URLs to identify abstract MeshObjects, instead of scripts, as discussed above, there must be another way of expressing how to render information. There is. In InfoGrid, it is called Viewlets.
The third core idea says: “neither a MeshObject by itself, nor a CGI script, determines by itself what is shown on screen, but a MeshObject and a Viewlet determine it together”.
For example, you could have a Customer Highlight Viewlet in your application that knows how to show the most important information about a Customer. In addition, you could have a Customer Detail Viewlet that provides much more verbose output. And you could retain the Property Sheet Viewlet that comes with InfoGrid that shows you the properties and relationships of any MeshObject on the level a programmer would look at it.
You could define, for example, that your application’s OrderViewlet is not suitable to render a Customer MeshObject, and that you prefer the summary viewlet over the detail viewlet. The applicable Viewlets are ranked, and the best choice is chosen. In our example, that is the Customer Highlight Viewlet. InfoGrid will then run that Viewlet, with the Customer MeshObject as parameter, to render the Customer.
This very REST-ful design has shown to make it much easier to rapidly develop web applications, in particular those that have different classes of users: one user doesn’t like the way you render their object? Just add another Viewlet to the application.
To remember: MeshObject plus Viewlet determines what is shown on screen. If no Viewlet is specified, the best-matching default is chosen.
By the way, did we mention that InfoGrid is very modular? You could fairly easily replace the Viewlet Framework with any other way of rendering MeshObjects that you could think of. InfoGrid used to run under Apache struts, so it definitely can be made to work. We just like Viewlets better because they leverage URLs to MeshObjects.
Imagine if your favorite computer operating system did not have access control on files in the file system. Imagine that access control to files would have to be implemented in each application, rather than by the operating system. As software architects, we would shudder. As security architects, we would give up.
But that is exactly the state of affairs for today’s typical web applications built using a relational database. It is the web application that enforces (or fails to enforce) which data object a user has access to. The relational database just authenticates the application, and then lets the application access all data, regardless of which user is using the application.
We think that is a very bad pattern. So InfoGrid puts security into the data layer instead of the application layer. Basically, it means that MeshObjects themselves decide who gets to perform which operation on the MeshObject, and make sure that only those users who are allowed to do so, may do so.
The processing thread processes the application logic, and as part of that, accesses one or more MeshObjects. For example, the application may attempt to read or write a property, determine related MeshObjects, bless or unbless a MeshObject, etc. When invoked, the MeshObject knows the operation to be performed, and the identity of the user attempting to perform the operation, and thus can decide whether or not to proceed.
In practice, this means MeshObjects delegate to an AccessManager that encapsulates the access control rules. As usual in InfoGrid, you are free to define your own AccessManager, including the no-security-one that says “let anybody do anything”, like a relational database would.
InfoGrid provides one particularly interesting AccessManager implementation that enables developers to define ACL-based access control rights using models, just like other InfoGrid models, which we think is cute.
To make this core idea memorable, you may want to remember that two different users can access the exact same MeshObject with the exact same application logic, and get two different results, because their access privileges are different.
An important subtype of MeshBase is the NetMeshBase. A NetMeshBase acts just like a MeshBase, with all of the associated features, except that it also knows how to keep the MeshObjects it holds consistent with replicas of the same MeshObjects in other NetMeshBases. Actually, in this context we should call the MeshObjects NetMeshObjects, because they are subtyped in parallel with NetMeshBase.
This feat is accomplished via a protocol called XPRISO, which stands for eXtensible Protocol for the Replication, Integration and Synchronization of distributed Objects. It’s a fairly complex protocol, and there is more information about it in other places; this slide deck can only give an overview.
1. “Hello, do you have a NetMeshObject with identifier http://example.com/abcd. If so, could you send me a read-only copy, and promise to notify me every time you update it?”
2. “Hello, I just changed property InGoodStanding from TRUE to FALSE on Customer NetMeshObject with identifier XYZ. I also unblessed its relationship to Order NetMeshObject 123. I believe you have a replica from me of that object. Because I updated my copy, you might want to update your replica, too and so I thought you might want to know”.
3. “Hello, please stop sending me updates for NetMeshObject 456. I have destroyed my copy and don’t need any more updates.”
Today, InfoGrid mostly uses XPRISO for the purposes of operating ShadowMeshBases and Probes as we discuss on the next slide. There is currently no code that allows XPRISO to operate over the network. But we expect that there will be; it’s a matter of “mere” programming to add it. Once it exists, an almost limitless number of NetMeshBases will be able to cooperate to serve an almost limitless graph of MeshObjects, many of which may not be InfoGrid native. Which brings us to the last core idea.
That is a powerful idea. Much developer effort today is spent all over the industry in accessing legacy (or modern) data through APIs and certain file formats, and even more effort is spent on transforming that data into a form that is suitable for any given application. InfoGrid enables developers to avoid a large amount of that work by making it easy for developers to make any kind of external data appear local, within a NetMeshBase, as a set of NetMeshObjects, just like any native InfoGrid object.
Imagine the power of this idea. If all of the world’s information can indeed be made look the same, uniform, automatically updated, replicated, stored etc. etc., the world is indeed a better place. For programmers, at least.
The diagram tries to convey this on a very high level. It shows data in many different places outside of InfoGrid, such as databases of various kinds, files of various kinds, and so forth. By virtue of XPRISO and the Probe Framework (about which you can find out more at infogrid.org) that external data can be replicated into a NetMeshBase, and become part of the regular graph of NetMeshObjects. As you would expect, InfoGrid takes care of keeping what’s in the NetMeshBase consistent with the external data over time even if the external data keeps changing after it has been imported. And, the arrows between the NetMeshBase and external data stores shown in the diagram are indeed bidirectional.
As a result, developers can simply pretend that the outside complexity of incompatible and disjointed data stores simply does not exist, because all they need to deal with for their application are MeshObjects and their relationships.
Where does this leave us? Well, admittedly it might leave you slightly overwhelmed, as some of these core ideas behind InfoGrid are rather different from the traditional ideas for how to build web applications. You may also be skeptical about whether this can actually be made to work.
But we hope you agree with us that if it can be made to work, the ideas behind InfoGrid are very powerful to overcome some of the limitations of the traditional way of implementing web applications.
So the best way to convince you that it can indeed work might be to suggest that you get your hands on some of the demonstration applications, like the MeshWorld or the NetMeshWorld, that can be used to explore these core ideas with actual running code. Of course, InfoGrid is open-source, so you can always look at the code itself.
For additional information, code, documentation and demonstration applications, please visit us at infogrid.org.