Community Developers need to maintain and improve DSpace.
End User Developers need to Customize installations.
Developers manipulating same code.
Results in Developers butting heads over merging contributions and new versions.
repeated changes to adapt DSpace to new customizations.
But is it the developers fault that these conflict occur?
No, its not their fault?
Need to innovate,
Need to change code,
Need to solve immediate issues first.
But is it the developers fault that these conflict occur?
No, its not their fault?
Need to innovate,
Need to change code,
Need to solve immediate issues first.
But is it the developers fault that these conflict occur?
No, its not their fault?
Need to innovate,
Need to change code,
Need to solve immediate issues first.
But is it the developers fault that these conflict occur?
No, its not their fault?
Need to innovate,
Need to change code,
Need to solve immediate issues first.
While DSpace Static Manager Design is simple. While they helped original developers be innovative and get their job done, they now limit us in the DSpace of today. And static methods do not make an API, Application Programming Interface) static methods are not extensible. Static methods cannot be overridden or re-implemented And we have a lot of these in DSpace Examples: PluginManager, ConfigurationManager, DatabaseManager, BitstreamStorageManager They are even present in DSpaceObject Model Static method even reference into other static Manager methods hardcoded in implementation. Users forced to customize static methods directly Repackaging modules: same Class, different behavior. Everyone (Users and Developers) are changing the implementation directly.
So, Anti-patterns represent possibly bad practices we might wnat to consider avoiding in the codebase.
I can identify a few in both DSpace and how we use/customize it...
[click] Hardcoding the Initialization of Configuration Hardcoding Database Access into Data Model.
[click] God Objects, ConfigurationManager (config, but also templates and logging) Context (db, events, cache, authen) DSpaceObject (data model and storage)
[click] JAR Hell, users have to alter source and override implementation to change behavior Continuing practice with Maven and Overlays... leads to “jar hell” and “classloader hell” where users change behavior by readding same class with changes in front of original ont he classpath.
Not in criticism of all the hard work that went into DSpace initially. But an analysis of the codebase given best practices that evolved since its creation.
So, Anti-patterns represent possibly bad practices we might wnat to consider avoiding in the codebase.
I can identify a few in both DSpace and how we use/customize it...
[click] Hardcoding the Initialization of Configuration Hardcoding Database Access into Data Model.
[click] God Objects, ConfigurationManager (config, but also templates and logging) Context (db, events, cache, authen) DSpaceObject (data model and storage)
[click] JAR Hell, users have to alter source and override implementation to change behavior Continuing practice with Maven and Overlays... leads to “jar hell” and “classloader hell” where users change behavior by readding same class with changes in front of original ont he classpath.
Not in criticism of all the hard work that went into DSpace initially. But an analysis of the codebase given best practices that evolved since its creation.
So, Anti-patterns represent possibly bad practices we might wnat to consider avoiding in the codebase.
I can identify a few in both DSpace and how we use/customize it...
[click] Hardcoding the Initialization of Configuration Hardcoding Database Access into Data Model.
[click] God Objects, ConfigurationManager (config, but also templates and logging) Context (db, events, cache, authen) DSpaceObject (data model and storage)
[click] JAR Hell, users have to alter source and override implementation to change behavior Continuing practice with Maven and Overlays... leads to “jar hell” and “classloader hell” where users change behavior by readding same class with changes in front of original ont he classpath.
Not in criticism of all the hard work that went into DSpace initially. But an analysis of the codebase given best practices that evolved since its creation.
[click] Many clients with similar need for customization.
[click] All Products dependent on DSpace, in some cases we have some cases, we too overlay existing dspace-api classes and are subject to these problems during upgrading.
[click] But, as a commercial company, we NEED to guarantee upgrade path for our clients.
[click] Thus we really need stable and clearly defined API in DSpace. Both for reliability and for our own customiations.
[click] Many clients with similar need for customization.
[click] All Products dependent on DSpace, in some cases we have some cases, we too overlay existing dspace-api classes and are subject to these problems during upgrading.
[click] But, as a commercial company, we NEED to guarantee upgrade path for our clients.
[click] Thus we really need stable and clearly defined API in DSpace. Both for reliability and for our own customiations.
[click] Many clients with similar need for customization.
[click] All Products dependent on DSpace, in some cases we have some cases, we too overlay existing dspace-api classes and are subject to these problems during upgrading.
[click] But, as a commercial company, we NEED to guarantee upgrade path for our clients.
[click] Thus we really need stable and clearly defined API in DSpace. Both for reliability and for our own customiations.
[click] Many clients with similar need for customization.
[click] All Products dependent on DSpace, in some cases we have some cases, we too overlay existing dspace-api classes and are subject to these problems during upgrading.
[click] But, as a commercial company, we NEED to guarantee upgrade path for our clients.
[click] Thus we really need stable and clearly defined API in DSpace. Both for reliability and for our own customiations.
But what is modularity... separating the code into smaller and more specific projects.
We work to separate the libraries into Functionally separate units.
Use dependency relationships to clarify inter-dependencies.
But... Have to be cautious that we don’t increase complexity that we are seeking to reduce.
We do need modularity. But, modularity is not a complete solution on its own. We need something else...
We need to eliminate dependencies where ever we can and focus on implementations not being tightly bound to other implementations.
[click] and most importantly, we need to identify key important interfaces as contracts within the application.
API are contracts, implementation are separate. What to change something, write your own service or reimplement an existing service.Everything is replaceable. Everything is packaged separately.
Removes Hardcode: Data Models are anemic, Services implemented separate from interfaces used by applications.
Lessens JAR Hell: API contracts, default implementations off limits.Want to change behavior, write changes separately.
Removes God Objects: Services separate functional areas, separate Data Models without interdependency assure separation.
API are contracts, implementation are separate. What to change something, write your own service or reimplement an existing service.Everything is replaceable. Everything is packaged separately.
Removes Hardcode: Data Models are anemic, Services implemented separate from interfaces used by applications.
Lessens JAR Hell: API contracts, default implementations off limits.Want to change behavior, write changes separately.
Removes God Objects: Services separate functional areas, separate Data Models without interdependency assure separation.
API are contracts, implementation are separate. What to change something, write your own service or reimplement an existing service.Everything is replaceable. Everything is packaged separately.
Removes Hardcode: Data Models are anemic, Services implemented separate from interfaces used by applications.
Lessens JAR Hell: API contracts, default implementations off limits.Want to change behavior, write changes separately.
Removes God Objects: Services separate functional areas, separate Data Models without interdependency assure separation.
Not actually part of the DSPace 1.6.0 trunk More like a 3rd party dependency Released separately under its own revision path Dividied into 3 projects API ....
ConfigurationService: Configuration properties for the DSpace system
Request Service: Transactional window for completing tasks. Provides hooks for Request Interceptors to allow other services to participate in the transaction.
Session Service: Provides a state across requests.
Caching Service: Holds onto references to objects for later use.
Point out that :
Contains configuration User defined Asynchronous DSpace Services.
exists in both XMLUI and JSPUI
Note that more files can be added here,
Which is a more appropriate use of overlays that replacing files wholesale.
Spring XML configuration example
Same analogy in Java
But there are other ways as well. Google Guice, annotations, etc.
So our first reall example of service usage in DSpace 1.6
An example of firing events..
Request is transactional window User can get it from Request service Not passed around Retained in Thread Local Database Connection no longer transactional window (not present in 1.6) Code must register Request-Interceptors to close database transactions.
Inversion of Control will free the User/Developer to be able to choose the appropriate configuration and modules used by DSpace. However, we may want to port whatever we can in DSpace 1.x to retain a continuum of the intelligent work and bug fixes that have gone into 1.x
Rather than Hardcoded Initialization forcing DSpace Users to accept DSpace existing Configuration. Or altering that implmentation directly.
Decide what implementations should be isolated from API directly used by users customizing DSpace.
Inversion of Control will free the User/Developer to be able to choose the appropriate configuration and modules used by DSpace. However, we may want to port whatever we can in DSpace 1.x to retain a continuum of the intelligent work and bug fixes that have gone into 1.x
Rather than Hardcoded Initialization forcing DSpace Users to accept DSpace existing Configuration. Or altering that implmentation directly.
Decide what implementations should be isolated from API directly used by users customizing DSpace.
Inversion of Control will free the User/Developer to be able to choose the appropriate configuration and modules used by DSpace. However, we may want to port whatever we can in DSpace 1.x to retain a continuum of the intelligent work and bug fixes that have gone into 1.x
Rather than Hardcoded Initialization forcing DSpace Users to accept DSpace existing Configuration. Or altering that implmentation directly.
Decide what implementations should be isolated from API directly used by users customizing DSpace.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
[click] DSpace 2.0 is a successful, but immense and multifaceted project.
[click] Will take multiple version releases introducing new functionality as we go
[click] Work is incremental, projects need to remain tractable and adopted quickly
[click] Work needs to be kept close to the trunk where it can be integrated.
[click] Services are here as the first step towards that integration.
[click] All Developers need to plan together to for these migrations
My ultimate hope is to emphasize to the community the immense need for you to come forward and contribute within the DSpace developers group, not only working on 2.0, but also on the neccessary effort that will be needed to bring these needed improvements to mainstream usage within the DSpace community.
In doing this integration of dspace-services into DSpace 1.6.0 my interest is in assuring the preservation of this effort by spearheading its adoption onto the community.
We really need to thank Aaron Zeckoski for providing us with dspace-services.
We would not have this solution if it were not for his initiative and the contribution of his time by CARET and funding by JISC.
We really need to thank Aaron Zeckoski for providing us with dspace-services.
We would not have this solution if it were not for his initiative and the contribution of his time by CARET and JISC.
BACK-PORTING DSPACE 2.0:
DSPACE SERVICES
Mark Diggory DSUG 2009
Conflicts in DSpace.
Developers vs. Developers?
But is it Their Fault?
Pioneer Argonne computer scientist Jean F. Hall.
But is it Their Fault?
No, Developers:
Pioneer Argonne computer scientist Jean F. Hall.
But is it Their Fault?
No, Developers:
Need to Innovate
Pioneer Argonne computer scientist Jean F. Hall.
But is it Their Fault?
No, Developers:
Need to Innovate
Need to change code
Pioneer Argonne computer scientist Jean F. Hall.
But is it Their Fault?
No, Developers:
Need to Innovate
Need to change code
Need to solve immediate
issues formost.
Pioneer Argonne computer scientist Jean F. Hall.
Static code is not extensible...
public class StaticManager {
public static Object getSomething(Object object) {
SomeOtherManager.doSomethingElse(...);
}
}
Consider Anti-Patterns
Consider Anti-Patterns
Hardcoding:
Configuration is hardcoded into static “Managers”
Database CRUD is hardcoded into DpaceObjects.”
Consider Anti-Patterns
Hardcoding:
Configuration is hardcoded into static “Managers”
Database CRUD is hardcoded into DpaceObjects.”
God Object:
ConfigurationManager, Context, DSpaceObject
Concentrate too much functionality in a class
Consider Anti-Patterns
Hardcoding:
Configuration is hardcoded into static “Managers”
Database CRUD is hardcoded into DpaceObjects.”
God Object:
ConfigurationManager, Context, DSpaceObject
Concentrate too much functionality in a class
JAR Hell:
Users resort to classpath ordering to overload core API.
User override classes directly to change behavior.
Importance to @mire?
Importance to @mire?
Many clients with similar need for
customization.
Importance to @mire?
Many clients with similar need for
customization.
All Products dependent on DSpace.
Importance to @mire?
Many clients with similar need for
customization.
All Products dependent on DSpace.
We have to guaruntee upgrade path.
Importance to @mire?
Many clients with similar need for
customization.
All Products dependent on DSpace.
We have to guaruntee upgrade path.
Need stability and modularity in DSpace.
Services: Can Help
Removes Hardcode:
Data Models are anemic, Services implemented
separate from interfaces used by applications.
Services: Can Help
Removes Hardcode:
Data Models are anemic, Services implemented
separate from interfaces used by applications.
Lessens JAR Hell:
API contracts, default implementations off limits.
Want to change behavior, write changes separately.
Services: Can Help
Removes Hardcode:
Data Models are anemic, Services implemented
separate from interfaces used by applications.
Lessens JAR Hell:
API contracts, default implementations off limits.
Want to change behavior, write changes separately.
Removes God Objects:
Services separate functional areas, separate Data
Models without interdependency assure separation.
Services: Architecture:
services-util services-api services-impl
/* Instantiate the Utility Class */
DSpace dspace = new DSpace();
/* Access get the Service Manager by convenience method */
ServiceManager manager = dspace.getServiceManager();
/* Or access by convenience method for default services */
EventService service = dspace.getEventService();
<<Interface>>
DSpace
ServiceManager
<<Interface>>
EventService
Services: Default Services
dspace-xmlui-webapp services-util
Spring Application Context
DSpace
services-api
registerService <<Interface>>
Your Own Services
ServiceManager
Your Own Event registerEventListener <<Interface>>
Listeners EventService
addConfiguration
<<Interface>>
Your Own Configs
ConfigurationService
addInterceptor
Your Own Request <<Interface>>
Interceptors RequestService
<<Interface>>
SessionService
Spring:
Java Analogy
DSpace dspace = new DSpace();
EventService service = dspace.getEventService();
MyEventListener listener = new MyEventListener();
service.registerEventListener(listener);
DSpace 1.6 Statistics
Our first example of service usage
DSpace XMLUI Webapp
Cocoon
UsageEvent EventService SolrUsage
LoggingActor EventListener
SolrLogger DSpace
Solr
Webapp
fireEvent receiveEvent post HTTP
Solr
HTTP LogUsage
Request EventListener
receiveEvent Writes to log dspace
.log
Services: Firing Events
DSpace dspace = new DSpace();
Event event = new UsageEvent(
UsageEvent.Action.VIEW,
request,
context,
object);
dspace.getEventService().fireEvent(event);
Proposed Next Steps:
Integrate remaining Services
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
UserService: Auth and Permissions
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
UserService: Auth and Permissions
StorageService: ContentStorage
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
UserService: Auth and Permissions
StorageService: ContentStorage
MetaRegistryService: Content Models,
Metadata Schema, DCMI Application Profiles.
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
UserService: Auth and Permissions
StorageService: ContentStorage
MetaRegistryService: Content Models,
Metadata Schema, DCMI Application Profiles.
SearchService: Unified search and browse
Proposed Next Steps:
Integrate remaining Services
DSpaceDataSource: DB Connection Pool
UserService: Auth and Permissions
StorageService: ContentStorage
MetaRegistryService: Content Models,
Metadata Schema, DCMI Application Profiles.
SearchService: Unified search and browse
MappingService: External Identifier Mapping to
DSpace objects.
Proposed Next Steps:
Replacement of Legacy Managers
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
ConfigurationManager <-------------- ConfigurationService
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
ConfigurationManager <-------------- ConfigurationService
DatabaseManager <---------- DSpaceDataSource Service
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
ConfigurationManager <-------------- ConfigurationService
DatabaseManager <---------- DSpaceDataSource Service
EPerson/ResourceBundle <-------------------- UserService
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
ConfigurationManager <-------------- ConfigurationService
DatabaseManager <---------- DSpaceDataSource Service
EPerson/ResourceBundle <-------------------- UserService
DSO and BitstreamStorage <------------- StorageService
Proposed Next Steps:
Replacement of Legacy Managers
EventManager <-------------------------------- EventService
ConfigurationManager <-------------- ConfigurationService
DatabaseManager <---------- DSpaceDataSource Service
EPerson/ResourceBundle <-------------------- UserService
DSO and BitstreamStorage <------------- StorageService
Search and Configurable Browse <--------- SearchSevice
Proposed Next Steps:
Remove God Objects
DSpace
Database
Manager
Database
Pool
Context
DB
cac
ID
he
Authentication
Manager
Proposed Next Steps:
Remove God Objects
DSpace
Database
Manager
Database Context is composite object
Pool
Context
DB
cac
ID
he
Authentication
Manager
Proposed Next Steps:
Remove God Objects
DSpace
Database
Manager
Database Context is composite object
Pool
Gets passed around everywhere
Context
DB
cac
ID
he
Authentication
Manager
Proposed Next Steps:
Remove God Objects
DSpace
Database
Manager
Database Context is composite object
Pool
Gets passed around everywhere
Context
Represents:
DB
cac
ID
he
Authentication
Manager
Proposed Next Steps:
Remove God Objects
DSpace
Database
Manager
Database Context is composite object
Pool
Gets passed around everywhere
Context
Represents:
DB
cac
State, Identity, Transaction
ID
he
Authentication
Manager
Proposed Next Steps:
Kernel as “Context Container”
DSpace Kernel Service
Manager
Session
Session
Service
Request
Request
Service
Your Cache
Code Service
User
Service
Data
Source
Service
Proposed Next Steps:
Liberate the Implementation
Proposed Next Steps:
Liberate the Implementation
Remove Static Accessors allowing for proper API
contracts and usage of Extensibility.
Proposed Next Steps:
Liberate the Implementation
Remove Static Accessors allowing for proper API
contracts and usage of Extensibility.
Decouple Initialization of “StaticManagers” as
Services into either core Spring, Guice or Application
startup.
Proposed Next Steps:
Liberate the Implementation
Remove Static Accessors allowing for proper API
contracts and usage of Extensibility.
Decouple Initialization of “StaticManagers” as
Services into either core Spring, Guice or Application
startup.
Enforce contracts and backward compatability
as a community practice to assure reliable API +
Services.
In Summary
In Summary
DSpace 2.0 is successful project to date.
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
Work is incremental, projects need to be tractable.
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
Work is incremental, projects need to be tractable.
Work needs to be kept close to the trunk
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
Work is incremental, projects need to be tractable.
Work needs to be kept close to the trunk
DSpace Services are here as the first step.
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
Work is incremental, projects need to be tractable.
Work needs to be kept close to the trunk
DSpace Services are here as the first step.
Faster when we all collaborate in migration activities.
In Summary
DSpace 2.0 is successful project to date.
Yet, will take multiple releases to integrate.
Work is incremental, projects need to be tractable.
Work needs to be kept close to the trunk
DSpace Services are here as the first step.
Faster when we all collaborate in migration activities.
Could always use a little more “$upport”
Special Thanks:
Ben Bosman Graham Triggs
Art Lowel Kevin Van de velde
Bradley McLean Aaron Zeckoski
Supporting Organizations:
Mark Diggory
mdiggory@atmire.com
BACK-PORTING DSPACE 2.0:
DSPACE SERVICES
Mark Diggory DSUG 2009
The release of "DSpace Services" represents the fir more
The release of "DSpace Services" represents the first offering of the DSpace 2.0 technology developed with funding from JISC, and staffing support from the DSpace Foundation, CARET, MIT, @mire, Open Repository, and HP.
This is a presentation on what DSpace services are and how they integrate into DSpace 1.6.0
Thanks to the efforts on this initiative by Mark Diggory from @mire, Aaron Zeckoski from CARET, Ben Bosman from @mire, Graham Triggs from Open Repository and Bradley McLean from DuraSpace Foundation. top less
0 comments
Post a comment