Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Memory

API

Java	
  -­‐	
  Advanced	
  Memory	
  Management	
  
Chris6an	
  Campo	
  -­‐	
  EclipseCon	
  Europe	
  2013	...
Agenda	
  
• 	
  Overview	
  

• 	
  Challenges	
  
• 	
  Cache	
  Problems	
  
• 	
  API	
  

Copyright © 2013 compeople ...
Overview	
  
Private filesystem
Hierarchical datastructure
• root, folders, files, attributes, binary content

Simultaneou...
Data	
  model	
  
"/"

/docs

/docs/a.txt

<root>

"docs"

"images"

"a.txt"
Attributes:
-  lastmodified
-  length
-  owne...
Components	
  
Application

Synchronizer(Daemon)
Store API
Files / Directories
Nodes
SQL Access

Copyright © 2013 compeopl...
Challenges	
  
• 	
  Repe66ve	
  queries	
  (aHributes)	
  
›  Windows	
  assumes	
  any	
  FileSystem	
  opera6on	
  as	
...
Developer	
  stereotypes	
  

Application
Developer
Framework
Developer
DB Developer
• ACID
• transactions
• logic should ...
Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
Cache	
  -­‐	
  Requirements	
  

Rule #1: Cache improves speed but does not change functionality
Rule #2: Cache must not ...
Virtual	
  File	
  System	
  -­‐	
  FileCache	
  

Copyright © 2013 compeople AG, Made available under the Eclipse Public ...
Virtual	
  File	
  System	
  -­‐	
  FileCache	
  

tempting
to add a
cache

often lists
children of a
directory

build
"My...
VFS	
  -­‐	
  "MyCache"	
  V1.0	
  
private WeakHashMap<String, List<GFile> > cache =
new WeakHashMap<String, List<GFile> ...
VFS	
  -­‐	
  "MyCache"	
  V1.0	
  -­‐	
  junit	
  test	
  
// create + write content
for (int x=0; x<100; x++) {
GFile fi...
VFS	
  -­‐	
  "MyCache"	
  V1.0	
  -­‐	
  junit	
  test	
  
// create + write content + check
for (int x=0; x<100; x++) {
...
VFS	
  -­‐	
  MyCache	
  V2.0	
  
private long EXPIRATION_TIME = 200;
private WeakHashMap<String, MyCacheEntry> cache =
ne...
Virtual	
  FS	
  -­‐	
  MyCache	
  V2.0	
  -­‐	
  more	
  problems	
  ...	
  
• 	
  It	
  was	
  hard	
  to	
  find	
  a	
 ...
Virtual	
  FS	
  -­‐	
  LoadingCache	
  (Guava)	
  
private final LoadingCache<String, List<GFile>> cache =
CacheBuilder.n...
Virtual	
  FS	
  -­‐	
  invalida6ng	
  CacheEntries	
  
public GFile create(GFile file) {
GFile g = store.create(file);
ca...
AHribute	
  Cache	
  

Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
AHribute	
  Cache	
  
• 	
  heavy	
  load	
  on	
  the	
  db	
  
• 	
  a	
  lot	
  of	
  read	
  opera6ons	
  on	
  aHribu...
AHribute	
  Cache	
  
private final Map<String, String> attributesCache;
public String getAttribute( final String name ) {...
AHribute	
  Cache	
  -­‐	
  Junit	
  test	
  
// previous value of length = "1057"
db.transaction(new DbRunnable() {
prote...
AHribute	
  Cache	
  -­‐	
  Transac6ons	
  
private final Map<String, Optional<String>> attributesCache;
private void init...
AHribute	
  Cache	
  -­‐	
  Transac6ons	
  
db.transaction(new DbRunnable() {
try {
store.put(file, inputStream);
file.set...
AHribute	
  Cache	
  -­‐	
  Isola6on	
  
Component A

Component B

Transaction 1

Transaction 2

???

x=?
GFile

Copyright...
AHributeCache	
  -­‐	
  Isola6on	
  -­‐	
  Solu6on	
  1	
  
public void setAttribute( final String name, String value ) {
...
AHributeCache	
  -­‐	
  Isola6on	
  -­‐	
  Solu6on	
  2	
  (beHer)	
  
private HashMap<String, String> cache=new HashMap<S...
ChunkCache	
  -­‐	
  data	
  with	
  dirty	
  state	
  

Copyright © 2013 compeople AG, Made available under the Eclipse P...
ChunkCache	
  -­‐	
  data	
  with	
  dirty	
  state	
  
Virtual FileSystem
• 
• 
• 
• 
• 
• 

ChunkFileStore

"write offse...
ChunkCache	
  -­‐	
  data	
  with	
  dirty	
  state	
  
Virtual FileSystem
•  "write offset=110.000 length=200"
•  "write....
ChunkCache	
  -­‐	
  data	
  with	
  dirty	
  state	
  
private final LoadingCache<Integer, Chunk> cache =
CacheBuilder.ne...
Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
API	
  -­‐	
  query	
  -­‐	
  list	
  
query	
  for	
  resources	
  sa6sfying	
  a	
  condi6on	
  
	
  

ArrayList<GFile> ...
API	
  -­‐	
  query	
  -­‐	
  lazy	
  list	
  

List<GFile> query("type='file' and lastmodified>'20130901'");

•  List can...
API	
  -­‐	
  query	
  -­‐	
  iterator	
  

Iterator<GFile> query("type='file' and lastmodified>'20130901'");

•  Iterator...
API	
  -­‐	
  track	
  objects	
  "idea"	
  
• 	
  track	
  "framework"	
  objects	
  that	
  the	
  applica6on	
  holds	
...
API	
  -­‐	
  famous	
  "count"	
  implementa6on	
  

public long notYetSyncFiles() {
localStore.query("type='file' and ch...
API	
  -­‐	
  mapping	
  to	
  String	
  
• 	
  fields	
  that	
  have	
  a	
  limited	
  number	
  of	
  values	
  
›  tru...
Cache	
  /	
  API	
  -­‐	
  Summary	
  

Cache must not
change the
behaviour

Use existing
Cache
implementations

Caches m...
Copyright © 2009 compeople AG, Made available under the Eclipse Public License v 1.0
Upcoming SlideShare
Loading in …5
×

Ece2013 Java Advanced Memorymanagement

856 views

Published on

Talk on Java Memory Management from eclipsecon europe 2013. The talk is about pitfalls when you use Java Caches and API tips for better scaling with growing data

Published in: Technology, Sports
  • Be the first to comment

Ece2013 Java Advanced Memorymanagement

  1. 1. Memory API Java  -­‐  Advanced  Memory  Management   Chris6an  Campo  -­‐  EclipseCon  Europe  2013  
  2. 2. Agenda   •   Overview   •   Challenges   •   Cache  Problems   •   API   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  3. 3. Overview   Private filesystem Hierarchical datastructure • root, folders, files, attributes, binary content Simultaneous read/write from multiple threads • application UI, synchronizer, filesystem driver etc. Stored in sql database Schema is dynamic (columns added at runtime) No OR-Mapper Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  4. 4. Data  model   "/" /docs /docs/a.txt <root> "docs" "images" "a.txt" Attributes: -  lastmodified -  length -  owner -  customernumber Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0 /images
  5. 5. Components   Application Synchronizer(Daemon) Store API Files / Directories Nodes SQL Access Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0 Virtual File System
  6. 6. Challenges   •   Repe66ve  queries  (aHributes)   ›  Windows  assumes  any  FileSystem  opera6on  as  cheap   •   Applica6on  developer  don't  think  about  internals   ›  they  assume  that  any  API  call  is  cheap  and  fast   •   Data  structure  is  dynamic   •   Growing  size  of  data   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  7. 7. Developer  stereotypes   Application Developer Framework Developer DB Developer • ACID • transactions • logic should be a stored procedure • db has buffer (=cache) • clean and small API • refactor often • DBs are too slow à cache Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0 • never refactor :-) • wrap every framework • Framework is too slow à cache • why cant I access the DB directly ? :-)
  8. 8. Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  9. 9. Cache  -­‐  Requirements   Rule #1: Cache improves speed but does not change functionality Rule #2: Cache must not lead to OOM Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  10. 10. Virtual  File  System  -­‐  FileCache   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  11. 11. Virtual  File  System  -­‐  FileCache   tempting to add a cache often lists children of a directory build "MyCache" :-) Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  12. 12. VFS  -­‐  "MyCache"  V1.0   private WeakHashMap<String, List<GFile> > cache = new WeakHashMap<String, List<GFile> >(); private Store store; public synchronized List<GFile> list( final GFile gFile ) { List<GFile> list = cache.get(gFile.getPath()); if ( list == null) { list = store.list(goyaFile ); cache.put(gFile.getPath(), list ); } return list; } Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  13. 13. VFS  -­‐  "MyCache"  V1.0  -­‐  junit  test   // create + write content for (int x=0; x<100; x++) { GFile file = store.create(new GFile("/docs/test" +x+ ".txt")); } // check for (int x=0; x<100; x++) { List<GFile> list = list(new GFile("/docs")); assertTrue(list.contains(new GFile("/docs/test" +x+ ".txt"))); } // list from "MyCache" public synchronized List<GFile> list( final GFile gFile ) {} OK + FASTER with Cache Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  14. 14. VFS  -­‐  "MyCache"  V1.0  -­‐  junit  test   // create + write content + check for (int x=0; x<100; x++) { GFile file = store.create(new GFile("/docs/test“ +x+ ".txt")); List<GFile> list = list(new GFile("/docs")); assertTrue(list.contains(new GFile("/docs/test“ +x+ ".txt"))); } // list from "MyCache" public synchronized List<GFile> list( final Store store, final GFile gFile ) {} FAILS with Cache -> expiration time for cache entries Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  15. 15. VFS  -­‐  MyCache  V2.0   private long EXPIRATION_TIME = 200; private WeakHashMap<String, MyCacheEntry> cache = new WeakHashMap<String, MyCacheEntry>(); public synchronized List<GFile> list( final LocalStore store, final GFile gFile ) { final MyCacheEntry myCacheEntry = cache.get(gFile.getPath()); if ( myCacheEntry == null || System.currentTimeMillis() - { class MyCacheEntry public List<GFile> listGFiles; EXPIRATION_TIME > myCacheEntry.currentTimeMillis ) { public long currentTime; final List<GoyaFile> listGoyaFiles = store.list( goyaFile ); MyCacheEntry( final List<GFile> cache.put(gFile.getPath(), new MyCacheEntry( listGoyaFiles ) ); listGFiles ) { this.listGFiles = listGFiles; return listGoyaFiles; this.currentTime = System.currentTimeMillis(); } } else { } return myCacheEntry.listGoyaFiles; } Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  16. 16. Virtual  FS  -­‐  MyCache  V2.0  -­‐  more  problems  ...   •   It  was  hard  to  find  a  good  value  for  EXPIRATION_TIME   ›  200  ms  effec6ve  but  it  was  easy  to  get  wrong  HITS   ›  10  ms  beFer  (not  always  right)  but  not  many  HITS   •   implement  a  maximum  number  of  entries   •   find  means  to  "know"  when  CacheEntries  are  invalid   •   sta6s6cs  ?   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  17. 17. Virtual  FS  -­‐  LoadingCache  (Guava)   private final LoadingCache<String, List<GFile>> cache = CacheBuilder.newBuilder() .maximumSize( 10 ) .softValues() .expireAfterWrite( 200, TimeUnit.MILLISECONDS) .build( new CacheLoader<String, List<GFile>>() { public List<GFile> load( final String path ) { return store.list( new GFile( path ) ); } } ); Rule #3: Never write your own Cache implementation Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  18. 18. Virtual  FS  -­‐  invalida6ng  CacheEntries   public GFile create(GFile file) { GFile g = store.create(file); cache.flush(); return g; } public GFile create(GFile file) { GFile g = store.create(file); cache.remove(getParent(file).getPath()); return g; } Rule #4: Have a strategy for removing cache entries (expiration timing alone does not work) Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  19. 19. AHribute  Cache   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  20. 20. AHribute  Cache   •   heavy  load  on  the  db   •   a  lot  of  read  opera6ons  on  aHributes   •   not  many  writes   •   à  cache  aHributes   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  21. 21. AHribute  Cache   private final Map<String, String> attributesCache; public String getAttribute( final String name ) { String value = attributesCache.get( name ); if (value!=null) { return value; } ... // read it from DB } public void setAttribute( final String name, String value) { if ( !Objects.equal( value, attributesCache.get( name ) ) ) { attributesCache.put( name, value ); ... // update attribute in the database } } Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  22. 22. AHribute  Cache  -­‐  Junit  test   // previous value of length = "1057" db.transaction(new DbRunnable() { protected void run() { file.setAttribute("length", "0"); long newLength = store.put(file, inputStream); file.setAttribute("length", new String(newLength)) public void setAttribute(final String name,String value) { } if(!Objects.equal(value, attributeCache.get(name))){ }); attributesCache.put( name, value ); ... ... // update attribute in the database } file.setAttribute("length", "0"); •  •  •  •  IOException reading inputStream transaction rolls back attribute length = "0" in Cache and has "1057" in DB setAttribute("length","0") after Transaction, does not reach DB Rule #5: Cache should be aware of transaction rollbacks Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  23. 23. AHribute  Cache  -­‐  Transac6ons   private final Map<String, Optional<String>> attributesCache; private void init() { getTransaction().register(this); } public String getAttribute( final String key ) { ... } public void setAttribute( final String name, String value) {} public void rollback() { // called when transaction rolls back attributesCache.flush(); } Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  24. 24. AHribute  Cache  -­‐  Transac6ons   db.transaction(new DbRunnable() { try { store.put(file, inputStream); file.setAttribute("loaded","true"); } catch (IOException ioe) { file.setAttribute("loaded","false"); throw ioe; } }); •  IOException -> transaction rollback •  loaded = "false" NOT in Cache, NOT in DB •  Cache changes behaviour once it does Transaction rollback. •  Rule #6 Dont return with Exception if data should not rollback Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  25. 25. AHribute  Cache  -­‐  Isola6on   Component A Component B Transaction 1 Transaction 2 ??? x=? GFile Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  26. 26. AHributeCache  -­‐  Isola6on  -­‐  Solu6on  1   public void setAttribute( final String name, String value ) { ... getTransaction().getContext().set( ... + name, value ); ... } public String getAttribute(String name) { ... String value = getTransaction().getContext().get( ... + name ); ... } •  store cached values global IN the transaction context •  however all attribute values are stored in Transaction Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  27. 27. AHributeCache  -­‐  Isola6on  -­‐  Solu6on  2  (beHer)   private HashMap<String, String> cache=new HashMap<String,String>(); public void setAttribute( final String name, String value) { ... cache.put(getTransactionId() + name, value); ... } public String getAttribute(String name) { ... String value = cache.get(getTransactionId() + name); ... } store cached values local depending on transaction-id Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  28. 28. ChunkCache  -­‐  data  with  dirty  state   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  29. 29. ChunkCache  -­‐  data  with  dirty  state   Virtual FileSystem •  •  •  •  •  •  ChunkFileStore "write offset=110.000 length=2000" "write..." "write..." "write..." "write..." "write..." 010110010101 010101010101 010110010101 010101101000 010101010101 010110010101 11110101 010101101000 010101010101 010110010101 11110101 010101101000 010101010101 11110101 010101101000 11110101 • VFS  writes  and  reads  data  in  random  "segments"   ›  i.e.  "write  offset  110.000  length  200"   •   Stored  on  disk  in  1  MB  "Chunks"   Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0 Chunks (1 MB)
  30. 30. ChunkCache  -­‐  data  with  dirty  state   Virtual FileSystem •  "write offset=110.000 length=200" •  "write..." ChunkCache 010110010101 010101010101 010110010101 010101101000 010101010101 11110101 010101101000 11110101 •   Cache  writes  lazy   •   Cache  maintained  in  dirty  state   •   LoadingCache  used   •   6meout,  maxsize   ChunkFileStore 010110010101 010101010101 010110010101 010101101000 010101010101 010110010101 11110101 010101101000 010101010101 010110010101 11110101 010101101000 010101010101 11110101 010101101000 11110101 Chunks (1 MB) Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  31. 31. ChunkCache  -­‐  data  with  dirty  state   private final LoadingCache<Integer, Chunk> cache = CacheBuilder.newBuilder() .maximumSize(4) .expireAfterAccess(2, TimeUnit.SECONDS) .removalListener(new RemovalListener<Integer, Chunk>() { public void onRemoval(final RemovalNotification<Integer, Chunk> notification ) { if ( notification.getValue().isDirty() ) { // write chunk •  write to disk on remove } •  cache entries only expire on access !! } •  no automatic housekeeping }) .build( new CacheLoader<Integer, Chunk>() { public Chunk load( final Integer key ) throws Exception { // load Chunk for key } } ); Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  32. 32. Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  33. 33. API  -­‐  query  -­‐  list   query  for  resources  sa6sfying  a  condi6on     ArrayList<GFile> query("type='file' and lastmodified>'20130901' "); •  •  •  •  can produce a lot of instances returns only upon complete resultset returns all result instances memory footprint unpredicable (maybe large) Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  34. 34. API  -­‐  query  -­‐  lazy  list   List<GFile> query("type='file' and lastmodified>'20130901'"); •  List can be lazy and custom implementation •  that only stores primkeys •  builds instance on demand •  returns only upon complete resultset •  less memory footprint Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  35. 35. API  -­‐  query  -­‐  iterator   Iterator<GFile> query("type='file' and lastmodified>'20130901'"); •  Iterator can be lazy and custom implementation •  and builds instance on demand •  uses DB cursor and reads on demand •  less memory footprint •  result can only be consumed once Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  36. 36. API  -­‐  track  objects  "idea"   •   track  "framework"  objects  that  the  applica6on  holds  on  too   •   for  sta6s6cs   •   for  error  logging   ›  i.e.  "more  than  10.000  file  resources  WARN"   •   Implementa6on   ›  store  all  returned  object  in  a  WeakHashMap   ›  GC  removes  objects  when  there  is  no  hard  reference   ›  WeakHashMap.size()  can  give  you  an  ESTIMATE     Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  37. 37. API  -­‐  famous  "count"  implementa6on   public long notYetSyncFiles() { localStore.query("type='file' and changed='true'").size(); } WASTE public long notYetSyncFiles() { localStore.queryCount("type='file' and changed='true'"); } OK looks trivial, but its a real world example :-) Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  38. 38. API  -­‐  mapping  to  String   •   fields  that  have  a  limited  number  of  values   ›  true  /  false   ›  bitsets  (1,8,128,32)   ›  filetypes  'file'  /  'directory'   •  intern() is expensive •  PermGen stores 'intern' •  small space •  since Java7 in Heap ›  etc.   •   if  you  read  those  a  lot  from  DB  or  remote  services*   ›  you  quickly  have  10.000  or  more  instances  of  'true'   if (value.equals("true") || value.equals("false") || ... { value = value.intern(value); } Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  39. 39. Cache  /  API  -­‐  Summary   Cache must not change the behaviour Use existing Cache implementations Caches must be aware of database behaviour (ACID) API must scale Copyright © 2013 compeople AG, Made available under the Eclipse Public License v 1.0
  40. 40. Copyright © 2009 compeople AG, Made available under the Eclipse Public License v 1.0

×