This e-book is an accompaniment to the book "Creative Workshop: 80 Challenges to Sharpen Your Design Skills," more details here: http://www.davidsherwin.com/creative
"Creative Workshop" contains 80 creative challenges that will help any designer reach a breadth of stronger design solutions, in various media, within any set time period. Exercises range from creating a typeface in an hour, to designing a paper robot in an afternoon, to designing web pages and other interactive experiences. Each exercise includes compelling visual solutions from other designers and background stories to help designers increase their capacity to innovate.
Before the book, however, there was a quarter-long class where design students had to complete 80 projects in just 11 weeks. This Teacher's Guide describes the pedagogical methods behind the book, how to create your own Creative Workshop class or workshop series, as well as how to utilize challenges from the book most effectively in a classroom setting. This text is intended for teachers of design and creative thinking, but it may also be helpful for designers and creative managers.
This e-book is an accompaniment to the book "Creative Workshop: 80 Challenges to Sharpen Your Design Skills," more details here: http://www.davidsherwin.com/creative
"Creative Workshop" contains 80 creative challenges that will help any designer reach a breadth of stronger design solutions, in various media, within any set time period. Exercises range from creating a typeface in an hour, to designing a paper robot in an afternoon, to designing web pages and other interactive experiences. Each exercise includes compelling visual solutions from other designers and background stories to help designers increase their capacity to innovate.
Before the book, however, there was a quarter-long class where design students had to complete 80 projects in just 11 weeks. This Teacher's Guide describes the pedagogical methods behind the book, how to create your own Creative Workshop class or workshop series, as well as how to utilize challenges from the book most effectively in a classroom setting. This text is intended for teachers of design and creative thinking, but it may also be helpful for designers and creative managers.
This is the case study conducted on Japan largest online retail chain store Rakuten Ichiba, and also its latest expansion to Rakuten Taiwan Joint Venture with President Chain Store, the largest chain store owner in Taiwan that own 7-Eleven.
International Trade Compliance Strategy Responsibility MatrixGHY International
A quick reference tool that supports the white paper, The Case for an Integrated Trade Compliance Strategy. It shows a road map of relationships, owners, and tasks that are intertwined when an organization is active in international trade. This road map can assist an organization to benchmark their current practice versus that proposed with an Integrated Trade Strategy.
The Content Creation Workflow of the Ship Simulator Game - A Case StudyWolfgang Hürst
Note: You can DOWNLOAD these slides from the FOCUS K3D portal at
http://www.focusk3d.eu/gaming-and-simulation-awg -------------------------------------------------------------------
CASA workshop 3AMIGAS
(supported by FOCUS K3D and GATE)
Keynote presentation no. 2:
Pjotr Van Schothorst, VSTEP B.V.,
Rotterdam, The Netherlands
http://www.cs.uu.nl/events/3amigas/
http://www.focusk3d.eu/
http://gate.gameresearch.nl
This is the case study conducted on Japan largest online retail chain store Rakuten Ichiba, and also its latest expansion to Rakuten Taiwan Joint Venture with President Chain Store, the largest chain store owner in Taiwan that own 7-Eleven.
International Trade Compliance Strategy Responsibility MatrixGHY International
A quick reference tool that supports the white paper, The Case for an Integrated Trade Compliance Strategy. It shows a road map of relationships, owners, and tasks that are intertwined when an organization is active in international trade. This road map can assist an organization to benchmark their current practice versus that proposed with an Integrated Trade Strategy.
The Content Creation Workflow of the Ship Simulator Game - A Case StudyWolfgang Hürst
Note: You can DOWNLOAD these slides from the FOCUS K3D portal at
http://www.focusk3d.eu/gaming-and-simulation-awg -------------------------------------------------------------------
CASA workshop 3AMIGAS
(supported by FOCUS K3D and GATE)
Keynote presentation no. 2:
Pjotr Van Schothorst, VSTEP B.V.,
Rotterdam, The Netherlands
http://www.cs.uu.nl/events/3amigas/
http://www.focusk3d.eu/
http://gate.gameresearch.nl
In my presentation, I will summarize the applied and practical aspects of creating sustainable software products. What does it mean - "green" software for users and developers? I want to explain how creating “green” software can be driven by multiple organizational layers. And how building “green” software products can help the organization increase overall software product efficiency.
This presentation introduces the OWASP Top 10:2021.
It explains how to look at the data related to OWASP Top 10:2021, and provides detailed explanations of items with distinctive data. It also introduces the OWASP Project related to each item.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Elevating Tactical DDD Patterns Through Object Calisthenics
Cassandra conference
1. Issues and Tips for Big Data
on Cassandra
Shotaro Kamio
Architecture and Core Technology dept., DU, Rakuten, Inc. 1
2. Contents
1 Big Data Problem in Rakuten
2 Contributions to Cassandra Project
3 System Architecture
4 Details and Tips
5 Conclusion
2
3. Contents
1 Big Data Problem in Rakuten
2 Contributions to Cassandra Project
3 System Architecture
4 Details and Tips
5 Conclusion
3
4.
Total size
M
on
th
-Y
Ju ear
n
De -9
c 7
Ju -97
n
De -9
c- 8
Ju 98
n
De -99
c
Ju -99
n
Ja -00
n
Ju -00
n
De -01
c
Ju -01
n
De -0
c 2
Ju -02
n
De -0
More than 1 billion records.
c- 3
Ju 03
n
De -0
c 4
– Double its size every second year.
Ju -04
n
De -05
User data increases exponentially.
c
Ju -05
n
De -06
c
Ju -06
n
De -07
c
Ju -07
n
De -0
Big Data Problem in Rakuten
c- 8
Ju 08
n
De -0
c 9
Ju -09
2 years
n
De -1
c- 0
We need a scalable solution to handle this big data.
x2
10
4
5. Importance of Data Store in Rakuten
• Rakuten have a lot of data
– User data, item data, reviews, etc.
• Expect connectivity to Hadoop
• High-performance, fault-tolerant, scalable
storage is necessary → Cassandra
Service A Service B Service C …
Data A Data B
5
6. Performance of New System (Cassandra)
Store all data in 1 day
– Achieved 15,000 updates/sec with quorum.
– 50 times faster than DB.
15,000 updates/sec
Good read throughput
– Handle more than 100 read threads at a
time.
x 50
DB New
6
7. Contents
1 Big Data Problem in Rakuten
2 Contributions to Cassandra Project
3 System Architecture
4 Details and Tips
5 Conclusion
7
8. Contributions to Cassandra Project
• Tested 0.7.x - 0.8.x
• Bug reports / Feedback to JIRA
– CASSANDRA-2212, 2297, 2406, 2557, 2626 and more
– Bugs related to specific condition, secondary index and large
dataset.
• Contribute patches
– Talk this in later slides.
8
9. JIRA: Overflow in bytesPastMark(..)
• https://issues.apache.org/jira/browse/CASSANDRA-2297
• Hit the error on a row which is more than 60GB
– The row has column families of super column type
• bytesPastMark method was fixed to return long value.
9
10. JIRA: Stack overflow while compacting
• https://issues.apache.org/jira/browse/CASSANDRA-2626
• Long series of compaction causes stack overflow.
← This occurs with large dataset.
• Helped debugging.
10
11. Challenges in OSS
• Not well tested with real big data.
→ Rakuten can feedback a lot to community.
– Bug report, patches, and communication.
• OSS becomes much stable.
Feedback
11
12. Contribution of Patches
• Column name aliasing
– Encode column name in compact way.
– Useful to reduce data size for structured (relational)
data.
– Reduce SSTable size by 15%.
• Variable-length quantity (VLQ) compression
– Reduce encoding overhead in columns
– Reduce SSTable size by 17%.
12
13. VLQ Compression Patch
• Serializer is changed to use VLQ encoding.
• Typical column has fixed length of:
– 2 bytes for column name length
– 1 byte for flag
– 8 bytes for TTL, deletion time
– 8 bytes for timestamp
– 4 bytes for length of value.
• Those encoding overheads are reduced.
13
14. Contents
1 Big Data Problem in Rakuten
2 Contributions to Cassandra Project
3 System Architecture
4 Details and Tips
5 Conclusion
14
15. System Architecture
DB
…
DB
Cassandra 1
B atch
Data
feeder
DB Services
B atch
…
DB
…
DB
Cassandra 2
Backup
15
16. System Architecture
DB
…
DB
Cassandra 1
B atch
Data
feeder
DB Services
B atch
…
DB
…
DB
Cassandra 2
Backup
16
17. Planning: Schema Design
• Data modeling is a key of scalability.
• Design schema
– Query patterns for super column and normal column.
• Think queries based on use cases.
– Batch operation to reduce number of requests because Thrift has
communication overhead.
• Secondary Index
– We used it to find out updated data.
• Choose partitioner appropriately.
– One partitioner for a cluster.
17
18. Secondary Index
• Pros
– Useful to query based on a column value.
– It can reduce consistency problem.
– For example, to query updated data based on update-time.
• Cons
– Performance of complex query depends on data.
E.g., Year == 2011 and Price < 100
18
19. A Bit Detail of Secondary Index
Works like a hash + filters.
1. Pick up a row which has a key for the index (hash).
2. Apply filters.
– Collect the result if all filters are matched.
1. Repeat until the requested number of rows are obtained.
E.g., Year == 2011 and Price < 100
Key1 Year = 2011
Key2 Year = 2011 Price = 1,000
Many keys of year = 2011,
Key3 Year = 2011 Price = 10 but a few results.
Key4 Year = 2011 Price = 10,000
Key5 Year = 2011 Price = 200
19
20. A Bit Detail of Secondary Index (2)
Consider the frequency of results for the query
– Very few result in large data set → query might get
timeout.
Careful data/query design is necessary at this moment.
Improvement is discussed: CASSANDRA-2915
20
21. Planning: Data Size Estimation
• Estimate future data volume
• Serialization overhead: x 3 - 4
– Big overhead for small data.
– We improved with custom patches, compression code
• Cassandra 1.0 can use Snappy/Deflate compression.
• Replication: x 3 (depends on your decision)
• Compaction: x 2 or above
21
22. Other Factors for Data Size
• Obsolete SSTables
– Disk usage may keep high after compaction.
– Cassandra 0.8.x relies on GC to remove obsolete SSTables.
– Improved in 1.0.
• How to balance data distribution
– Disk usage can be unbalanced (ByteOrderedPartitioner).
– Partitioning, key design, initial token assignment.
– Very helpful if you know data in advance.
• Backup scheme affects disk space
– Need backup space.
– Discuss later.
22
23. Configuration
• We adopted Cassandra 0.8.x + custom patches.
• Without mmap
– No noticeable difference on performance
– Easier to monitor and debug memory usage and GC related
issues
• ulimit
– Avoid file descriptor shortage. Need more than number of db
files. Bug??
– “memlock unlimited” for JNA
– Make /etc/security/limits.d/cassandra.conf (Redhat)
23
24. JVM / GC
• Have to avoid Full GC anytime.
• JVM cannot utilize large heap over 15G.
– Slow GC. Can be unstable.
– Don’t give too much data/cache into heap.
– Off-heap cache is available in 0.8.1
• Cassandra may use more memory than heap size.
– ulimit –d 25000000 (max data segment size)
– ulimit –v 75000000 (max virtual memory size)
• Need benchmark to know appropriate parameters.
24
25. Parameter Tuning for Failure Detector
• Cassandra uses Phi Accrual Failure Detector
– The Φ Accrual Failure Detector [SRDS'04]
double phi(long tnow)
• Failure detection error occurs {
when node is having too much int size = arrivalIntervals_.size();
double log = 0d;
access and/or GC running if ( size > 0 )
{
double t = tnow - tLast_;
• Depends on number of nodes: double probability = p(t);
log = (-1) * Math.log10( probability );
– Larger cluster, larger number. }
return log;
}
double p(double t)
{
double mean = mean();
double exponent = (-1)*(t)/mean;
return Math.pow(Math.E, exponent);
}
25
26. Hardware
• Benchmark is important to decide hardware.
– Requirements for performance, data size, etc.
– Cassandra is good at utilizing CPU cores.
• Network ports will be bottleneck to scale-out…
– Large number of low-spec servers or
– Small number of high-spec servers.
Our case:
• High-spec CPU and SSD drives
• 2 clusters (active and test cluster)
26
27. System Architecture
DB
…
DB
Cassandra 1
B atch
Data
feeder
DB Services
B atch
…
DB
…
DB
Cassandra 2
Backup
27
28. Customize Hector Library
• Query can timeout on Cassandra:
– When Cassandra is in high load temporarily.
– Request of large result set
– Timeout of secondary index query
• Hector retries forever when query get timed-out.
• Client cannot detect infinite loop.
• Customize:
– 3 Timeouts to return exception to client.
28
29. System Architecture
DB
…
DB
Cassandra 1
B atch
Data
feeder
DB Services
B atch
…
DB
…
DB
Cassandra 2
Backup
29
30. Testing: Data Consistency Check Tool
• We wanted to make sure data is not corrupted within
Cassandra.
• Made a tool to check the data consistency.
Input data
- Insert (Periodically
- Update comes in)
- Delete Process A
Insert, update, and
delete data
Another
Process B Cassandra
database
Compare data with that
in Cassandra
30
31. Testing: Data Consistency Check Tool (2)
Compare only keys of data, not contents.
Useful to diagnose which part is wrong in test phase.
We found out other team’s bug as well
31
32. Repair
• Some types of query doesn’t trigger read repair.
• Nodetool repair is tricky on big data.
– Disk usage
– Time consuming
→ Read all data afterward: Read repair
• Discussion for improvement is going on:
– CASSANDRA-2699
32
33. System Architecture
DB
…
DB
Cassandra 1
B atch
Data
feeder
DB Services
B atch
…
DB
…
DB
Cassandra 2
Backup
33
34. Backup Scheme
Backup might be required to shorten recovery time.
1. Snapshot to local disk
– Plan disk size at server estimation phase.
1. Full backup of input data
– We had full data feed several times for various reasons:
E.g., Logic change, schema change, data corruption, etc.
DB
Incoming
…
DB
data Cassandra
Backup
Snapshot
Snapshot
Snapshot
34
35. Contents
1 Big Data Problem in Rakuten
2 Contributions to Cassandra Project
3 System Architecture
4 Details and Tips
5 Conclusion
35
38. We are hiring! 中途採用を大募集しております!
楽天のMission
人と社会を(ネットを通じて)Empowermentし
自らの成功を通じ社会を変革し豊かにする
楽天のGOAL
To become No.1
Internet Service Company
in the World
楽天のMission&GOALに共感いただける方は是非ご連絡ください!
tech-career@mail.rakuten.com
38