We presented at the Israeli 3rd Java Technology Day, the largest SUN Microsystems/MySQL event in israel. We presented here the essentials parts of building a real life web/enterprise system that needs to handle the performance needs of 1 billion events per day (a case study from the ad networks billing systems). We presented the adoption rate in the internet, Load Balancers (HAProxy, Apache, Radware, F5, Cisco), Web Servers, In Memory Database (IMDB inc. Memcached, Gigaspaces, Teracotta and Oracle Coherence) and finally Sharding (inc. Veritical, Static Horizontal and dynamic). A great example for a performance boosting architecture.
How your very large databases can work in the cloud computing world?Moshe Kaplan
How your very large databases can work in the cloud computing world?
Presented by Moshe Kaplan from RockeTier, a performance expert and scale out architect at the IGT Cloud Computing work group at April 20, 2009
Cloud computing is famous for its flexibility, dynamic nature and ability to infinite growth. However, infinite growth means very large databases with billions of records in it. This leads us to a paradox: "How can weak servers support very large databases which usually require several CPUs and dedicated hardware?"
The Internet industry proved it can be done. These days many of the Internet giants, processing billions of events every day, are based on cloud computing architecture such and sharding. What is Sharding ? What kinds of Sharding can you implement? What are the best practices?
RockeTier specializes in analyzing and boosting existing software
performance and developing efficient high-load systems
You can contact Mr. Kaplan at moshe.kaplan at rocketier.com, read his blog at http://top-performance.blogspot.com and visit RockeTier site at www.rocketier.com
A presentation I gave in the AlphaGeeks meetup regarding NoSQL and other methods to boost software system performance. We were talking about Sharding, NoSQL, Memcached, In Memory Databases, CouchDB, Hadoop and SimpleDB.
Feel free to contact me regarding the presentation. Moshe Kaplan.
Mastering Oracle® Hyperion EPM Metadata in a distributed organizationOrchestra Networks
In this case study from Kscope14, hear how The United Technologies Corporation, a globally distributed, Fortune 500 company manages their Oracle Hyperion EPM metadata using Orchestra Networks’ cost-effective, DRM alternative: MDM for Oracle Hyperion EPM. The session covered practical issues: data exchange with HFM, governance, workflow. Also discussed, how other alternatives compare and their key differences.
"How to document your decisions", Dmytro Ovcharenko Fwdays
We will perform architecture kata around a proposed business case. We will review ADD in detail. How usually architecture vision document looks like. How to match your architecture drivers and proposed architecture decisions in architecture views. We will review what is ATAM and how to perform analysis of your decisions in the right way. And finally, we will create an architecture vision document from scratch.
Learn About:
Newly added features such as Forefront Endpoint Protection and connectors to Intune and Azure
Support features like IOS and Linux, including extended device management
Architectural layout for design considerations, including the CAS Server and the elimination of Native Mode
New ways to configure and deploy your software updates
System Center integration and automation with other System Center products, such as Service Manager and the Data Warehouse connector
How your very large databases can work in the cloud computing world?Moshe Kaplan
How your very large databases can work in the cloud computing world?
Presented by Moshe Kaplan from RockeTier, a performance expert and scale out architect at the IGT Cloud Computing work group at April 20, 2009
Cloud computing is famous for its flexibility, dynamic nature and ability to infinite growth. However, infinite growth means very large databases with billions of records in it. This leads us to a paradox: "How can weak servers support very large databases which usually require several CPUs and dedicated hardware?"
The Internet industry proved it can be done. These days many of the Internet giants, processing billions of events every day, are based on cloud computing architecture such and sharding. What is Sharding ? What kinds of Sharding can you implement? What are the best practices?
RockeTier specializes in analyzing and boosting existing software
performance and developing efficient high-load systems
You can contact Mr. Kaplan at moshe.kaplan at rocketier.com, read his blog at http://top-performance.blogspot.com and visit RockeTier site at www.rocketier.com
A presentation I gave in the AlphaGeeks meetup regarding NoSQL and other methods to boost software system performance. We were talking about Sharding, NoSQL, Memcached, In Memory Databases, CouchDB, Hadoop and SimpleDB.
Feel free to contact me regarding the presentation. Moshe Kaplan.
Mastering Oracle® Hyperion EPM Metadata in a distributed organizationOrchestra Networks
In this case study from Kscope14, hear how The United Technologies Corporation, a globally distributed, Fortune 500 company manages their Oracle Hyperion EPM metadata using Orchestra Networks’ cost-effective, DRM alternative: MDM for Oracle Hyperion EPM. The session covered practical issues: data exchange with HFM, governance, workflow. Also discussed, how other alternatives compare and their key differences.
"How to document your decisions", Dmytro Ovcharenko Fwdays
We will perform architecture kata around a proposed business case. We will review ADD in detail. How usually architecture vision document looks like. How to match your architecture drivers and proposed architecture decisions in architecture views. We will review what is ATAM and how to perform analysis of your decisions in the right way. And finally, we will create an architecture vision document from scratch.
Learn About:
Newly added features such as Forefront Endpoint Protection and connectors to Intune and Azure
Support features like IOS and Linux, including extended device management
Architectural layout for design considerations, including the CAS Server and the elimination of Native Mode
New ways to configure and deploy your software updates
System Center integration and automation with other System Center products, such as Service Manager and the Data Warehouse connector
In the last past months we at RockeTier were working with several large organizations in three aspects: 1) boosting existing software performance (lean projects); 2) design new systems which are capable process billions of events per day based on commodity hardware and software and 3) establishing processes in large organization that support the life cycle of performance from event management, problem management to establishing a continues performance boosting to the organization systems from RFI to production. This presentation was presented to a large telecommunication industry company. This company is considering implementing a 360 degrees performance boosting project along its main product lines.
Contino Webinar - Migrating your Trading Workloads to the CloudBen Saunders
Benjamin Wootton, Contino Co-founder and CTO with a decade of IB experience, and Ben Saunders, experienced FIS DevOps consultant, will explore how our DevOps framework (Continuum) can help you move to the cloud as quickly and easily as possible.
This webinar covers:
The foundations for migrating trading apps and data to the cloud swiftly and safely
Ensuring compliance with regulatory controls
Architecting and optimizing your trading applications for optimal cloud performance
Integrating tools and processes to streamline app and data migration
Nearly every IT leader these days is wresting with Shadow IT – the dynamic in which end users obtain IT solutions from cloud service providers without informing corporate IT.
How to Migrate Applications Off a MainframeVMware Tanzu
Ah, the mainframe. Peel back many transactional business applications at any enterprise and you’ll find a mainframe application under there. It’s often where the crown jewels of the business’ data and core transactions are processed. The tooling for these applications is dated and new code is infrequent, but moving off is seen as risky. No one. Wants. To. Touch. Mainframes.
But mainframe applications don’t have to be the electric third rail. Modernizing, even pieces of those mainframe workloads into modern frameworks on modern platforms, has huge payoffs. Developers can gain all the productivity benefits of modern tooling. Not to mention the scaling, security, and cost benefits.
So, how do you get started modernizing applications off a mainframe? Join Rohit Kelapure, Consulting Practice Lead at Pivotal, as he shares lessons from projects with enterprises to move workloads off of mainframes. You’ll learn:
● How to decide what to modernize first by looking at business requirements AND the existing codebase
● How to take a test-driven approach to minimize risks in decomposing the mainframe application
● What to use as a replacement or evolution of mainframe schedulers
● How to include COBOL and other mainframe developers in the process to retain institutional knowledge and defuse project detractors
● How to replatform mainframe applications to the cloud leveraging a spectrum of techniques
Presenter : Rohit Kelapure, Consulting Practice Lead, Pivotal
Modernizing Traditional Apps with a Docker is a great way to start your containerization strategy and to prove to your leadership there is value. MetLife has outlined the early stages of their journey at past DockerCons, but have now moved past the POC stage and are in the process of transforming our technology portfolio globally. We are driving change on multiple levels in many ways, and now we've created a long-term strategy outlining how we will reach our technology goals using containers. At MetLife, we realized that once a PoC is complete, incorporating an entire application portfolio can be challenging: Where do you focus for the largest benefit to the organization? How and what do I spend the savings on if I want to ensure maximum value? How should people & processes change to align to match our transformation goals?
In this session you’ll hear more about how MetLife has built their strategy to go forward, and you’ll learn how Docker's strategic approach to containerization and transformation can be applied in your company, too.
Performance Optimization: Incorporating Database and Code Optimzitation Into ...Michael Findling
Embarcadero offers database and code optimization tools for application developers, database developers, and database administrators to prevent, find, and fix problems that impact performance throughout the system development lifecycle.
In the last past months we at RockeTier were working with several large organizations in three aspects: 1) boosting existing software performance (lean projects); 2) design new systems which are capable process billions of events per day based on commodity hardware and software and 3) establishing processes in large organization that support the life cycle of performance from event management, problem management to establishing a continues performance boosting to the organization systems from RFI to production. This presentation was presented to a large telecommunication industry company. This company is considering implementing a 360 degrees performance boosting project along its main product lines.
Contino Webinar - Migrating your Trading Workloads to the CloudBen Saunders
Benjamin Wootton, Contino Co-founder and CTO with a decade of IB experience, and Ben Saunders, experienced FIS DevOps consultant, will explore how our DevOps framework (Continuum) can help you move to the cloud as quickly and easily as possible.
This webinar covers:
The foundations for migrating trading apps and data to the cloud swiftly and safely
Ensuring compliance with regulatory controls
Architecting and optimizing your trading applications for optimal cloud performance
Integrating tools and processes to streamline app and data migration
Nearly every IT leader these days is wresting with Shadow IT – the dynamic in which end users obtain IT solutions from cloud service providers without informing corporate IT.
How to Migrate Applications Off a MainframeVMware Tanzu
Ah, the mainframe. Peel back many transactional business applications at any enterprise and you’ll find a mainframe application under there. It’s often where the crown jewels of the business’ data and core transactions are processed. The tooling for these applications is dated and new code is infrequent, but moving off is seen as risky. No one. Wants. To. Touch. Mainframes.
But mainframe applications don’t have to be the electric third rail. Modernizing, even pieces of those mainframe workloads into modern frameworks on modern platforms, has huge payoffs. Developers can gain all the productivity benefits of modern tooling. Not to mention the scaling, security, and cost benefits.
So, how do you get started modernizing applications off a mainframe? Join Rohit Kelapure, Consulting Practice Lead at Pivotal, as he shares lessons from projects with enterprises to move workloads off of mainframes. You’ll learn:
● How to decide what to modernize first by looking at business requirements AND the existing codebase
● How to take a test-driven approach to minimize risks in decomposing the mainframe application
● What to use as a replacement or evolution of mainframe schedulers
● How to include COBOL and other mainframe developers in the process to retain institutional knowledge and defuse project detractors
● How to replatform mainframe applications to the cloud leveraging a spectrum of techniques
Presenter : Rohit Kelapure, Consulting Practice Lead, Pivotal
Modernizing Traditional Apps with a Docker is a great way to start your containerization strategy and to prove to your leadership there is value. MetLife has outlined the early stages of their journey at past DockerCons, but have now moved past the POC stage and are in the process of transforming our technology portfolio globally. We are driving change on multiple levels in many ways, and now we've created a long-term strategy outlining how we will reach our technology goals using containers. At MetLife, we realized that once a PoC is complete, incorporating an entire application portfolio can be challenging: Where do you focus for the largest benefit to the organization? How and what do I spend the savings on if I want to ensure maximum value? How should people & processes change to align to match our transformation goals?
In this session you’ll hear more about how MetLife has built their strategy to go forward, and you’ll learn how Docker's strategic approach to containerization and transformation can be applied in your company, too.
Performance Optimization: Incorporating Database and Code Optimzitation Into ...Michael Findling
Embarcadero offers database and code optimization tools for application developers, database developers, and database administrators to prevent, find, and fix problems that impact performance throughout the system development lifecycle.
Scaling managed MySQL Platform in Flipkart - (Sachin Japate - Flipkart) - Myd...Mydbops
Mydbops Opensource Database Meetup -13.
Flipkart built and scaled one of the largest managed MySQL platforms, managing over 700+ MySQL clusters, 2500+ VMs, and a 1000TB+ storage footprint. The presentation through the journey of problems that have been solved through the platform, features, and architecture of the product.
This week I had a session w/ one the Israeli largest Telcos, regarding BillRun!, their new billing solution: an open source billing solution that is based on MongoDB. We covered in this 3 days course: 1) NoSQL background, 2) MongoDB introduction and setup; 3) NoSQL Data Model; 4) NoSQL query language and aggregation framework; 5) Performance tuning; 6) Operations: backup, restore, monitoring and security and 7) HA and scale using Replica set and Sharding
Best Practices for Big Data:
- Search using Elasticsearch
- High throughput using Redis
- Document database using MongoDB
- Eventually consistent using Cassandra
- Streaming and Map Reduce using Spark
An introduction to MongoDB: Why should you choose, and how will you migrate to it? What you should consider and What to learn? Syntax, highlights and more
Web Systems Architecture by Moshe KaplanMoshe Kaplan
If you are planning a large scale web site, or consider how to take you site to the next level, you should have participated in this workshop. Feel free to connect for further details.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1. moshe . [email_address] . com http :// top - performance . blogspot . com 1 Billion Events Per Day The Internet Building Blocks Moshe Kaplan, RockeTier The Performance Experts
2. 1 Billion Events Per Day The Internet Building Blocks [email_address] http://top-performance.blogspot.com
18. Step III – In Memory Database UPDATE … SET HIT=HIT+1 UPDATE … SET HIT=HIT+1 Validate Validate Banner 1 Hit Banner 2 Hit
19. Step III – In Memory Database IMDB UPDATE … SET HIT=HIT+1 UPDATE … SET HIT=HIT+1 Validate Validate UPDATE SET HIT=HIT+41 UPDATE SET HIT=HIT+22 UPDATE SET HIT=HIT+87 Banner 1 Hit Banner 2 Hit
33. Startup your Engines Thank you [email_address] http://top-performance.blogspot.com Our Methodology Performance problems are extremely complex and due to the diferent technologies deployed, each case is unique. A “typical” performance problem requires delving into databases, application servers, client technology, code in difering programming languages and system and software architectures. RockeTier implements a unique methodology in order to simplify the problem and evaluate each performance bottleneck, providing both an immediate efective relief and when necessary, design a gradual roadmap to speed up your software system and make it scalable and robust. Our 5 steps methodology : 1. Detect: Pinpoint your performance bottlenecks using various tools including load and stress tools, code profiling, database profiling, network sniffing and code review to detect performance bottlenecks in specific components. 2. Rate: Grade each bottleneck by importance and provide immediate practical recommendations and performance boost estimations. 3. Immediate effective relief: Provide immediate fixes and workarounds in a short time frame helping you meet your urgent business needs. 4. Roadmap Planning: When necessary, redesign next generation Solutions, using proven robust and scalable solutions such as grid and in memory databases. 5. Scale up and Scale out: In cases where redesign is necessary - RockeTier provides implementation or software design description (SDD), and guidance for in house programmers for the implementation of the next generation scalable system, which will meet your growing business needs. Your Value Business: Achieve your business performance requirements. GreenIT: Protect the environment and reduce CO2 emissions. Bottom Line: Reducing hardware and 3rd party software cost. The Performance Experts Success Stories The Finance Sector: An international insurance company managing over 20 Billion US dollars in assets was facing poor performance in its core life insurance policy software system. The RockeTier team detected bottlenecks originating from several software infrastructure modules. A practical solution was implemented. The customer’s success criteria was a 20% decrease in insurance policy creation run time, Our solution provided a 40% decrease in run time! Telecom: A VC backed start-up company was facing critical installation problems in the leading Israeli cellular operator. Knowing that existing system performance would not meet client requirements, the company asked RockeTier to help it boost its performance. RockeTier evaluated the system and implemented a workaround to the system database architecture, boosting the overall system performance by 30%. Following that. the RockeTier team designed the company’s next generation architecture, meeting a throughput of 3000Mbps by design. RockeTier at a Glance RockeTier is a software solutions company, which utilizes its knowledge and skills to help companies from both the enterprise sector and the start-up industry. RockeTier has numerous success stories in solving customers’ system performance bottlenecks and scale out limitations, providing immediate improvements and workarounds in a short time frame and, when necessary, redesign and implementation of the next generation solutions employing grid and/or in- memory databases in the Web 2.0, Telecom and finance markets. Web 2.0: a start-up company providing an innovative electronic advertising and billing system was facing its technological limits. The RockeTier team evaluated and redesigned its system architecture and is currently implementing a scale out grid mechanism and caching algorithms. The solution supports 20 times the original capacity using the same hardware. Moreover it supports semi-linear growth (by simple scale out) and high availability requirements. “ 20% reduction in transaction time within 3 months” “ Boost Performance by a factor of 200” “ 200 million events per day”
Editor's Notes
Initial setup of a u-Page from toolbar 14M users today 20M users 2009Q1 600K downloads per day Moving from 1:N templates to 10-15% users that updates the page (from 1:1000 templates/users ratio to 1:7 templates/users) Statistics: user changes: 1 per week, upload: 2 times a day Wants to use the default if the user did not modified the template. However, wants to support pushed changes from templates to users pages, even if those were changed Have the same situation now, but now it’s saved on the users desktop Current applicative cache: - int: toolbar_id, list of int: widget_id[] - int: widget_id, object: widget - int: user_id, object: user Table design: - User id, tab id and user_tab_id are GUID – meaning that they can be distributed between databases - Other are not (may be needed to support?) - Expected XML scheme size: 45KB (up to 500KB) Why migration to MySQL will be problematic (they are already using the current SQL Server SP features): - Maximal row size * http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html * The maximum row length, except for VARBINARY , VARCHAR , BLOB and TEXT columns, is slightly less than half of a database page. That is, the maximum row length is about 8000 bytes - Scope identity - For XML - Bulk Insert from XML - Rollback/Transaction - BEGIN TRY Applicative changes by the user: - Move - Delete - Add - Minimize - Change internal parameters Options to be considered: - SQL Server based large machine - Sharding based on MySQL, SQL Server - Gigaspaces solution: felt that it’s a large machine, and they prefer the database way Other options - Saving only XML in the current way - Save the summed page configuration in an XML, so little read should be done from the DB (Tab based) - Write 20K files of 40KB each on the my laptop HD: 149s - Read XML: 109.5s - Write to DB: 1789s Use serialization of .Net - Save the XML on the disk in order to avoid variable length fields - Use memcached to hold the hash of users? Things to be considered: - What horizontal sharding algorithm should be selected - Hibernate Shards – provided by Google. Still beta-testing phase - What vertical sharding tables should be spitted to different databases - How do you manage so many databases (distribute data and so on) - There is not really an option to do that - Defining optimal table sizes - Retrieval of data from the disk vs. Getting data from the tables: Tab (1 per displayed page), Zone (3 per displayed page), Widget instance (10-300 per displayed page + should be extracted with/out zones) - OLAP solution to merge data OLAP solution Toolbar design: - Saving
Our Methodology Performance problems are extremely complex and due to the diferent technologies deployed, each case is unique. A “typical” performance problem requires delving into databases, application servers, client technology, code in difering programming languages and system and software architectures. RockeTier implements a unique methodology in order to simplify the problem and evaluate each performance bottleneck, providing both an immediate efective relief and when necessary, design a gradual roadmap to speed up your software system and make it scalable and robust. Our 5 steps methodology : 1. Detect: Pinpoint your performance bottlenecks using various tools including load and stress tools, code profiling, database profiling, network sniffing and code review to detect performance bottlenecks in specific components. 2. Rate: Grade each bottleneck by importance and provide immediate practical recommendations and performance boost estimations. 3. Immediate effective relief: Provide immediate fixes and workarounds in a short time frame helping you meet your urgent business needs. 4. Roadmap Planning: When necessary, redesign next generation Solutions, using proven robust and scalable solutions such as grid and in memory databases. 5. Scale up and Scale out: In cases where redesign is necessary - RockeTier provides implementation or software design description (SDD), and guidance for in house programmers for the implementation of the next generation scalable system, which will meet your growing business needs. Your Value Business: Achieve your business performance requirements. GreenIT: Protect the environment and reduce CO2 emissions. Bottom Line: Reducing hardware and 3rd party software cost. The Performance Experts Success Stories The Finance Sector: An international insurance company managing over 20 Billion US dollars in assets was facing poor performance in its core life insurance policy software system. The RockeTier team detected bottlenecks originating from several software infrastructure modules. A practical solution was implemented. The customer’s success criteria was a 20% decrease in insurance policy creation run time, Our solution provided a 40% decrease in run time! Telecom: A VC backed start-up company was facing critical installation problems in the leading Israeli cellular operator. Knowing that existing system performance would not meet client requirements, the company asked RockeTier to help it boost its performance. RockeTier evaluated the system and implemented a workaround to the system database architecture, boosting the overall system performance by 30%. Following that. the RockeTier team designed the company’s next generation architecture, meeting a throughput of 3000Mbps by design. RockeTier at a Glance RockeTier is a software solutions company, which utilizes its knowledge and skills to help companies from both the enterprise sector and the start-up industry. RockeTier has numerous success stories in solving customers’ system performance bottlenecks and scale out limitations, providing immediate improvements and workarounds in a short time frame and, when necessary, redesign and implementation of the next generation solutions employing grid and/or in- memory databases in the Web 2.0, Telecom and finance markets. Web 2.0: a start-up company providing an innovative electronic advertising and billing system was facing its technological limits. The RockeTier team evaluated and redesigned its system architecture and is currently implementing a scale out grid mechanism and caching algorithms. The solution supports 20 times the original capacity using the same hardware. Moreover it supports semi-linear growth (by simple scale out) and high availability requirements. “ 20% reduction in transaction time within 3 months” “ Boost Performance by a factor of 200” “ 200 million events per day”
The world is changing more and more fast You have to minimize NRE You must support unexpected demand You must provide top service (people now leaves cell operator after single incident, rather then 5, 10 years ago Firms a vulnerable – Citi worth 20Bill $ instead of 200 Bill a year ago Break and Mortar bookstores 15 years ago and Amazon…. Will it happen again to banks, insurance, real estate agencies… IS YOUR MARKET THE NEXT FOR PENETRATION – Finance? Real Estate? How to win a rival that is not existing yet? http://www.johnmwillis.com/ibm/cloud-computing-and-the-enterprise/ -
Sharded database – OLTP Little/No reporting OLAP – must be implemented for reporting Loads data from sharded DBs Custom mechanism Any commercial
Start with nothing: storage, FW, LB, Server and grow… Can buy servers for more than hour