Skilling for SMAC by Anand Deshpande, Founder, Chairman and Managing Director, Persistent Systems


Published on

Social-Mobile-Analytics-Cloud (SMAC) technologies will play a significant role in the building of next generation software products and solutions. Building solutions on this SMAC-stack requires a unique set of technical skills, different from the traditional software programming. This presentation focuses on some of the upcoming SMAC trends and the skills needed to stay relevant as these trends unfold.

Published in: Technology
  • Be the first to comment

Skilling for SMAC by Anand Deshpande, Founder, Chairman and Managing Director, Persistent Systems

  1. 1. Skilling for SMAC Anand Deshpande June 2014 27 July 2014 1
  2. 2. 27 July 2014 2 We are at the early stage of a huge technology wave!
  3. 3. 27 July 2014 3 It would be fun to ride the wave …
  4. 4. 27 July 2014 4 … however, we could very easily be swept away by the wave
  5. 5. 27 July 2014 5 This presentation is targeted to all those who are already in the water and must avoid getting washed away and those who want to ride the waves and join the excitement!
  6. 6. 27 July 2014 6 This presentation is targeted to all those who are already in the water and must avoid getting washed away and those who want to ride the waves and join the excitement!
  7. 7. What are these technology trends? 27 July 2014 7
  8. 8. Meaning of SMAC, & smack … SMAC1 /smæk /[smak] noun (used with technology) n.intr 1. Is an amalgamation of Social, Mobile, Analytics & Cloud Computing technologies for radically improving the effectiveness of a business process and/or change the underlying business model: Netflix used SMAC to drive Blockbuster out of business by re-imagining how movies were rented.. smack2 /smæk /[smak] verb (used with object) v.intr 1. to strike sharply, especially with the open hand or a flat object. 2. to drive or send with a sharp, resounding blow or stroke: to smack a ball over a fence.27 July 2014 8
  9. 9. Cloud computing is already mainstream! ● Driven by economics ... … about 30% reduction in costs! – Cost of infrastructure – Cost of management of infrastructure 927 July 2014
  10. 10. 27 July 2014 10
  11. 11. Growth is being driven by larger companies investing in cloud infrastructure. 27 July 2014 11
  12. 12. 27 July 2014 12
  13. 13. Why does the cloud work? Flexibility and agility Cost Device and Location Independence Scalability Multi-tenancy Security Maintenance Metering 27 July 2014 13
  14. 14. Mobility will be all pervasive ● Mobility is not just small screen. There is more. – Anytime, Anywhere access – Built for Presence and Location, – Task specific apps ● Mobile Devices  tablets and smart phones  smart devices  smart sensors ● Build for mobility. 1427 July 2014
  15. 15. 27 July 2014 15 As of Q1 2014: India has the second largest mobile subscriber base of 933 million subscribers. 91.59 million of these are smartphones. 2014-2018-india-china-australia-japan-growth/#ixzz35SdE8Huy
  16. 16. Sensors  Connectivity  People  Processes 27 July 2014 16
  17. 17. 27 July 2014 17
  18. 18. 27 July 2014 18
  19. 19. 27 July 2014 19
  20. 20. 27 July 2014 20
  21. 21. 27 July 2014 21
  22. 22. Availability and ability to analyze large volumes of data has created new data- centric businesses. 2227 July 2014
  23. 23. 27 July 2014 23
  24. 24. 27 July 2014 24(
  25. 25. Big Buzz around Big Data 2527 July 2014
  26. 26. The Path to Data Stack 3.0: Must support Variety, Volume and Velocity 26 Data Stack 3.0 Dynamic Data Platform Uncovering Key Insights Schema less Approach PBs of Data End User Direct Access Structured + Semi Structured Data Stack 2.0 Enterprise Data Warehouse Support for Decision Making Un-normalized Dimensional Model TBs of Data End User Access Through Reports Structured Data Stack 1.0 Relational Database Systems Recording Business Events Highly Normalized Data GBs of Data End User Access through Ent Apps Structured
  27. 27. We live in a world of platforms and apps 2727 July 2014
  28. 28. More than 75 Billion Apps Downloaded on the Apple AppStore. 27 July 2014 28
  29. 29. We are moving to a world of disposable apps in the enterprise! ● DIY apps ● Rapid development cycles ● Task-centric apps ● Common API-based platform 27 July 2014 29
  30. 30. Open APIs has made the App EcoSystem possible. 27 July 2014 30
  31. 31. Enterprise Value is Shifting to Data 31 Mainframe Operating Systems ERP Apps Data 20152008 Database 19951985197527 July 2014
  32. 32. User Experience is most important Mobile Devices Touch Gestures Voice Wearable Devices Sensor Networks Augmented Reality Large Touch Monitors 27 July 2014 32
  33. 33. Rethinking the user 27 July 2014 33 and their need to be social
  34. 34. Focus on User Experience $$
  35. 35. The cost of software development is significantly lower in the new cloud based software stack 35
  36. 36. 36 Team Size Time Period Traditional Ways On the Cloud 27 July 2014 (<½ the time) x (<½ the team) < (¼ the cost)!
  37. 37. Ability to delay structure has profound impact on the architecture of modern systems 37 Schema/ Structure Data Focus on Read Structure rather than Write Structure
  38. 38. Agile Development Cycle is best suited for Digital Transformation Projects 27 July 2014 38
  39. 39. SMAC - Together! 39 Social • Leveraging the Collective. • Reaching out to the Collective Mobility • Always Connected - collecting & consuming. • Spatial & temporal Context. Analytics • Observe & Optimize the process from data. • Respond to customer in-situ. Cloud • Flexible, & Cost Effective Service Model. • Adapt to changing needs Social Stakeholder Smarts Setup Leveraging SMAC to enhance digital experience. 27 July 2014
  40. 40. 27 July 2014 40
  41. 41. Why Software is Eating the World … 27 July 2014 41 More and more major businesses and industries are being run on software and delivered as online services—from movies to agriculture to national defense. Many of the winners are Silicon Valley-style entrepreneurial technology companies that are invading and overturning established industry structures. Over the next 10 years, I expect many more industries to be disrupted by software, with new world-beating Silicon Valley companies doing the disruption in more cases than not.” The Wall Street Journal in August 2011
  42. 42. Some obvious things 27 July 2014 42 ● Software is being built by small teams with short cycles – Every individual needs to know more. ● Technology is changing rapidly – Must track and learn new technologies all the time. ● Scale is very important – Systems are being built to handle very large volumes ● Data Science is crucial – Sound fundamentals in Statistics and Algorithms is a must
  43. 43. Adopt 𝜋 strategy for building skills: Combine breadth and depth ● Breadth across a wide number of technology areas. ● Depth in two areas: – One technology area – One domain area 27 July 2014 43 Breadth depth
  44. 44. 27 July 2014 44
  45. 45. Framework for Modern Architecture 27 July 2014 45 conceptstools Fundamentals OS/Infrastructure ProcessandTools Data Middleware App Frameworks Mobility/ User Experience
  46. 46. Basic Skills ● Good communication skills and ability to work in teams – English Writing and Speaking skills – Documentation ● Good Typing skills – Fast, all finger typing ● Spreadsheet and Presentation skills – Must know how to manipulate with spreadsheets and make presentations ● Editors – Text Editors such as emacs, vi 27 July 2014 46 Fundamentals
  47. 47. 27 July 2014 47 People think that computer science is the art of geniuses but the actual reality is the opposite, just many people doing things that build on each other, like a wall of mini stones. Donald Knuth
  48. 48. Fundamentals of Computer Science ● Data Structures – Arrays, Heaps, Lists, Trees, Hash tables ● Algorithms – Sorting, Searching, Encryption, ● Complexity – Big O notation, space-time tradeoffs ● Machine Learning – Data Mining, Statistics 27 July 2014 48 Fundamentals
  49. 49. Scale and Data Science are Crucial in the New World ● Statistics and Mathematics – Good understanding of Mathematics especially Statistics – Some skills in Matlab or equivalent ● Machine Learning is an integral part of automation – Algorithms – Data Mining – Tools for data mining 27 July 2014 49 Fundamentals
  50. 50. Hands-on Programming Skills is a must! ● Programming – Object-oriented, Asynchronous Programming, Recursion, Regular Expressions, Patterns - MVC ● Programming Languages – One language for core and heavy-duty programming such as Java – One language which is interpreted and is good for scripting such as Python, – Language of the web -- Javascript – One language that is different – functional, event-based, such as Erlang or Scala 27 July 2014 50 Fundamentals
  51. 51. 27 July 2014 51
  52. 52. User Experience and Design Thinking 27 July 2014 52 Fundamentals
  53. 53. Processes and Tools ● Agile Development – Is well suited for small teams and short cycles ● DevOps – Hottest buzzword … about combining development and operations. ● Tools – Release Management – Tools for automation 27 July 2014 53 ProcessandTools
  54. 54. Use Agile, a tried and tested methodology 27 July 2014 54 Build Incrementally & Iteratively Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan ProcessandTools
  55. 55. 27 July 2014 55 “The best programs are written so that computing machines can perform them quickly and so that human beings can understand them clearly. A programmer is ideally an essayist who works with traditional aesthetic and literary forms as well as mathematical concepts, to communicate the way that an algorithm works and to convince a reader that the results will be correct.” ― Donald Ervin Knuth, Selected Papers on Computer Science
  56. 56. SAF: Scaled Agile Framework ProcessandTools
  57. 57. What is DevOps? ● DevOps is the blending of tasks performed by a company's – application development and – systems operations teams. ● DevOps is a philosophy or cultural approach that promotes better – communication between the two teams as more elements of operations become programmable. 27 July 2014 57 ProcessandTools
  58. 58. DevOps Toolkit ● configuration management tools like Puppet and Chef, ● a repository like GitHub for storing versions of code, Jira for issues and bugs ● tools for monitoring how changes to code affect the environment like Nagios and ● scripting languages like Python, Perl, PHP and Javascript. 27 July 2014 58 ProcessandTools
  59. 59. ● Each team comprises of 7 ±2 people ● Scrum Master: Dedicated or shared ● Product Owner (PO): Only one for a team, PO may work with multiple teams ● Team can be based on Features, Components, Products etc. ● About 100-125 people max at every Agile Release Train (ART) ● Scrum with extreme programming XP engineering practices are used. ● Design/Build/Test (DBT) teams deliver working, fully tested software every two weeks. There are five to nine members of each team ● Every tests, every codes – Developers write code and automated unit tests – Testers work in parallel to write automated acceptance test cases – Developers are Testers work together to satisfy ‘definition of done’ SAFe: Team ProcessandTools
  60. 60. 27 July 2014 60 Without real experience in using the computer to get useful results the computer science major is apt to know all about the marvelous tool except how to use it. Such a person is a mere technician, skilled in manipulating the tool but with little sense of how and when to use it for its basic purposes. • Richard Hamming, 1968 Turing Award lecture, Journal of the ACM 16 (1), January 1969, p.6
  61. 61. Technology Stack ● This is evolving very fast ● Deployments are in the cloud ● Data from Internet of Things ● Consumption on mobile devices with different form factors ● Today’s environment is about – Agility – short, iterative cycles – Scale – performance and volume 27 July 2014 61 conceptstools OS / Infrastructure Data Middleware App Frameworks Mobility/ User Experience
  62. 62. OS/Infrastructure ● Cloud Deployments ● Virtualization ● SDX: Software Defined X={Storage, Network, Data Center, …} ● Linux, Android, iOS ● Cloud based development 27 July 2014 62 OS / Infrastructure
  63. 63. 27 July 2014 63 A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable. Leslie Lamport Turing Award Winner
  64. 64. Data ● Relational Databases ● NoSQL Databases ● Data at Scale ● Fast access and manipulation ● From data to insights ● SQL ● MongoDB ● Cassandra ● Hadoop/Map Reduce 27 July 2014 64 Data
  65. 65. Middleware ● Web Services ● Security and Identity Management ● RESTful APIs ● XML 27 July 2014 65 Middleware
  66. 66. App Frameworks ● Standard patterns that programmers use to ensure speed etc. ● MVC ● Javascript is most popular ● Node.js ● Angular, Express, Ember 27 July 2014 66 App Frameworks
  67. 67. Mobility / User Experience ● Mobility and Mobile devices ● Technology is changing very rapidly ● User Experience is most crucial ● Android/iOS ● HTML5/CSS3 ● Angular.JS ● D3.js 27 July 2014 67 Mobility/ User Experience
  68. 68. 27 July 2014 68 The rise of Google, the rise of Facebook, the rise of Apple, I think are proof that there is a place for computer science as something that solves problems that people face every day. Eric Schmidt Read more at 3lRgZF.99
  69. 69. The Best Time to Plant a Tree 27 July 2014 69 The best time to plant a tree was 20 years ago. The second best time is now. Chinese Proverb
  70. 70. Useful links • GitHub guides 27 July 2014 70 What is it? Any professional software development project makes use of a version control system. Git is one of them. It is essentially a repository of files, typically source code files, shared between multiple people on the team. It tracks changes made to the files by different people, notifies them about any conflicts in their changes and enables them to resolve the conflicts, thus maintaining a single view of the source code for the entire team. Github is a website that hosts repositories using Git as the version control system. for Version Control
  71. 71. What is it? Software projects need to be planned and tracked to ensure that they are delivered on time, within budget and with desired quality. Jira is a project management tool that adopts the commonly used and preferred Agile methodology. Jira has features to plan sprints, view the backlog, track key metrics, trace requirement to code, and to enable teams to collaborate. Useful links ● Jira test project ● Jira Agile ● Jira demo videos 27 July 2014 71 for Agile project management
  72. 72. What is it? Chef automates the configuration and management of the infrastructure, i.e. servers and applications. It makes it easy to deploy the infrastructure to any physical, virtual, or cloud location, no matter what the size of the infrastructure is. Chef enables the system administrator to create “recipes” or abstract definitions describing how each part of the infrastructure should be built and managed. These definitions are managed like source code. Chef then applies those definitions to servers and applications, as specified, resulting in a fully automated infrastructure. Useful links ● Getting started with Chef ● Learn Chef ● Learning Chef Series 27 July 2014 72 for automated infrastructure management
  73. 73. What is it? APIs are windows through which applications on the web communicate with each other and share data. Facebook, Amazon, Google Maps and innumerable others expose their services over APIs which enables developers to programmatically avail their services and create interesting mashups. For example, wherever you see a “Share this on Facebook” icon, a Facebook API is being called in the background. REST is an architectural style of building APIs and a popularly used one. It leverages the HTTP protocol that has made the web what it is today. Useful links ● How to GET a Cup of Coffee ● REST in Practice by Jim Webber ● How to use APIs from Twitter, Google & Facebook 27 July 2014 73 for opening up services and data
  74. 74. What is it? OAuth is important for enabling web apps to use the services of other apps through APIs. It is an authentication protocol that allows the end user to say, “like” any blog she comes across using her Facebook profile without the blogging website getting access to her Facebook password. Useful links ● OAuth for your API: The Big Picture ● Twitter Dev FAQ for OAuth 27 July 2014 74 for security in the world of connected APIs
  75. 75. What is it? Javascript is THE language of the Web and is increasingly penetrating in the full stack of modern application development. Javascript Object Notation (JSON) is a format to represent data during interchange such as over an API. It is popular because of its language indepenence, lightweight (so network friendly), human readable, and can be easily parsed by machines. Useful links ● Javascript: The Good Parts by Douglas Crockford ● JSON for beginners videos 27 July 2014 75 for representing interchangeable data
  76. 76. What is it? AngularJS is a entirely client- side framework for creating web apps such as Gmail with complex dynamics. It brings MVC capability to browser based apps. It does this through a rich vocabulary of directives that embed dynamic action. Useful links 27 July 2014 76 for creating complex web apps
  77. 77. What is it? D3.js is a Javascript library for creating powerful visualizations based on data. Data is an important asset and can give critical insights, but by itself data is difficult to make sense of. What is needed are visual views of the data that bring out the meaning in it. Useful links 27 July 2014 77 for creating powerful visualizations
  78. 78. What is it? Massive amount of data is being generated continuously. To process large data sets, scale-out architecture and MapReduce are used to divide the large data into small chunks and process it in parallel on commodity hardware. Hadoop is an open-source implementation of MapReduce. Useful links ● ● 2.1/mapred_tutorial.html 27 July 2014 78 for processing large data sets
  79. 79. 27 July 2014 79 for building event-driven applications What is it? Node.js and express.js together are a framework for easily building real-time event-driven apps with high throughput and low latency, example web chat. They are both in Javascript which allows for code sharing on server and client side. Together with Angular and MongoDB they are known as the MEAN stack or Javascript full stack. Useful links ● Understanding event-driven programming ● MEAN Stack
  80. 80. 27 July 2014 80 for creating quick prototypes What is it? HTML5 and CSS3 are the latest of these standards. They can be used for creating static prototypes quickly. Useful links ● HTML5 Introduction ● CSS3 Introduction
  81. 81. 27 July 2014 81 SDKs for creating mobile apps What is it? iPhone and Android software development kits (SDKs) allow developers to create native apps that can be run on mobile and tablet devices. Useful resources ● iOS Dev Center ● Android Developers
  82. 82. Useful resources ● Visual guide to NoSql systems ● Comparison of NoSql DBs 27 July 2014 82 for storing really BIG data Pick Two Partition Tolerance Availability Consistency MongoDB CassandraNeo4j, RDBMS What is it?
  83. 83. 27 July 2014 83 What is it? Python works on the philosophy that speed of development is important. Readable and terse code is part of this, and so is access to powerful constructs that avoid tedious repetition of code. Useful resources ● Python Tutorial ● Django Getting Started for getting up and running quickly
  84. 84. 27 July 2014 84 What is it? Gets you the best of all programming worlds. Manage your domain complexities with OO constructs while implementing behaviors with the power of Functional programming. All this comes with the time-tested goodness of JVM. Useful resources ● Course on Scala by Martin Odersky for all the heavy-lifting
  85. 85. 27 July 2014 85 What is it? xUnit testing frameworks allow automation of unit tests ensuring that existing code doesn’t break as more and more code gets added. JUnit, PyUnit and Qunit complement to programming languages Scala, Python and Javascript. Useful resources ● JUnit ● PyUnit ● QUnit JUnit, PyUnit, QUnit to create robust code
  86. 86. 27 July 2014 86 Vi, Sublime Text Editors for writing code What is it? Use a powerful text editor such as Vi or Sublime to punch in code. NOT Notepad. Useful resources ● Vi Editor ● Sublime Text
  87. 87. 27 July 2014 87 Shell Commands & Scripting What is it? Most elements of your stack have shell interfaces. Be comfortable with shell commands and have a basic knowledge of shell scripting to be able to quickly run and automate most tasks such as builds, debugging etc. Useful resources • The Unix Programming Environment by Kernighan & Pike