Software Development Whats & Whys


Published on

Sarcastic approach to software engineers needs. Code review, release management, caching, logging, product managers, load balancer, bandwidth, archiving, deployment, testing, agile, scrum, kanban, git, team management, meetings..

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Development Whats & Whys

  1. 1. WHATs & WHYs What you need and why you need it! Harun Yardımcı Software Architect @ eBay
  2. 2. Notes on Notes • Slides are unordered • There are always alternative tools. Concentrate on the idea! • Not talking about technology HELP!! Someone please make this presentation beautiful
  3. 3. Release Management • Release Notes You have to know exactly what is deployed and when it is deployed. There is no exception. • Release Manager You need this title to give the responsibility. So you can blame someone. Can’t hire a new guy, just hand it to smartest one. • Project Manager Usually they will tell you what needs to be deployed.
  4. 4. Code Review • How do you know you write a good quality code? Share your code with your colleagues, they will enjoy to embarrass you • Tell others that their code smells So you have a chance to get your revenge • Standards and best practices There is always a better way and probably someone did it
  5. 5. Network Bandwidth • Measure the network bandwidth on each machine. You will need to know it just before pulling hairs out of your head when your application is not responding. o volker gropp o Iptraf There are many others. Choose one and go.
  6. 6. MHA for MySQL • Master High Availability Manager and tools for MySQL Tools are faster than you! They don’t get tired and don’t complains. There is nothing to loose since it works when your master DB is crashed
  7. 7. Load Balancer • Hardware Balancer o Citrix NetScaler o F5 More features and better scalability. Costs you more. • Software Balancer o HAProxy Software load balancers seems cheaper and they perform quite well but you need a good hardware to run it!
  8. 8. Online Schema Change Who wants to shutdown the site and alter a table for an hour, then wait for the slaves to catch up? No one! Not any more. o pt-online-schema-change o oak-online-alter-table Openark and Percona has many more useful tools. Use them! Tools are they to make your life easier.
  9. 9. Archiving • Move old and expired data to archive o Small dataset and index = $$ CASH $$ o Faster response, low latency = $$ CASH $$ o Maintainable database = $$ CASH $$ o Faster dumps and restores = $$ CASH $$ Don’t forget to write scripts to get it back when needed!
  10. 10. No Database Dependency • NO TRIGGERS • NO STORED PROCEDURES • NO VIEWS o Nobody will know the business logic o Impossible to learn how a piece of code is functioning o Not easy to maintain Forget immediately what your old-school database teacher told you in the college. Don’t use them.
  11. 11. CACHING • Cache everything • More importantly evict everything if you can’t evict do not cache • Web Developers Toolbox: o Varnish, Squid, etc. o Memcached, Redis, Couchbase, etc. o Hazelcast, Terracotta, Coherence, EhCache, etc. CPU cache, disk buffer, page cache, browser cache, DNS cache, Proxy cache, HTTP Etag, Expire Headers, ….
  12. 12. Server Configuration Management • Declare once execute many times Write a manifesto file as an configuration and define the standards. Each servers configuration will be identical. o Puppet o Chef Easy and fast to configure a new server Very fast to change any settings Identical configurations Central Control More holidays for system admins
  13. 13. LOGGING • Don’t generate logs if you don’t read them. At least you can save I/O on disks. • Centralized logs enables you to search every action of a component (user, product, sale, etc) Writing a proper log in each application, makes you enable to search for actions of userId:101. o Logstash o Graylog2 Your boss will love you as long as you can read the past! Capped collection, Elastic Search, Gelfino, Grok, Kibana, etc.
  14. 14. Auto Deployment This is 21st Century! Manual deployment is unacceptable. Don’t wait anyone to ask you to do it. Use Jenkins, Maven, Ant, Bash Script, Fabric, Capistrano, Python, Perl or whatever you can.
  15. 15. Still reading this? Don’t! Go and write a script. Now!
  16. 16. Test Environments • Separated Test Environments • Create Test Data • Test more than once: o Staging Have more than one test environment Testing affects to number of bugs dramatically.
  17. 17. Test Data • Every application has to create its own test data. o Your application, your test data. You know better and It is your responsibility. • Easy to test it on every environment • Reduces dependencies • No need to database syncs to test DBs
  18. 18. Data Sanitizing • Some users have same passwords for Facebook, Gmail and so • Users emails, bank accounts, what they bought, what they like are valuable data Can be sold to third parties or can be used to threaten someone • Users data is personal and it should be protected Some employees are very curious about others, specially about their friends. Block DB Access, Authentication, Auditing, Data Encryption There is no “Freedom of Information Law” on your users data
  19. 19. Development Environments • Own Machines (Leads to “but it works on my machine” problem) • Virtual Machines • Dedicated Servers Give them options and let them choose one they feel comfortable. Should be Simple and Fast!
  20. 20. Agile Methodology Developers loves it! • Basic Idea: o Small, well defined tasks o Solve problems and unclear points immediately o Be a communication bridge o Remove disturbance o Plan ahead o Measure performance o Learn the limits Scrum, Kanban, Scrumban, XP, Kaizen, Six Sigma, etc..
  21. 21. Monitoring • Monitor everything load, disk, network, memory, db response, cache response, data size, counts, 404s, 500s, every little tiny bit can be important. • Define milestones • Watch after deployments • Write Health Check services, checking the port not enough • Analyze Trends Forecast the load on next Christmas, campaign, holiday
  22. 22. Source Control • Use GIT If you think that you don’t need it, then most probably you are doing something wrong. GitHub, Gitolite, Gitosis, Gitstash, Gitlist, Gitweb, etc.. Define Branching Model Authenticate and Authorize the Access
  23. 23. Project Managers • You need someone who can move some excel boxes from one to another cell • Sometimes some cells needs to be merged together Those excel cells are project plan, execution plan, priority of tasks, risk analysis, etc. They typically have extensive background in development I wish so!
  24. 24. Asynchronize • Don’t keep users waiting • Add some actions to a queue • Reduce Race-Conditions Everything will be okay! In time
  25. 25. Enable / Disable • Flexibility • Partially Maintenance • Less Downtime • Decoupled features Enable / Disable messages for a time while payment is working or any feature..
  26. 26. Budget For Teams • Go for lunch, take them to cinema, do some team building activities They will never forget Loyalty will increase Know each other better Increase communication between members • Have a half day off once a month to: Do something fun together Get relaxed Listen their problems They need to talk each other and to do something together!
  27. 27. Legacy Codes and Systems • Laugh at them • Make fun of them • Delete them • Remove them • Burn them Be courage about designing and creating them from scratch. They are not so big things as you think!
  28. 28. Meetings • Don’t have meetings • If you really need it o Stand-up meetings (Less time) o Invite very few people o Have someone to take notes o Define time box (30 min. max) • Need more? Break it into small pieces o Share outcome and result of the meeting
  29. 29. Other Headaches • Cross Functional Teams Good! • Outsourcing Generally Bad! • Centralization / Decentralization Varies case by case. Just think about it before you start. • Details and early optimizations blocks you to start something • People need uninterrupted time to finish their tasks • Context Over Consistency What makes sense here may not makes sense there* • Spend some time with other departments * Rework
  30. 30. Thanks! • Have a question? • Liked it? o Use it, share it, show it to your boss • Didn’t like it? o Pretend that you liked it and help me to make it better