Technical details with lot of numbers. Git, Redmine, Hipchat, 10 654 working hours and more. Now, and only now, you could see background details of very sophisticated e-commerce solution.

1. Divante
eBusiness
Agency

2. Hello
World!

3. About
Divante
• eBusiness
Agency
from
Poland
• Opera=ng
since
2008
• Over
150
people
in
our
office
in
Wroclaw,
Poland
• Clients
from
Europe
and
the
U.S.
• Within
your
reach:
1.5h
flight
from
London,
Berlin,
Oslo,
Amsterdam,
Paris
• SCRUM
methodology,
ensuring
high
quality
and
flexible
approach
to
business
requirements
• Case
studies:
hRp://divante.co/porTolio/

4. Team
• Over
150
employees
• Over
80
developers
(PHP,
NET,
RoR,
iOS,
Android)
• 6
Magento
cer=fied
developers
• 10
project
managers
• 12
UX
designers
• 5
QA
testers

5. Hello
CDP!

6. The
Challenge
6
To
develop
a
new
version
of
CDP.pl
-­‐
with
the
support
for
both
online
and
offline
products,
with
a
full
stack
logis>cs,
and
the
business
process
support.
Main
challenges:
-­‐ Very
short
Time
To
Market
(the
project
started
on
1
Feb,
the
first
publica=on
date
was
planned
for
September)
-­‐ ElasCc
approach
–
some
reqs.
needed
to
be
defined
later,
mostly
logis=c
opera=ons
(almost)
real
Cme
stock
management
for
both
internal
and
external
stocks
–
about
100
000+
SKUs
-­‐ High
scalability
(up
to
3-­‐6K
users
on-­‐site)
–
to
be
ready
for
„The
Witcher
3”
premiere
-­‐ Online
content
delivery
–
audiobooks
(Audioteka.pl),
videos
(+DRM),
e-­‐books
(eLibri),
and
game
downloads
(external
CDN
integra=on
–
Atende
CDN)
-­‐ Lots
of
integraCons
and
BP
automaCon
–
AX
ERP,
Azymut,
DDD,
AB,
Ac=on,
ABC
Data,
Mailchimp,
Rekman,
Monolith,
Inpost,
DPD
…
-­‐ Other
custom
features:
PIM,
ShopInShop,
Preorders
…
-­‐ Full
mobile
support
(RWD)
-­‐ No
trade-­‐offs
on
UI/UX
(a
coopera=on
with
Ars
Thanea)

7. The
Numbers
7
CDP.pl
was
a
rather
huge
IT
project.
Let
the
numbers
speak:
-­‐ 10
564
working
hours
-­‐ 12months
+
21
days
of
work
-­‐ 12
team
members
-­‐ more
than
7000
git
commits
-­‐ 2500+
redmine
=ckets
closed
-­‐ 15
servers
behind
app
–
2
proxies,
5
apps,
2
searches,
2
workers,
2
db,
2
sta=cs
-­‐ SLA
–
high
priority
recoveries
within
1h,
24/7
-­‐ 200
CPU
cores
+
700GB
of
RAM

8. The
Process
8
Analysis
and
planning,
es=mates
• collec=ng
data
• es=mates
• planning
Design
• interac=ve
projects
• graphics
Implementa=on
• according
to
Divante’s
good
prac=ces
Go
live
• func=onal
tests
• security
• integra=on
Stabiliza=on
• SLA
assurance
Ars
Thanea
+
CDP
Divante
+
CDP
Design
Code
Test
Feedback

9. The
Process
Throughout
the
project
we:
• Used
„SCRUM”
-­‐
daily
mee=ngs,
demos,
2
week
long
sprints
• Did
analysis
(BA
works
on
Redmine
where
backlog
was
and
analyse
one
sprint
ahead)
• Tests
goes
along
with
development
(each
=cket
had
to
be
closed
by
a
tester)
• Did
frontend
works
along
with
backend
development
(we
started
with
integra=ons/
backend
works,
while
wai=ng
for
the
final
mockups
and
graphics)
• Pay-­‐as-­‐you-­‐go
-­‐
aqer
each
sprint-­‐based
payment
acceptance
• Weekly
repor=ng
• Toggl
to
measure
=me
• Each
sprint
-­‐
a
summary
and
a
plan
of
the
coming
week
Ini>ally,
we
formally
scheduled
only
the
first
2-­‐3
months,
then
we
switched
to
the
backlog
/
sprint
planning,
and
returned
to
the
schedules
in
the
last
3
months
to
es>mate
the
publica>on
date.

10. The
Tools
• Redmine
–
backlog
+
issue
tracking
+
es=ma=ons
• Toggl.com
–
=me
tracking
(for
further
es=ma=ons/reports)
• Git+Gitlab
-­‐
source
code
+
code
review
• phpStorm
–
IDE
by
choice
for
devs.
and
frontend
devs.
• Google
hangouts
(daily
mee=ngs)
• Hipchat
–
internal
communica=on
• Excel
–
es=ma=ons,
• MS
Project
–
scheduling
purposes
(to
be
honest:
we
don’t
like
this
app)
To
maintain
development
process
we
used
very
simple
tools
.

11. The
Team
- PM
–
Tomek
–
responsible
for
the
final
results
and
for
the
process,
gathering
requirements
and
some=mes
also
for
analysis
and
tests,
- Tech
TL
–
Tomek
–
the
technical
team
leader
–
dev.
planning,
and
architecture
...
- QA
-­‐
Damin,
Łukasz
–
testers,
- Core
dev.
team
–
Tomek,
Paweł,
Maciek,
Marceli,
Kuba,
Kamil,
Adrian,
Anton
–
they
made
most
of
the
commits
;)
- Frontend
dev.
team
–
Tomek
and
Marek
–
they
cooperated
with
Ars
Thanea
-­‐
the
second
most-­‐commiRed
group
in
the
project
;)
- Sys.
Adms.
–
Paweł,
Marcin
–
responsible
for
the
infrastructure
and
scalability
architecture
- BA
–
Bartek,
Agata
–
gathering
requirements
and
process
analysis
We
created
a
fully
integrated
team
of
equal
members
on
CDP,
Divante’s
and
AT
sides.
We
helped
recruit
and
train
CDP
IT
Team.
We
work
together.

12. The
Team

13. The
Team

14. The
Team
Effort
13
months
of
work
was
shorter
than
we
thought
at
the
beginning!
The
bigger
a
team
is,
the
more
>me
you
need
for
organiza>on
and
communica>on.
16w
overall
analysis
=me
600
hrs
48w
backend
dev.
+
tests
First
2
months:
4
devs.
Next
10
months:
6-­‐7
devs.
7000
hrs
+
20w
frontend
dev.
-­‐
ongoing
coop.
with
AT
2
frontend
devs.
1600
hrs
about
30%
=me
for
PM
and
communica=on

15. Wireframes
/
Design
About
80
pages
of
designs
(AWD
was
not
designed
on
mockups)

16. Gfx
+
Frontend
Dev.
About
80
pages
of
designs
(AWD
was
not
designed
on
mockups)
-­‐ AWD
for
all
major
mobile
plaTorms
-­‐ Ongoing
tests
with
Ars
Thanea
-­‐ CSS3
+
sass,
jQuery

17. Coding
-­‐ PHP
+
Magento1
-­‐ Peer
code
review
on
a
daily
basis
(phabricator)
-­‐ GiTlow
branching
model
(hRp://danielkummer.github.io/git-­‐flow-­‐cheatsheet/)
-­‐ Zend
Coding
Standards
(yep
and
Magento1)

18. To
Op=mize
Early?
-­‐ At
the
beginning
of
the
project
we
designed
the
architecture
(it
stayed
mostly
unchanged
in
the
process)
-­‐ We
designed
applica=on
with
performance
on
our
minds
from
the
start
-­‐ We
created
a
11
point-­‐long
rules
booklet
for
developers,
star=ng
from
day
one:
-­‐ Be
prepared
to
read/write
the
DB
replica=on
in
all
places
in
the
code
-­‐ Be
prepared
for
Varnish
as
HTTP
proxy
-­‐ Use
SOLR
search
for
search
and
catalog
browsing
-­‐ Use
async
indexa=on
for
Magento
-­‐ Always
use
the
“flat”
mechanism
in
Magento
-­‐ Use
Redis
for
cache
and
sessions
-­‐ Carry
out
a
performance
test
at
least
once
a
week
-­‐ A
daily
code
review
is
a
must
-­‐ Log
everything
(it
could
be
useful
later
on)
-­‐ A
page
response
=me
cannot
be
longer
than
1
sec
-­‐ Popular
pages
(product,
category,
hp)
make
no
SQL
queries
when
loaded
from
cache
-­‐ Strictly
speaking:
In
fact,
we
did
some
op=miza=ons
at
the
end
of
the
project
(some
more
about
it
–
later
on)

19. QA
-­‐ Func=onal
tests
on
daily
basis
(the
testers
closed
=ckets)
-­‐ 2
UI
audits
by
Ars
Thanea
-­‐ 3
weeks
of
UATs
-­‐ Performance
tests
(jMeter)
-­‐ Security
tests
(skipfish
+
code
review)
-­‐ Selenium
tests
for
key
paths

20. Magento
–
Benefits
and
Challenges
-­‐ With
100
000+
SKUs,
performance
should
be
treated
seriously
on
Magento
(we
use
SOLR
and
MongoDB
for
search
and
integra=ons)
-­‐ Most
of
the
admin-­‐panel
func=ons
available
in
standard
Magento
–
orders,
customers,
etc.
–
are
used
without
any
major
modifica=ons
-­‐ We
have
to
implement
custom
checkout,
stock
management,
and
online
content
func=onali=es
-­‐ Magento
implies
making
some
architectural
decisions
and
using
paRerns
to
make
the
code
more
understandable
for
new
developers
Magento
is
the
most
popular
open
source
e-­‐commerce
pla]orm.
It
is
an
industry
standard
in
e-­‐Commerce.

21. Custom
Features
Most
dev.
>me
was
devoted
to
two
tasks:
integra>ons
and
custom
features.
AX
IP
SOLR
KEYS
SSM
VARNISH
SIS/HP
CHECKOUT
SHELF

22. -­‐ Integra=on
with
product
suppliers,
like:
Azymut,
DDD,
AB,
ABC
Data,
Ac=on
…
-­‐ Management
of
descrip=ons,
prices,
photos
–
merging
products
from
diverse
suppliers
-­‐ Use
of
MongoDB
for
incoming
data
and
Gearman
Queue
server
for
async
workers
IP
Custom
Features:
IP
(Integra=on
Panel)

23. Custom
Features:
Shelf
-­‐ Allowing
users
to
download
games,
find
the
keys
to
games,
listen
to
audiobooks,
and
download
e-­‐books
SHELF

24. SIS/
HP
Custom
Features:
Shop
In
Shop
/
Home
Page
-­‐ Editor
func=on
for
the
HP
and
Shop
in
Shop
sec=ons
with
a
mobile
device
support
-­‐ Thanks
to
Magento,
two
step
view
paRern,
mosaics
can
be
embedded
almost
everywhere

25. Integra=ons
The
Webshop
is
never
an
only
puzzle.
Integra>ons
are
far
from
trivial,
in
most
cases.
We
have
made
the
following
integra=ons:
-­‐ Microsoq
AX
–
stock
updates,
orders,
and
RMAs
-­‐ Azymut,
DDD,
Ac=on,
AB,
ABC
Data
–
stock
updates
and
product
data
(feed
for
IP)
-­‐ Audioteka
–
audiobooks
with
watermarks
-­‐ Atende
–
CDN
opera=ons
on
downloadable
games
and
movies
-­‐ eLibri
–
ebook
downloads
and
watermarks
-­‐ DPD,
Poczta
Polska,
Xpress
Kurierzy,
Paczkomaty
Inpost,
and
PwR
–
parcel
tracking
-­‐ payU,
paypall,
and
inpay
(bitcoins)

26. High
Scalability
We
are
prepared
for
horizontal
scaling
of
most
parts
of
the
CDP.pl
infrastructure.
-­‐ Infrastructure
divided
into
separated
layers:
proxy-­‐>app-­‐>db
+
worker
+
sta=c
/
CDN
-­‐ VirtualizaCon
based
(KVM)
2N
architecture
(with
hot-­‐backup
nodes
ready
to
be
switched
on
during
peaks)
-­‐ Varnish
+
ESI
for
full
page
caching
-­‐ Redis
for
session
management
and
cache
-­‐ Master/Slave
replicaCon
of
Percona
DB
(na=ve
Magento
mechanism)
–
on
each
app
server
there
is
a
Percona
Slave
-­‐ Gearman
queues
for
=me
consuming
tasks
(mostly
integra=ons
/
ERP
backed
processes)
-­‐ We
use
New
Relic
for
performance
monitoring
and
sugges=ons
-­‐ SOLR
as
a
base
for
catalog
opera=ons
(search
and
browsing)
–
it
bypasses
most
db
opera=ons
on
catalog
browsing

27.
• Divante
S.W.A.T.
Group
• Data
recovery
plan
(speed,
scope,
schedule,
procedures)
• Documenta=on
for
administrators,
procedures,
and
a
QA
list
• Fully
automa=c
monitoring
(we
use
Zabbix)
• Task
automa=on
(Ansible,
Chef)
• SLA
assurance
–
guaranteed
availability,
fixing
issues
within
1h
SLA
SLA
warranty
is
an
opConal
service
we
offer
along
with
the
infrastructure
/
hos=ng.
Consul=ng
services
cover:
S.W.A.T. GROUP
Divante
immediately takes actions on errors
and technical problems. A maximum
recovery time is
1 hour.

28. Thank
You!
Piotr
Karwatka,
CTO
Contact
me
directly
on:
– e-­‐mail:
pkarwatka@divante.pl
– linkedin:
hRps://pl.linkedin.com/in/piotrkarwatka
– skype:
pkarwatka
– phone:
0048
501
601
055
Our
website:
hRp://divante.co
ul.
Kościuszki
14,
50-­‐038
Wrocław,
Poland
(hRp://divante.co)