Generative Adversarial Network-based Visual Aware Interactive
Fashion Design Framework
By Ashenafi Workie
Advisor :Prof Yun koo Chung(PhD.)
Presentation Outline
 Introduction
 Statement of problem
 Objective
 Scope, constraints & contribution
 Related works
 Theoretical & conceptual framework
 Proposed Methods
 Thesis budget and schedule timeline
 Conclusion
2
33
Domains in the fashion design requires
 Effective interactive Design
Interactive design may applied in
 Clothing and painting art
 Building and home utility products
This can be achieve using generative model approach giving user given
input for product category
Introduction
44
Generating conditioned image
 Color
 shape
 style
 Pattern
 texture etc.)
Improve some nose and artifacts
Pull to public cloth dataset
Motivation
55
 challenging task
hag subjectivity
Focus on designer interest
(no more creativity) and Difficult to
share style
Difficulties in training and produce
high-resolution.[2]
Statement of problem
66
Here are research question
 REQ1: What are the effective ways to interactive fashion design?
 REQ2: What kind proposed solution to be used for generation of image for
better interactive design?
 REQ3: How the quality of generated fashion design can be improved and
comparison of results based evaluation metrics?
Research Question
77
Design and develop visual aware interactive fashion design framework
through deep generative network to improve quality and fine-grained style.
General Objective
88
Specific Objective
Here are list of specific objective
Investigate the domain using Systematic Literature review
Collect dataset from different source
Prepare annotation and preprocessing the data
Create a fusion CGAN and SRGAN
Train the model and feed the data
Present the result and evaluate model
99
Scope
Here are list of scope of research
Collecting and pre-processing the dataset
Creating a fusion of conditional GAN ,where it easy for controlling and
SRGAN to enhance the quality of generated image
Present result and evaluation result
Constraints
No sufficient dataset available
lack of high computational resource(GPU) .
1010
[2] Kang et.al Visually-Aware Fashion Recommendation and Design with
Generative Image Models.
Methods :GAN
 Objective : suggesting existing items on fashion images for both prediction
and design
 Gap(future research) :
 Poor quality and
 difficult control fine-grained style
Related works
1111
Related works
[5] Y. R et.al (2018) FashionGAN: Display your fashion design using
Conditional Generative Adversarial Nets
Methods :FashionGAN
Objective :Automatically generated fashion image and establishes a
bijection relationship between generated image .
Gap(future research) :
 workable for single-color and regular
it cannot map an irregular one with multiple colors and style
Needed further research
1212
Related works
Author and Title Methodology Objective GAP(future work)
[1] Yildirim et.al (2019)
Generating High-
Resolution Fashion Model
Images Wearing Custom
Outfits
 unconditional
 Conditional
GAN
 Transfer the
style/color
 generate images
of fashion models
• Quality is not good
and consistent
model
[3] Christoph et. Al A
Generative Model of
People in Clothing
• graphics
pipeline
design and
• 3D data
acquisition
 Semantic
segmentation
prediction
 texture prediction
• High
computational
resources for
rendering
1313
Related works cont...d
Author and Title Methodology Objective GAP(future work)
[4] Zhu et.al (2017)
Be Your Own Prada:
Fashion Synthesis with
Structural Coherence
 semantic
segmentation
map.
 compositional
mapping layer
collecting
sentence
• model
can also render textured
background .
• Do not post
processing of
the background
[6] Pandey
et al (2019)Poly-gan:
Multi-conditioned Gan
For Fashion Synthesis
• Poly-GAN  multiple inputs and is
suitable for many
tasks, image
alignment, image
stitching and
inpainting
Design
architecture to
apply for a variety
of applications.
1414
Theoretical framework of GAN
CGAN is very similar to GAN, except both
the generator and discriminator are
conditioned on some extra information, y.
Gan’s have two networks
1.Generator G takes some random
noise and tries to produce real input
training images by fooling the
discriminator D.
2.Discriminator D tries to classify the
input being real if it’s coming from the
training data and fake if it’s coming
from the generator G.
1515
Methodology
Dataset
 Scrapping website like pintrest , esy.com
 Download from channel and page
 Request image from fashion studio
 Capture image
Preprocessing
Augmentation, Scaling , background
Feature extraction
 GaitGAN:[34] is used to reduce the effect
of variations
 PCA-> to reduce the dimension
1
PreprocessingDataset
2
Feature Ex and Red
3 4
Train for
CGAN
Train for
SRGAN
Present
result
5 6
1616
LR image
HR image
generator
High quality fake or real
Dataset
Proposed architecture workflow
Xr
Y
Y
Z
Generator
Discriminator
Generator
Discriminator
D
D
A Generator(An artist) neural network.
•Discriminator has to
correctly label real images
which are coming from
training data set as “real”.
1717
Cont...d
CGAN for generating condition fashion images
SRGAN to Refine the quality from generated one
Nobility
 Enhancement of Quality image
 control fine-grained style
 Improve to work with multiple
color
1818
Research tools
Software tools
Tensor-flow
Keras
MS office
Mendeley desktop
 Colab
Hardware tools
 GPU
RAM
SSD
Camera
1919
Contribution and application of result
Collecting dataset and pull to github
Creating of fusion algorithm CGAN and SRGAN
Enhance quality generated image(SRGAN) and Control fine grained style
(CGAN)
Application of result
Style transfer across multiple domain image generation
Image to image translation :learn mapping input and output
photo editing and image enhancement
2020
Budget plan (operation cost)
Operation cost
Item No Item Name Unit price Quantity * unit Total Source
1 Data collection 2000 3*2000 6000
Coveredby
Researchfund
2 Data preparation 1500 3*1500 4500
3 Print cost and binding 650 4*650 2600
3 Transport cost 300 6*300 1800
5 Contingency plan (10%) 10% 1*2500 2500
Total operation cost 14700
Total material cost 7800
Total cost operation +material _contingency =25,000 ETB
Material cost
Item no Item name Specification Unit price Qty. Total
1 GPU MSI RTX 2070 250$ 1 -
2 Flash USB SanDisk 32GB 10$ 1 -
Total 210 USD =7800 ETB
2121
July 2019 days Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
Literature review 334
Writing Proposal 99
Proposal defense -
Data collection 120
Data preparation 50
Train model 60
Verify the model 90
Result Report and
summarization
90
TodayTodayCompleteComplete IncompleteIncomplete Not StartedNot Started
2020
June 2020
Months
Research timeline and schedule
(+251) 919-283004
(+251) 933-880022
Ashumare.com
Ashenafiworkie@gmail.com
Thank You
By Ashenafi Workie

M sc thesis proposal v4

  • 1.
    Generative Adversarial Network-basedVisual Aware Interactive Fashion Design Framework By Ashenafi Workie Advisor :Prof Yun koo Chung(PhD.)
  • 2.
    Presentation Outline  Introduction Statement of problem  Objective  Scope, constraints & contribution  Related works  Theoretical & conceptual framework  Proposed Methods  Thesis budget and schedule timeline  Conclusion 2
  • 3.
    33 Domains in thefashion design requires  Effective interactive Design Interactive design may applied in  Clothing and painting art  Building and home utility products This can be achieve using generative model approach giving user given input for product category Introduction
  • 4.
    44 Generating conditioned image Color  shape  style  Pattern  texture etc.) Improve some nose and artifacts Pull to public cloth dataset Motivation
  • 5.
    55  challenging task hagsubjectivity Focus on designer interest (no more creativity) and Difficult to share style Difficulties in training and produce high-resolution.[2] Statement of problem
  • 6.
    66 Here are researchquestion  REQ1: What are the effective ways to interactive fashion design?  REQ2: What kind proposed solution to be used for generation of image for better interactive design?  REQ3: How the quality of generated fashion design can be improved and comparison of results based evaluation metrics? Research Question
  • 7.
    77 Design and developvisual aware interactive fashion design framework through deep generative network to improve quality and fine-grained style. General Objective
  • 8.
    88 Specific Objective Here arelist of specific objective Investigate the domain using Systematic Literature review Collect dataset from different source Prepare annotation and preprocessing the data Create a fusion CGAN and SRGAN Train the model and feed the data Present the result and evaluate model
  • 9.
    99 Scope Here are listof scope of research Collecting and pre-processing the dataset Creating a fusion of conditional GAN ,where it easy for controlling and SRGAN to enhance the quality of generated image Present result and evaluation result Constraints No sufficient dataset available lack of high computational resource(GPU) .
  • 10.
    1010 [2] Kang et.alVisually-Aware Fashion Recommendation and Design with Generative Image Models. Methods :GAN  Objective : suggesting existing items on fashion images for both prediction and design  Gap(future research) :  Poor quality and  difficult control fine-grained style Related works
  • 11.
    1111 Related works [5] Y.R et.al (2018) FashionGAN: Display your fashion design using Conditional Generative Adversarial Nets Methods :FashionGAN Objective :Automatically generated fashion image and establishes a bijection relationship between generated image . Gap(future research) :  workable for single-color and regular it cannot map an irregular one with multiple colors and style Needed further research
  • 12.
    1212 Related works Author andTitle Methodology Objective GAP(future work) [1] Yildirim et.al (2019) Generating High- Resolution Fashion Model Images Wearing Custom Outfits  unconditional  Conditional GAN  Transfer the style/color  generate images of fashion models • Quality is not good and consistent model [3] Christoph et. Al A Generative Model of People in Clothing • graphics pipeline design and • 3D data acquisition  Semantic segmentation prediction  texture prediction • High computational resources for rendering
  • 13.
    1313 Related works cont...d Authorand Title Methodology Objective GAP(future work) [4] Zhu et.al (2017) Be Your Own Prada: Fashion Synthesis with Structural Coherence  semantic segmentation map.  compositional mapping layer collecting sentence • model can also render textured background . • Do not post processing of the background [6] Pandey et al (2019)Poly-gan: Multi-conditioned Gan For Fashion Synthesis • Poly-GAN  multiple inputs and is suitable for many tasks, image alignment, image stitching and inpainting Design architecture to apply for a variety of applications.
  • 14.
    1414 Theoretical framework ofGAN CGAN is very similar to GAN, except both the generator and discriminator are conditioned on some extra information, y. Gan’s have two networks 1.Generator G takes some random noise and tries to produce real input training images by fooling the discriminator D. 2.Discriminator D tries to classify the input being real if it’s coming from the training data and fake if it’s coming from the generator G.
  • 15.
    1515 Methodology Dataset  Scrapping websitelike pintrest , esy.com  Download from channel and page  Request image from fashion studio  Capture image Preprocessing Augmentation, Scaling , background Feature extraction  GaitGAN:[34] is used to reduce the effect of variations  PCA-> to reduce the dimension 1 PreprocessingDataset 2 Feature Ex and Red 3 4 Train for CGAN Train for SRGAN Present result 5 6
  • 16.
    1616 LR image HR image generator Highquality fake or real Dataset Proposed architecture workflow Xr Y Y Z Generator Discriminator Generator Discriminator D D A Generator(An artist) neural network. •Discriminator has to correctly label real images which are coming from training data set as “real”.
  • 17.
    1717 Cont...d CGAN for generatingcondition fashion images SRGAN to Refine the quality from generated one Nobility  Enhancement of Quality image  control fine-grained style  Improve to work with multiple color
  • 18.
    1818 Research tools Software tools Tensor-flow Keras MSoffice Mendeley desktop  Colab Hardware tools  GPU RAM SSD Camera
  • 19.
    1919 Contribution and applicationof result Collecting dataset and pull to github Creating of fusion algorithm CGAN and SRGAN Enhance quality generated image(SRGAN) and Control fine grained style (CGAN) Application of result Style transfer across multiple domain image generation Image to image translation :learn mapping input and output photo editing and image enhancement
  • 20.
    2020 Budget plan (operationcost) Operation cost Item No Item Name Unit price Quantity * unit Total Source 1 Data collection 2000 3*2000 6000 Coveredby Researchfund 2 Data preparation 1500 3*1500 4500 3 Print cost and binding 650 4*650 2600 3 Transport cost 300 6*300 1800 5 Contingency plan (10%) 10% 1*2500 2500 Total operation cost 14700 Total material cost 7800 Total cost operation +material _contingency =25,000 ETB Material cost Item no Item name Specification Unit price Qty. Total 1 GPU MSI RTX 2070 250$ 1 - 2 Flash USB SanDisk 32GB 10$ 1 - Total 210 USD =7800 ETB
  • 21.
    2121 July 2019 daysJul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Literature review 334 Writing Proposal 99 Proposal defense - Data collection 120 Data preparation 50 Train model 60 Verify the model 90 Result Report and summarization 90 TodayTodayCompleteComplete IncompleteIncomplete Not StartedNot Started 2020 June 2020 Months Research timeline and schedule
  • 22.