1. Previous work on green screen keying required controlled illumination and manual guidance, which is not feasible for real-time applications. 2. Recent approaches have used deep learning models but these struggle to achieve both high robustness and pixel-wise accuracy. 3. The proposed method in the paper uses a deeply guided linear model that first performs segmentation using a robust CNN, then applies a linear model to refine the segmentation, achieving both high robustness and accuracy for real-time green screen keying without specialized lighting or manual input.

1. A PROJECT REPORT ON
“Realtime Video Processing using
ChromaKey (GreenScreen)
Effect”
SUBMITTED IN THE PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THEAWARD OF THE DEGREE
BACHELOR OF ENGINEERING
(Information Technology)
BY
Gouri Hadgekar Exam No:72164049E
Saurabh Jagdale Exam No:72025164J
Samruddhi Ghatge Exam No:72025431M
Tanaya Kadam Exam No:72164051G
Department Of Information Technology
PES’s Modern College of Engineering, Pune
2022 -2023

2. CERTIFICATE
This is to certify that the Project Entitled
”Realtime Video Processing using ChromaKey (GreenScreen)
Effect”
Submitted by
Gouri Hadgekar Exam No:72164049E
Saurabh Jagdale Exam No:72025164J
Samruddhi Ghatge Exam No:72025431M
Tanaya Kadam Exam No:72164051G
is a bonafide work carried out by them under the supervision of Prof. Yogita
Fatangare and it is approved for the partial fulfillment of the requirement of
Savtribai Phule Pune university, Pune for the award of the degree of Bachelor
of En- gineering (Information Technology).
Guide Name H.O.D. Name Principal Name
Prof.Yogita Fatangare Prof.S.S.Deshpande Prof.K.R.Joshi
Place:- Pune
Date:-

3. Acknowledgments
Guidance is the key factor for achieving the goals. No work can be com- pleted
without correct help from experts, in this field. At this point I wish to express my
gratitude towards those who have made our project successful.
It is my great pleasure to acknowledge sense of gratitude to all, who made it possible
for me to complete this project with success. It gives me great pleasure to thank my
project guide, Prof. Yogita Fatangre for great support guidance and
encouragement from time to time during project work.
It is my sincere thanks to Prof. S.S.Deshpande (HOD Information Technology)
for their help and guidance, Also I would like to express my deep gratitude
towards all the teaching staff for their guidance during project presentations. I also
extends my sincere thanks to our beloved Principal K.R.Joshi for their support
and guidance.

4. ABSTRACT
We propose a method for creating a matte – the per-pixel foreground color
and alpha – of a person by taking photos or videos in an everyday setting
with a handheld camera. Most existing matting methods require a green screen
background or a manually created tree map to produce a good matte. Automatic,
tree map-free methods are appearing, but are not of comparablequality. In our tree
map free approach, we ask the user to take an additional photo of the background
without the subject at the time of capture.
This step requires a small amount of foresight but is far less time consuming
than creating a tree map. We train a deep network with an adversarial loss to
predict the matte. We first train a matting network with supervised loss on ground
truth data with synthetic composites. To bridge the domain gap to real imagery
with no labeling, we train another matting network guided by the first network
and by a discriminator that judges the quality of composites.We demonstrate
results on a wide variety of photos and videos and show significant improvement
over the state of the art.
Keywords : Green Screening, Neural Network, Deep Learning etc.

5. Contents
1 Introduction 1
1.1 Overview................................................................................................ 2
1.2 Motivation of the Project...................................................................... 2
1.3 Aim and Objective............................................................................2
1.4 Applications………………………………………………………..2
2 Literature Survey 3
3 Problem Statement Definition 8
3.1 Problem Statement Definition ..........................................................9
4 Software requirement specification 10
4.1 Assumption and Dependencies........................................................... 11
4.2 Functional Requirements..................................................................... 11
4.3 External interface requirements .......................................................... 11
4.3.1 user interface........................................................................... 11
4.4 Hardware Interfaces ............................................................................ 11
4.5 Software Interfaces.............................................................................. 12
4.6 Nonfunctional Interfaces ..................................................................... 12
4.6.1 Performance Requirements..................................................... 12
4.6.2 Safety requirements ................................................................ 12
4.6.3 Software Quality Attributes................................................12
4.7 System requirements ........................................................................... 12
4.7.1 Database requirements............................................................ 12
4.8 Software Requirements ....................................................................... 13
4.9 Hardware requirements ....................................................................... 13
4.10 Analysis Module. SDLC Module to be applied ...........................13
5 Flowchart 15
5.1 Flowchart of Project .......................................................................16
6 Project Requirements Specification 17
7 Proposed System Architecture 20
7.1 System Architecture........................................................................21

6. 8 System Design 22
8.1 Data Flow Diagram.........................................................................23
8.2 UML DIAGRAMS ...............................................................24
9 Project Implementation 28
9.1 Overview of Project Modules:............................................................ 29
9.2 Tools And Technologies used........................................................29
9.3 Algorithm ........................................................................................29
10 Test cases 31
10.1 Test cases............................................................................................. 32
10.2 Test Cases and Test Results...........................................................35
11 Project Plan 36
11.1 System Implementation Plan..........................................................37
12 Conclusion 38
12.1 Conclusion ........................................................................................... 39
13 References 40
Annexure A Appendix A 43
Annexure B Appendix B 45
Annexure C Appendix C 47
Annexure D Appendix D 50

7. List of Figures
4.1 SDLC Model Diagram................................................................... 14
5.1 Flowchart ............................................................................................. 16
7.1 system Architecture............................................................................. 21
8.1 Data Flow diagram......................................................................... 23
8.2 Data Flow diagram......................................................................... 23
8.3 Data Flow diagram......................................................................... 24
8.4 Class Diagram……………………………………………………..29
8.5 Use case Diagram............................................................................ 25
8.6 Activity Diagram ........................................................................... 26
8.7 Sequence Diagram.............................................................................. 27
10.1 GUI TESTING ................................................................... 32
10.2 Login test case.................................................................................... 33
10.3 Registration test case....................................................................... 3

8. CHAPTER 1
INTRODUCTION

9. 1.1 Overview
Have you ever watched a superhero movie where the hero soars through theair?
It looks so cool to see all the skyscrapers zooming by in the background.Have
you ever Questioned how such shots are filmed in movies?
A special tool known as a green screen is used to create the special effects
used in many television shows and movies, as well as weather fore- casts. What
makes it a ”green screen”? primarily because it has a sizable green screen! The
special effects procedure formally referred to as chromakey includes the green
screen as a crucial component.
Chromakeying, also referred to as colour keying, is the process of iden-
tifying a specific colour in an image and making it transparent with the aid of
computer software.
1.2 Motivation of the Project
In order to provide a real-time composited video stream and eliminate a step
between recording and video production, our motivation for our final projectis
to use our Neural Network and OpenCV of Chroma Key Compositing.
1.3 Aim and Objective
Aim of project to study on Realtime Image Processing using ChromaKey
(GreenScreen) Effect.
Objective is to give them ability to be expressive in ideas and thoughts.
You will be creating a web application which will perform real time video
processing of a webcam video stream consisting of a green screen and replaceit
with a background an image
1.4 Applications
Some of the applications of Green Screen Video Processing are:-
1. Personalized background for video meeting applications like Zoom
2. An important technique utilized for heavy computer graphics in film
industry.

10. CHAPTER 2
LITERATURE SURVEY

11. Sr.
No
Title Year of
Publication
Author Methodology
1. Automatic,
Illumination-Invariant
and Real-Time Green-
Screen Keying Using
Deeply Guided Linear
Models
2021 Hanxi Li 1,2,* ,
Wenyu Zhu 1,
Haiqiang Jin 2 and
YongMa 1
Without controlled
illumination and effective
guidance by humans, one
firstly needs a highly robust
segmentation algorithm to
distinguish background and
foreground. Motivated by the
success of deep learning.
However, as we explain later,
the robust CNN model can
hardly achieve high
robustness and high pixelwise
accuracy simultaneously,
especially when the time
budget is limited.
2. Background Matting:
The World is Your
Green Screen
2020 Soumyadip
Sengupta, Vivek
Jayaram, Brian
Curless, Steve
Seitz, and Ira
Kemelmacher-
Shlizerman.
Traditional approaches.,
Learning-based approaches.,
Matting with known natural
background., Video Matting,
Deep Network.
3. Chroma key
compositing with
FPGA
2022 Daniel Moon and
Thipok Ben Rak-
amnouykit
Our idea is that we would
segment the system into
sensors, memory, and the
algorithm that does the proper
chroma keying.
4. Real-Time High-
Resolution
Background Matting.
2020 Shanchuan Lin,
Andrey Ryabtsev.
We introduce a real-time,
high-resolution background
replace- ment technique
which operates at 30fps in 4K
resolution. Our technique is
based on background matting.
We introduce two largescale
video and image matting
datasets: Our approach yields
higher quality results
compared to the previous
state-of-the-art in background
matting.

12. CHAPTER 3
PROBLEM STATEMENT
DEFINITION

13. 3.1 Problem Statement Definition
• We propose a fully automatic and real-time green screen keying
system for unstructured applications like panels with ambient light,
shadows, and markson them.
• The robust CNN model is unable to achieve great robustness and
acceptable pixel wise precision when time is limited.

14. CHAPTER 4
SOFTWARE REQUIREMENT
SPECIFICATION

15. 4.1 Introduction
4.1.1 Purpose
Shooting with a green screening involves filming a person a person or adding visual
effects in front of a solid color. Then by digitally removing or “keying out” that color,
you can drop that scene onto the background of your choice in the post-production.
Removing the colored background is also referred to as “chroma keying”
4.1.2 Scope
Green screening mission is to inspire and educate the nomadic world of filming by
creating sustainable working practices. Which allows film production to remain
within a sound stage but depict disparate locations and sequences.
4.2 Assumption and Dependencies
• Space of screen, lighting, the camera and post production affects green screen
editing.
• User has to login to the system to enter the system for editing green screen image
or video.
• Green Screening is the actual technique of layering or composing two images based
on color hue which makes editing and processing complex.
4.3 Functional Requirements
4.3.1 System Feature
• This tool allows you to remove the green color background from video.
• It has green screen wizard that helps you to easily navigate 3d visuals.
• You can easily edit videos
• You can add custom texts, colors, emojis, etc.
4.4 External interface requirements
4.4.1 user interface
• Front-End: Tkinter framework of python
• Backend: Python

16. 4.5 Hardware Interfaces
• Since the application must run over the Internet, all the hardware
shall require to connect Internet will be hardware interface for the
system. As for
• e.g. WAN LAN, Ethernet Cross-Cable
4.6 Software Interfaces
Python:
It is a programming language we are using for both frontend and
backend.
Spyder:
It is an open source cross platform integrated development
environment for python development
DB SQLite:
SQLite is self-contained, file-based SQL database used with python
applications without having to install any additional software.
4.7 Nonfunctional Interfaces
4.7.1 Performance Requirements
The performance of the functions and every module must be well. The
overall performance of the software will enable the users to work efficiently.
Login should be done efficiently. green screen file should be uploaded properly.
Processing of image or video file should be effective.
4.7.2 Safety requirements
The application is designed in modules where errors can be detected.
This makes it easier to install and update new functionality if required.
4.7.3 Software Quality Attributes
Our software has many quality attributes that are given below: -
• Adaptability: This software is adaptable by all users.
• Availability: This software is freely available to all users.
The a
v
a
i
l
a
b
i
l
i
t
yof the software is easy for everyone.
• Maintainability: After the deployment of the project if any error
occursthen it can be easily maintained by the software developer.
• Reliability: The performance of the software is better which
will increasethe reliability of the Software.

17. 4.8 System requirements
4.8.1 Database requirements
• DB Browser SQLite:
SQLite is self-contained, file-based SQL database used with python applications without
having to install any additional software.
4.9 Software Requirements
• Operating System: Windows 10
• IDE: PyCharm, Spyder
• Programming Language: Python
• Software: Anaconda
4.10 Hardware requirements
• Hardware : intel core
• Speed : 2.80 GHz
• RAM : 8GB
• Hard Disk : 500 GB
• Key Board: Standard Windows Keyboard
4.11 Analysis Module. SDLC Module to be applied
Waterfall Model is a sequential model that divides software development
into different phases. Each phase is designed for performing specific activity
during SDLC phase. It was introduced in 1970 by Winston Royce.
This is used for our project. This model is simple and easy to understand
and use. It is easy to manage due to the rigidity of the model each phase has
specificdeliverables and a review process. Waterfall model works well for
smaller projects where requirements are clearly defined and very well
understood.

18. Figure 4.1: SDLC Model Diagram
4.11.1 Overview of Responsibilities of Developer
• To have understanding of the problem statement.
• To know what are the hardware and software
requirements of proposed system
• Performing research and creating plans that best suits the
project needs.
• To do planning various activates with the help of planner.
• Designing, programming, testing etc.
• Creating and constructing computer programs.
• Verify and deploy programs and system.
• Gather and evaluate user feedback

19. CHAPTER 5
FLOWCHART

20. 5.1 Flowchart of Project
Figure 5.1: Flowchart

21. CHAPTER 6
PROJECT REQUIREMENT
SPECIFICATION

22. Anaconda Navigator:
Anaconda is an open-source distribution of the Python and R programming
languages for data science that aims to simplify package management and
deployment. Package versions in Anaconda are managed by the package
management system, conda, which analyzes the current environment before
executing an installation to avoid disrupting other frameworks and packages.
Spyder:
Spyder is a free and open source scientific environment written in Python,
for Python, and designed by and for scientists, engineers and data analysts.It
features a unique combination of the advanced editing, analysis, debug-ging,
and profiling functionality of a comprehensive development tool withthe data
exploration, interactive execution, deep inspection, and beautiful visualization
capabilities of a scientific package.
DBsqlite3:
DB Browser for SQLite (DB4S) is a high quality, visual, open source
tool tocreate, design, and edit database files compatible with SQLite.
DB4S is for users and developers who want to create, search, and edit
databases. DB4S uses a familiar spreadsheet-like interface, and complicated SQL
commands do not have to be learned.
Controls and wizards are available for users to:Create
and compact database files
Create, define, modify and delete tables
Create, define, and delete indexes
Browse, edit, add, and delete records Search records
Import and export records as text.
Import and export tables from/to CSV files
Import and export databases from/to SQL dump filesIssue
SQL queries and inspect the results
Examine a log of all SQL commands issued by the application
Plot simple graphs based on table or query data
Python Language
Python is an easy to learn, powerful programming language. It has efficient
high-level data structures and a simple but effective approach to object-
oriented programming. Python’s elegant syntax and dynamic typing, together
with its interpreted nature, make it an ideal language for scripting and rapid
application development in many areas on most platforms.
The Python Language Reference gives a more formal definition of the
language. To write extensions in C or C++, read Extending andEmbedding
the Python Interpreter and Python/C API Reference Manual. There are also
several books covering Python in depth.

23. CHAPTER 7
PROPOSED SYSTEM
ARCHITECTURE

24. 7.1 System Architecture
Figure 7.1: system Architecture
Pre-Processing:
In this process we only reduce the size of uploaded image which is input from the user
Feature extraction:
This is the process where we extract the features like brightness, size, height, width and other
features from the given imported image. After pre-processing image will get converted into RGB
to black and white and by using gray to binary function of conversion of image into binary.
CNN:
Cnn is applied here by using HSV function which use to convert black value from unnecessary
background which is we have to remove from the image and add new background image for
realistic result.
Here, we are going to develop 4 modules which consists login module, registration module,
importing input and last module is output module.

25. CHAPTER 8
SYSTEM DESIGN

26. 8.1 Data Flow Diagram
In Data Flow Diagram,we Show that flow of data in our system in DFD0 we
show that base DFD in which rectangle present input as well as output and
circle show our system,In DFD1 we show actual input and actual output of
system input of our system is text or image and output is rumor detected like
wise in DFD 2 we present operation of user as well as admin.
Figure 8.1: Data Flow diagram
Figure 8.2: Data Flow diagram

27. Figure 8.3: Data Flow diagram
8.2 UML DIAGRAMS
Unified Modeling Language is a standard language for writing software blueprints.The
UML may be used to visualize,specify,construct and document the arti-
facts of a software intensive system.UML is process independent,although
optimally it should be used in process that is use case driven,architecture-
centric,iterative,and incremental.The Number of UML Diagram is available.

28. 8.4 – Class Diagram

29. Figure 8.5: Use case Diagram

30. Figure 8.6: Activity Diagram

31. Figure 8.7: Sequence Diagram

32. CHAPTER 9
PROJECT IMPLEMENTATIO

33. 9.1 Overview of Project Modules:
Pandas: Pandas is an open-source library that is made mainly for work-
ing with relational or labeled data both easily and intuitively. It provides
various data structures and operations for manipulating numerical data and
time series. This library is built on top of the NumPy library.
NumPy: NumPy is a Python library used for working with arrays. It
also has functions for working in domain of linear algebra, fourier transform,
and matrices.
import cv2:All packages contain Haar cascade files. cv2.data.haarcascades can
be used as a shortcut to the data folder.
Pillow:Pillow is the friendly PIL fork by Alex Clark and Contributors.
PILis the Python Imaging Library by Fredrik Lundh and Contributors.
9.2 Tools And Technologies used
Hardware Requirements:
• System Processors : Core2Duo
• Speed : 2.4 GHz
• Hard Disk : 150 GB
Software Requirements:
• Operating system : 32bit Windows 7 and on words
• Coding Language : Python
• IDE : visual studio,Eclipse
• Database : Sqlite
9.3 Algorithm
Convolutional Neural Networks:
These are used with an untrained CNN, which implies that every pixel of ev-
ery feature and every weight in every fully linked layer is randomly assigned.

34. Then, one by one, we start feeding images through it. CNN receives a vote
for each image it processes.
Step 1: Convolution. Step 1b: ReLU Layer.
Step 2: Pooling.
Step 3: Flattening.
Input image (starting point) Convolutional layer (convolution operation)
Pooling layer (pooling) Input layer for the artificial neural network (flat-
tening)Because of its great accuracy, CNNs are employed for picture cate-
gorization and recognition. The CNN uses a hierarchical model that builds
a network, similar to a funnel, and then outputs a fully-connected layer in
which all neurons are connected to each other and the output is processed.

35. CHAPTER 10
TEST CASES

36. 10.1 Test cases
Figure 10.1: GUI TESTING

37. Figure 10.2: Login test case

38. Figure 10.3: Registration test case

39. 10.2 Test Cases and Test Results
Test case : Login Screen- Sign up
Objective : Click on sign up button then check all required/ mandatory fields
with leaving all fields blank
Expected Result : All required/ mandatory fields should display with symbol
“*”. Instruction line “* field(s) are mandatory” should be displayed
Test case : Create a Password -Text Box Confirm Password -Text Box
Objective : Check the validation message for Password and Confirm Pass-
word field
Expected Result : Correct validation message should be displayed accord-
ingly or “Password and confirm password should be same” in place of “Pass-
word mismatch”.

40. CHAPTER 11
PROJECT PLAN

41. In this chapter we are going to have an overview about how much time does
it took to complete each task like- Preliminray Survey Introduction and Prob-
lem Statement, Literature Survey, Project Statement, Software Requirement
and Specification, System Design, Partial Report Submission, Architecture
Design, Implementation, Deployment, Testing, Paper Publish, Report Sub-
mission and etcetera. This chapter also gives focus on stakeholder list which
gives information about project type, customer of the proposed system, user and
project member who developed the system.
11.1 System Implementation Plan
The System Implementation plan table, shows the overall schedule of tasks
compilation and time duration required for each task.
Sr. No. Name/Title Start Date End Date
1 Preliminary Survey 20/08/2022 27/08/2022
2 Introduction and Problem
Statement
29/08/2022 04/09/2022
3 Literature Survey 06/09/2022 17/09/2022
4 Project Statement 20/09/2022 22/09/2022
5 Software Requirement And
Specification
25/09/2022 30/09/2022
6 System Design 04/10/2022 10/10/2022
7 Partial Report Submission 01/11/2022 08/11/2022
8 Architecture Design
9 Implementation
10 Deployement
11 Testing
12 Paper Publish
13 Report Submission

42. CHAPTER 12
CONCLUSION

43. 12.1 Conclusion
We developed a novel method for achieving automatic illumination-invariant
and real-time keying on green screens in this paper. Linear models and deep
learning outcomes were cleverly coupled to produce reliable matting resultsin
near-real time. A fresh green screen dataset was also created, with more
foreground variants and more difficult backdrops. To our knowledge, this is
the first algorithm that can do AIR keying, and the suggested dataset is also
the first green screen dataset that has been seen in the wild.
In future work,In the future, our work will focus on improving the quality of
the coarse output of the offline-trained CNN, which is very important to us.
In addition, we will apply our proposed approach to a higher image resolu-
tion and more complex scenes to verify its effectiveness.

44. CHAPTER 13
REFERENCES

45. 1 Yagiz Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, ˘and Wo-
jciech Matusik. Semantic soft segmentation. ACM Transactions on
Graphics (TOG), 37(4):72, 2018.
2 Yagiz Aksoy, Tunc Ozan Aydin, and Marc Pollefeys. Designing
effective inter-pixel information flow for natural image matting. In
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 29–37, 2017.
3 Shaofan Cai, Xiaoshuai Zhang, Haoqiang Fan, Haibin Huang,
Jiangyu Liu, Jiaming Liu, Jiaying Liu, Jue Wang, and Jian Sun.
Disentangled image matting. International Conference on Computer
Vision (ICCV), 2019.
4 Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff,
and Hartwig Adam. Encoder-decoder with atrous separable convolu-
tion for semantic image segmentation. In ECCV, 2018.
5 Quan Chen, Tiezheng Ge, Yanyu Xu, Zhiqiang Zhang, Xinxin Yang,
and Kun Gai. Semantic human matting. In 2018 ACM Multimedia
Conference on Multimedia Conference, pages 618–626. ACM, 2018.
6 Qifeng Chen, Dingzeyu Li, and Chi-Keung Tang. Knn matting. IEEE
transactions on pattern analysis and machine intelligence, 35(9):2175–2188,
2013.
7 Donghyeon Cho, Yu-Wing Tai, and Inso Kweon. Natural image mat-
ting using deep convolutional neural networks. In European Conference on
Computer Vision, pages 626–643. Springer, 2016.
8 Yung-Yu Chuang, Aseem Agarwala, Brian Curless, David H Salesin,
and Richard Szeliski. Video matting of complex scenes. In ACM Trans-
actions on Graphics (ToG), volume 21, pages 243–248. ACM, 2002.
9 Yung-Yu Chuang, Brian Curless, David H Salesin, and Richard
Szeliski.
A bayesian approach to digital matting. In CVPR (2), 2001.