The document proposes a design for a smart refrigerator that uses additional sensors and modules to identify food items inside, monitor their freshness, track quantities, and provide notifications. The key components proposed include a movable camera that can image all areas of the refrigerator shelves, gas sensors to detect food decay, and an app to control settings and view fridge contents and notifications. The intelligent fridge would use image recognition on photos to identify and count items, monitor freshness with gas sensors, and alert users when food needs replacing through a mobile app.

1. Smart Refrigerator
Team GAMECHANGERS
DIGIRUPT

2. Smart Refrigerator
● In the age of Smart Home appliances, it
becomes a necessity to own a refrigerator
which should be smart enough to take actions
based on the items available in it. With this
idea, we are trying to present a design model of
such a refrigerator . So, the slides will discuss
about only the new components that we are
trying to bring inside the fridge, and not the
existing ones.

3. Idea of design
● The idea is to suggest a design of the fridge in
addition to the already existing design and
components of the refrigerator. While the
temperature sensors, the humidity sensors,etc.
already exist , we have come out with some
extra modules inside the fridge which will play
good part in making the refrigerator one of the
smartest home appliances.

4. Components of the design
● Camera module
● Load cell sensors (optional)
● Food freshness by combining different Gas-detection sensors
● A mobile app to -
● Feed the input to the controlling system (One-time task)
● Check the notifications when needed
● Control the temperature when needed
● See the details of the items present
● Get a peek view of the fridge
● Enable/disable some features/functionalities

5. Components of design- I [Camera]
● A moveable lens which can traverse along the
edges of the fridge upto the corners of each
and every shelf ,i.e., all 8 corners in each
cuboid shelf
● This lens will click the pictures of the objects for further
processing it to identify the objects with their
colors,shapes,sizes and numbers. This will help us to
fetch the identity of the items and their quantity inside the
shelf.

6. Design
Traversal path of lens
on front side of the
fridge
Traversal path of lens
inside the fridge
Shelf of the fridge
Lens
door

7. Components of design- II [Load
sensors] (optional)
● We can also have load cell sensors on each and every shelf to
determine the weights of the items.However, the challenges are-
● Each shelf can have multiple variety of items.So, inspite of finding the identity of the item
with the camera, the determination of the weight of that particular item will be tough.
● One solution may be of pre-specifying each and every shelf with some sets of blocks to
place only particular type of items. By doing this, we can know the weight of a particular
type of item inside each shelf. Each and every such block can have load cell sensors
underneath it to measure weight of the items placed.diagram on next slide.
● We can skip this idea in order to remove the overhead of putting item-
specific blocks. If we skip the use of load-cell sensors, the only way to
determine the quantity of the items will be through the object detection
algorithm of the camera module. However, we will not be able to
determine the weight.

8. Blocks in each shelf
Shelf No. 1
Green
vegetables
Cooked
items
Fruits
Water
Bottles Soft
drinks
Ice-cream
Shelf No. 2
Fruits
Green vegetables
Water Bottles Cooked
items
Sweets Milk
Since such a design of the shelf may not be a feasible approach for
a fridge, we can skip the use of load-cell sensors, and do our task
with the camera module.

9. Components of design- III [Food
freshness]
● Sensors to identify the presence of some
gases, which are known to be emitted with the
decay of some of the food items.
● amines (from the decomposition of protein)
● methane (from anaerobic decomposition)
● hydrogen sulphide (from protein decomposition)
● Ethylene( ripening/rotting)
● Carbon-dioxide
● dust

10. Food freshness sensors
Amine sensor CO2 sensor Dust sensor
Ethylene sensor H2S sensor
Master Sensor module

11. Placement of sensors on each shelf
door
SensorSensor
Sensor
Sensor
Master sensor module
equipped with all the
Gas detection
sensors
Sensor

12. Movement of lens

13. Movement of lens

14. Movement of lens

15. Movement of lens

16. Movement of lens

17. Movement of lens

18. Movement of lens

19. Movement of lens

20. Movement of lens

21. Movement of lens

22. Movement of lens

23. Movement of lens

24. Movement of lens

25. Movement of lens

26. Movement of lens

27. Movement of lens

28. Movement of lens
● In this way, the whole refrigerator can be traversed
by a single camera lens,hence avoiding the use of
multiple fixed lens.
● Making the lens go to each and every corner will
help in getting more accurate image processing
out of multiple images from multiple angles.
● With more and more inputs, the camera module
will get trained over a period of time to give out the
perfect result of object identification and counting.

29. Mobile app to control actions
Home screen
myFridge
Go inside
Set favorite items Set quantity threshold
Control temperature Place a new item
Set purchase rules Set meal timings
Freshness of items Quantity of items
Wiki of items Get recipe suggestions
Check for notifications
My diet details
Peek inside myFridge
Disable some features

30. Set favorite items
carrots
fish
paneer
Water
Watermelons
cauliflower

31. Set quantity threshold
carrots
Milk bottles/packs
Water bottles
cauliflower
Ice-cream
lady's finger
Items Threshold value (nos.)
5
3
3
3
2
12

32. Place a new item not known to the
fridge
● Suppose we want to place lemons which is not
known to the fridge.
● Take pictures of lemons from different angles in
the phone app and tag with the name “Lemon”.
● Next time when lemons are placed , it will not
be new to the fridge.

33. Place a new item
Place a new item
Phone camera
Phone
camera
Phone
camera
Lemons
Upload to fridge database

34. Set purchase rules
Set purchase rules
When the quantity of my favorite items
reaches the threshold ,place purchase
order from BigBazar :
Carrots 1 kg
Cauliflower 2 nos.
Milk 1 litre
Lady's fingers 2 kg

35. Set meal timings and details
Breakfast 8:30 am
time Items to be
consumed
Milk
Apples
Oranges
Lunch 1:00 pm Nothing from
fridge
dinner 8:00 pm Milk
cauliflower
paneer
This feature will be used to regulate
the temperature inside the
fridge,e.g., milk is to be consumed
at 8:30 am, so the fridge will stop
cooling it 10 mins before that
Benefits:
Energy saving
Less waiting time for boiling the
milk
Notify the user of their meal timings

36. Freshness of items
Freshness level
Carrots
Milk
Fish
Bananas
Fresh
May start spoiling-
kept for the last 3 days
Started spoiling
May get spoiled due to
fish

37. Quantity of items
Quantity of currently placed items
Carrots
Fish
Bananas
Cauliflowers
Water Bottles
Milk
Bottles/packs
5
5
10
3
3
3

38. Get recipe suggestion
Suggest me some recipe
Message from fridge
Currently you have decent amount
of carrots, spinach and Paneer in
your fridge.
Try out :
Palak Paneer and Gaajar Ka
Halwa with Butter Naan.
Click here for the complete recipe.

39. Wiki of items
Want to know more about the items
in your fridge? Click on the items
You have :
Cauliflowers
Lady's fingers
Bananas
Carrots
Cauliflowers

40. My diet details
Food consumed in the last 7 days
Milk
Carrots
Paneer
Water
Cauliflowers
14 litres
5 kg
3 kg
28 bottles
5 kg
Fat
Proteins
Minerals
Sugar
Fiber
Vitamin K
Vitamin B6
Vitamin A
Vitamin C
Contents present Contents absent
Starch
Calcium
Vitamin B12

41. My diet details
Suggested items for a healthy diet
You may prefer one or more of
the following enriched with :
Calcium
Vitamin B12
Sesame Seeds,Cream
Cheese,Spinach
Eggs

42. Check for notifications
NOTIFICATIONS ACTIONS
No. of Water Bottles < 3
No. of Milk packets < 3
Fish - started decaying
No. of Carrots < 6
Paneer – kept in the
fridge for 6 days
Purchase req sent
Purchase req sent
Please remove it &
purchase req sent
Purchase req to be
sent when < 5
Use it fast. Storing it
for long may not be
a healthy idea

43. Peek inside myFridge
myFridge
Shelf no. 1
Shelf no. 2
Shelf no. 3
AR can be
introduced here.
User can click on
any of the items, and
thus can read the
wiki page of that
particular item,learn
about some new
recipe using that
item,place purchase
order for that
item,etc.

44. Enable/disable some features
Functionality control menu
Enable/disable auto-purchase options
Enable/disable meal timing notification
and self-temperature control option

45. Sample flow of functioning
Freshness
Sensor
Camera
Main Controller
decay of
fruits?
No
No
action
Quantity
goes below
threshold?
No
Yes
Yes
Notification
triggered
Mobile app
Bowl of fruits

46. Main Controller
● The central controller can be any of the high-
end Intel-powered boards like-
● Intel Galileo
● Intel Edison
● Arduino UNO,Raspberry Pi,etc.
● The camera module has to be supported with
illumination inside the fridge. A night-vision
camera can be a useful option.

47. Further scope
● The fridge can also be made intelligent enough to learn over a
period of time the weight of any item.
● To do this, a weight-sensor will be placed on the shelf.
Whenever, the user takes out some items,(say 2 carrots or a
bottle of milk) out of the fridge, the difference in the total weight
of the shelf will be noted down, and combining this information
with the camera's object counting algorithm, the fridge can know
what could have been the weight of those 2 carrots and a bottle
of milk.
● Over a period of time when the module gets trained, the fridge
will be intelligent enough to approximately tell the weight of the
item ,given its size,shape and identity.

48. Further scope
● In the era of Internet of Things, it's hard to
imagine that one thing can't talk to the other thing.
● In case of fridge, it may talk to a water purifier to
notify it that the number of water bottles has gone
below threshold, and probably some water is
needed in the fridge to be cooled. It can also talk
to a microwave oven during meal timings. With
such use cases, many applications can be
developed.

49. Queries
?

50. THANKS