1. Department of Electrical Engineering,
University of Engineering and Technology, Lahore
Home Assistant Robot with
Artificial Intelligence
Group No: 40
Project Advisor: Dr. Kashif Javed
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
2. Team Introduction
Hamid Nasir (Team Leader)
2014-EE-177
Specialization: Power
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
Muhammad Shazil Majeed
2014-EE-181
Specialization: Computer
Abdul Basit
2014-EE-168
Specialization: Computer
Nabeel Raza
2014-EE-164
Specialization: Computer
3. Problem Statement
• Usually housewives need help in bringing kitchen
utensils while cooking or serving food etc. There
should be an assistant robot that can search and fetch an
object which the owner requested.
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
4. Proposed Solution
Recognizing speech and identifying words.
Searching the requested object in the surrounding via
camera.
Segmenting the object from an image.
Triggering the robot to follow a path towards that object
Guiding the robotic arm to pick object
Returning back to the person along with the object
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
6. Speech Recognition
• Sample Collection
• Feature Extraction
• Classification
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
7. Feature Extraction
• Converting Recorded Sound in time domain samples
• Fourier Analysis of Recorded Sound Waves
• Fragmentation
• Mel-Frequency Cepstrum Coefficients
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
8. MFCC
• Mel-Frequency Cepstral Components
• Human Compatible Scale
• Mel Scale And Linear Scale
• Fourier Transform
• Map Power Spectrum to Mel Scale using Triangular
Overlapping Windows
• Cosine Transforms
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
9. Classification
• The resulting feature vectors are arranged in a matrix.
• The classification Problem can be handled by using
SVM or ANN
• SVM
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
10. Object Detection & Segmentation
• Image Pre-processing
• Feature Extraction (Inception Model)
• Output (Classification / Segmentation)
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
12. Training Deep learning model
• We trained DL model without large dataset and GPUs
• Data Augmentation
• Transfer Learning
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
14. Structure
• 2 cylindrical aluminum sheets
• 3 tires – 2 for driving and 1 for direction
• Three Floors
• Cylindrical acrylic sheet on the top of robot.
• 2 rectangular aluminum sheets for arms support.
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
15. Hardware
• Raspberrypi 3, Touchscreen LCD, arduino, USB
microphone, camera
• Two robotic arms
• Buck converter
• Two 12V DC motors , one servo for front tyre
• Two 12V batteries
• Monster moto driver for speed control
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
16. Applications
• Assistant for old people who can’t walk and bring
something themselves
• Assistant for patients
• Assistant for housewives who need a helping hand to
bring kitchen utensils
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
17. Audience
• Housewives, old, patients
• Can be used at home, hospital etc.
• Disabled persons
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
18. What we have done so far?
• Speech recognition
• Object Detection
• Hardware Assembly
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
19. Hardware Developed
• Structure and Body
• Movement Mechanism
• Robotic Arm
• Sound Input from Microphone
• Image Input using Camera
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
20.
21. Future Deliverables
• More Objects
• Face Recognition to come back to the same person
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
22. References
[1] Bhadragiri Jagan Mohan and Ramesh Babu N.,
“Speech recognition using MFCC and DTW,” in 2014
International Conference on Advances in Electrical
Engineering (ICAEE), 2014, pp. 1–4.
[2] C. Szegedy et al., “Going deeper with convolutions.
Undergraduate Final Year Project Presentation
Dated: 31
st
March, 2017
