Final Project Report

Automated Chocolate Sorting Process
Report
ROBT 4491
Vasiliy Baryshnikov Cesar Moreno
May 6, 2016

ii
Acknowledgement
We are thanking Pavlos Paleologue for providing us with the Robotic Arm (CRS 255), conveyor belt,
and a microcontroller (Launchpad MSP430). His advice in what we should focus on to achieve our
objectives was relevant and saved our team weeks of project time.
Many thanks to Chris Baitz for taking the time to study our project and providing us with important
feedback and guidelines, without which the project would not have been successful.
We thank Matt Rockall for his advice on project planning and documentation.
Also, we are thankful to Trevor VanLaerhoven and Som Sanjari for writing a very explicit report in
2015 about a similar project “Automated Packaging Process”, which prevented us from making the
same mistakes and allowed us to identify the relevant aspects to focus on.

iii
Abstract
The goal of this project is to completely automate the process of packaging chocolates. Purdy’s
Chocolates—one of Canada’s biggest chocolatiers—uses human labor to package their chocolates.
This method is both inefficient and expensive because it is a monotonous task for a human employee.
The proposed solution is to package the chocolates using a CRS A255 robotic arm, a Cognex Insight
5000 camera, a conveyor belt, and a microcontroller. The system communicates through the central
computer, which coordinates all the actions by communicating with all of the peripherals.
The process itself is able to sort between good and bad parts by only sending a valid part’s location to
the arm through the main VB.NET program. Consequently, the arm positions itself on the correct
location and picks up the part from the moving conveyer with the help of a rotary encoder. The
approach used for this project is modular in concept and can be expanded by adding more arms down
the packaging line in order to match the production rate at the factory.
By automating this process the efficiency and long term cost at a packaging line is greatly reduced.
The following document outlines the project’s operation, design, and implementation.

iv
Table of Contents
Acknowledgement...................................................................................................................ii
Abstract ................................................................................................................................iii
Table of Contents...................................................................................................................iv
List of Figures........................................................................................................................ v
Introduction .......................................................................................................................... 1
Project Scope/Objectives........................................................................................................ 1
Original Objective................................................................................................................ 1
Objectives Met .................................................................................................................... 1
Project Description ................................................................................................................ 2
Hardware Implementation ................................................................................................. 2
Robotic Arm CRS A255 and C500C Controller ................................................................... 2
Camera Cognex Insight 5000............................................................................................. 3
Main Controlling PC......................................................................................................... 3
LaunchPad Microcontroller ............................................................................................... 4
Light Sensor .................................................................................................................... 4
Conveyor Belt.................................................................................................................. 5
Rotary Encoder ................................................................................................................ 6
Software Implementation ................................................................................................... 6
Cognex InSight 4.9.3 ........................................................................................................ 6
Energia ........................................................................................................................... 8
Code Composer Studio 6.1.2.............................................................................................. 8
Microcontroller Code Operation......................................................................................... 8
Visual Studio Enterprise 2015............................................................................................ 8
VB.NET Code Operation .................................................................................................. 9
End-of-Project Plan.......................................................................................................... 10
Safety.................................................................................................................................. 10
Suggestions and Recommendations....................................................................................... 11
Conclusion........................................................................................................................... 11
References........................................................................................................................... 12

v
List of Figures
Figure 1: Robotic Arm CRS A255............................................................................................. 2
Figure 2: Camera Cognex Insight 5000 ...................................................................................... 3
Figure 3: Launchpad MSP430 G2553 ........................................................................................ 4
Figure 5: Light Sensor.............................................................................................................. 4
Figure 4: Conveyor Belt and Motor Driver ................................................................................. 5
Figure 6: Rotary Encoder SICK................................................................................................. 6
Figure 7: Part Detection ........................................................................................................... 7
Figure 8: Spreadsheet view....................................................................................................... 7
Figure 9: Drive Robot break down............................................................................................. 9
Figure 10: Process Diagram versus Time.................................................................................. 10

1
Introduction
As of the writing of this report Purdy’s chocolatier uses manual labor to sort and package chocolates.
This is not only an inefficient method but also expensive and the repetitiveness of the task can bring
discomfort and/or strain to the workers. This project automates the packaging process of parts with
sorting and part detection by bringing efficiencies up and costs down for a packaging process. The
project can also be expanded for any required capacity and can be used for any similar process beyond
the chocolate industry field.
Project Scope/Objectives
The following sections describe the objectives set at the start of the project and which of those were
met.
Original Objective
This section describes what we planned to achieve before starting the project.
The original goal was to upgrade the project from 2015 that the team did before us. First, we had to
bring the project to the working state as in 2015 when one robot was picking different shapes of
objects from the conveyor. The camera was used to identify the type and position of the objects on the
conveyor. We planned to involve the second robotic arm to improve effectiveness of the process and
add adjustment of the speed of the conveyor to make sure that all the objects are picked-up from the
conveyor. As the objects we planned to use were chocolates, we needed to place them into a box. The
defective chocolates are to be disposed. Also, we had to take into account the emergency situation,
when the entire process stops, but we can still see what went wrong on the PC by looking at the logs.
Objectives Met
This section describes the objectives that we achieved at the end of the project:
● Redesigned the entire project, which allowed us to pick objects from the moving conveyor
using the coordinates that we acquired from the camera.
● Redesigned the image recognition process to eliminate the error of getting coordinates and
type of the object wrong.
● Redesigned the software that interfaces the camera and drives the robotic arm. Our software
allows to process a great amount of objects at the same time.
● Implemented quality control by rejecting the object that didn’t meet the defined criteria.
● Logged the communication between the PC, robotic arm, camera module, and the
microcontroller for further statistical analysis.
● Interfaced an encoder to know the exact position of a part at any speed of the conveyor belt.

2
Project Description
The following section describes the hardware and software implementation as well as the desired
operation of the project.
Hardware Implementation
The equipment listed below was used to implement the automation process.
Robotic Arm CRS A255 and C500C Controller
Figure 1: Robotic Arm CRS A255
The arm used to pick up the chocolates is an old 5-axis industrial arm with the gripper force control. It
is connected to the C500C controller, a device that is used to communicate between the main PC and
the arm. The pendant is also connected to the C500C controller, and it is used to teach new locations
(variables) as well as to ‘home’ the arm, which is a required procedure at every start-up of the
controller. The arm is able to pick up one part at a time and place it on the correct tray. There is a
program loaded onto the controller called “test” that holds the position variables in a .V3 file.
However, all control of the arm is done through the VB.NET program (see “Software
Implementation” section).

3
Camera Cognex Insight 5000
Figure 2: Camera Cognex Insight 5000
The camera used is an industry grade vision tool used for part recognition and inspection. Every time
a part crosses the light sensor, the camera module sends a signal to trigger the camera to take a
picture, which is analyzed by the InSight software ultimately sending the corresponding coordinates
of an acceptable part to the main computer (see “Software Implementation” section).
Main Controlling PC
The main controlling PC is a DELL desktop computer running Windows 7. This PC hosts the Visual
Basic program Chocolate Factory that runs the process of interfacing camera module, robotic arm
controller, and a microcontroller through the serial ports.
We also used Cognex In-Sight Explorer 4.9.3 on the PC to program the camera module that does the
image analysis.
In addition, we used Visual Studio Enterprise 2015 on the PC to create Chocolate Factory program
using Visual Basic programming language. We also took advantage of the Team Foundation Server
to keep track of all the versions of our program from the day of its creation. Also, we used this service
to work on the VB.NET code independently from each other on separate computers.
We took advantage of Energia and Code Composer Studio 6.1.2 to program the microcontroller
Launchpad MSP430. Energia was useful to create a quick simple code to interface the light sensor,
trigger the camera module, and interface the rotary encoder to send current position to the main PC
through the USB. Code Composer Studio was the best to debug the microcontroller code as we were
working on it.
Windows XP, Active Robot, and Robcomm3 were very essential at the beginning of the project to
configure and set-up the robotic arm.

4
LaunchPad Microcontroller
Figure 3: Launchpad MSP430 G2553
We used MSP430 G2553 controller to interface our main PC to the peripherals of the project (see
“Appendix E Schematic Circuit Diagrams”). It is used to constantly poll the value of the
photoresistor, which the controller uses to trigger the camera through a level shifting circuit. The
LaunchPad is also connected to the rotary encoder’s A and B signals through another level shifting
circuit and is constantly sending the total encoder count to the main PC using the COM port (USB).
Light Sensor
Figure 4: Light Sensor
The light sensor consists of a white LED pointing directly at a photoresistor. This photoresistor is
used in a voltage divider that is connected to the microcontroller. As the lumens to the photoresistor
decrease, its resistance increases thereby changing the voltage drop across it. The microcontroller is
constantly polling the pin connected to that node, and, through its Analog to Digital Converter (ADC),
the controller reads a value from 0‒1024 (10 bit ADC). When a part crosses the light sensor, the
controller will compare the read ADC value to a predetermined ADC value, and if it the read value is
below the predefined value, the controller will send a signal to a level shifter to trigger the camera.
This predetermined value is called “Critical_Sensor_Value” in the microcontroller code, and it can be
calibrated for the ambient light present at the certain time and location.

5
Conveyor Belt
Figure 5: Conveyor Belt and Motor Driver
The project utilizes a 120-inch long conveyor belt that transports the parts from the camera to the arm.
The conveyor is moved by a 115v DC gear motor, which is powered by a motor driver. We set the
speed of the belt to a certain value by adjusting the knob (potentiometer) on the driver. We can choose
the conveyer speed to be from the slowest to the fastest on the driver, and, with the help of the rotary
encoder, we can have the system operating at any speed. The velocity of the belt is defined by the user
and is based on the minimum distance between moving parts to insure that robot picks all the parts.
We also attempted to implement an automatic speed control of the conveyer belt, but it was virtually
impossible to reverse engineer the circuit in a very short interval of time. The potentiometer (the
knob) that we wanted to replace is connected to a complex circuit, which schematic diagram is
unknown. Also, the low point of the potentiometer is not the ground, which will cause damage to the
Main PC if we connect a microcontroller to it directly.

6
Rotary Encoder
Figure 6: Rotary Encoder SICK
To have a closed loop system we used a rotary incremental encoder placed on the belt so that we
know the current position of the chocolates. The encoder is connected to the camera module’s 24V
voltage supply, and, through a level shifter, it sends encoder signals A and B to the microcontroller.
Because we decided to maximize the resolution of the position, we implemented quadrature in the
code, so we configured Launchpad microcontroller to have hardware interrupts that are triggered
every time there is a change on channel A or B, and over these interrupts, the encoder count value
gets updated. The controller constantly sends the encoder count to the main PC through the COM port
(USB).
Software Implementation
The following software was used to implement the analysis and control of the process.
Cognex InSight 4.9.3
The camera used in the project uses a program called InSight Explorer to analyze the pictures from
the camera. Using this program we created a .job file that detects parts with a certain shape and
contrast and sends their position to the main computer. The program we created has a picture of the
group of parts that needs to be detected, and by adjusting certain thresholds we regulate the size or
contrast level acceptable for the part to be identified within that group (see “Appendix A User
Manual”). The tool used to achieve this, called PatMax, can process up to ten parts of the same group
in one search region (i.e. a section of the picture, where the program looks for parts). This does not
pose a problem to our process since we identify parts as a group, but it can only process one part at a
time as it crosses the light sensor (see “Hardware Implementation” section).

7
Figure 7: Part Detection
The camera is set to trigger in a mode that is called “Camera”—a mode, in which a picture will be
taken if the trigger input on the camera module goes high (level triggered). This is achieved with the
microcontroller.
Figure 8: Spreadsheet view
In the “Spreadsheet View” we have the code that sends coordinates to the PC (see “Appendix C
Camera Code”). X-axis is along the width of the belt, and Y-axis is along the length of the conveyor,
and Y component of coordinate is not needed to be passed because the rotary encoder defines the
current Y coordinate for every object as it they move on the conveyer. There is a loop that checks the
positions of the parts and sends only a valid X coordinate to the main computer through the COM port
as a string. This process is repeated every time the camera takes a picture. In this code, we also sort
the parts by their position relatively to the arm and only send a coordinate of the part closest to the
arm.

8
Energia
(github.com, 2016)
The Energia IDE (Integrated Development Environment) was used to create the program that the
microcontroller used. Through the simplicity of the environment, we were able to write the bulk of the
program a lot faster. After creating the program, we imported the sketch into Code Composer
Studio—a more robust programming environment that is used in industries.
Code Composer Studio 6.1.2
(dev.ti.com, 2015)
Inside Code Composer Studio (CCS) we merged the programs we created in Energia into one file, and
through CCS environment we debugged the program.
Microcontroller Code Operation
The microcontroller code serves two main functions, send a signal to trigger the camera and send the
encoder count to the main PC. The first part is achieved by reading the 10 bit ADC which shows the
voltage value in the voltage divider from the photoresistor, and comparing it to a variable called
“Critical_Sensor_Value” (see “Appendix D Microcontroller Code”). If the ADC value is below that, it
means there is a part blocking the light sensor and sends a signal out of the microcontroller to a level
shifter to trigger the camera.
For the encoder count, we have signals A and B from the encoder going into the controller and using
quadrature and hardware interrupts we increment/decrement the encoder count. In the main loop we
constantly send serially, through a COM port, the encoder count value.
Visual Studio Enterprise 2015
To create the main program that communicates with the microcontroller, robot, and camera, we
decided to use Visual Basic.NET as the programming language and Visual Studio as the compiler.
We chose these tools because of the robustness of the compiler and the advantages that object
oriented programming gives when it comes to interfacing various processes at the same time.
We also used as a version control Team Foundation Server, which allowed us to save, recover, and
compare the versions as we were moving forwards with our Chocolate Factory program. In addition,
this service let us work on the code independently on different computers and merge them together
using the Merging Tool.

9
VB.NET Code Operation
We wrote Chocolate Factory program using VB.NET programing language, which runs on the main
PC and interfaces the camera module, the robot controller, and the microcontroller through the serial
port (see “Appendix B Chocolate Factory Code”). The most important concept laying in our code is
that PC considers each chocolate that is entered on the conveyer belt as an individual entity with its
own “bio”. Once the program receives the coordinates from the camera module, the program creates
an independent parallel process (task) “Math” corresponding with the acquired coordinate. When a
new task “Math” had started, we capture the encoder count to be a frame of reference for the
chocolate that had just entered. We already know the distance between the camera capture point and
the work object where the robotic arm will pick the part up, and we can track the current position of
the chocolate using the rotary encoder as it moves on the conveyer belt. Once the all the mathematical
computations are done, we start another independent task “Drive Robot”, which controls the motion
of the robotic arm. The moment when the chocolate arrives to the robot’s workspace, the arm will
pick it up and place the chocolate into the appropriate location.
Figure 9: Drive Robot break down
The advantage of having parallel tasks is that the program will process every task at the same time
without affecting other tasks. However, a problem can appear if more than one “Drive Robot” tasks
will be running at the same time; the robot will collide, because the robot will execute more than one
command at the same time. Therefore, we do not let the “Drive Robot” run before the completion of
the previous “Drive Robot” task.
The Process diagram on the next page describes the operation of the Chocolate Factory program when
there are multiple chocolates on the conveyer belt at the same time.

10
Figure 10: Process Diagram versus Time
End-of-Project Plan
The project will be left in its current state, along with software files for reference to be used by any
future teams that would like to take the project further by adding more features or creating their own
process. Additionally, we will make a video tutorial with detailed explanation on how to set and
operate our system for the future developers in 2017.
Safety
The Automated Chocolate Sorting Process has a lot of moving parts, so the system has many possible
hazards, which we prevent/stop by having emergency stops. The system is equipped with 3 safety
buttons that can be pressed at any moment to stop the process depending on the hazard level. The
C500C controller and the teach pendant are equipped with their own emergency buttons that cut
power to the arm in case it runs out of control. If the process is deemed to be in a high risk state, the
master emergency button must be pressed to cut power to the arm and the conveyor belt, but the
computer with the microcontroller remain on so that we can perform investigation.

11
Suggestions and Recommendations
Our recommendation to the team undertaking this project again in 2017 are the following.
First, fully automate the system by automatically adding and removing trays or boxes that are full
from the table. Second, put a hopper, which will feed the chocolates onto the conveyer in a
completely random order. The main PC program will control the Hopper. Third, add a second robotic
arm to the process, which will greatly increase the effectiveness of the system. Fourth, rewrite our
code for Chocolate Factory program in C# programming language. C# is very similar to VB.NET, but
it has a larger developer community, and it is very popular in programming industry. Also, rewriting
the code will bring the programmer to great understanding of what is happening in the program,
which will help you accomplish higher goals in the end.
Conclusion
Over the period from Jan 21, 2016 and May 6, 2016 (16 weeks), we have accomplished great success
despite all the difficulties we faced. The process is able to detect and sort between different parts. The
robotic arm will pick the chocolate up virtually at any speed of the conveyer. We can feed parts at the
same time randomly, and the system keeps track of all the objects that are on the conveyor, and the
robotic arm (if not constrained by its own speed limitation) can pick all the parts up from the moving
conveyer.

12
References
(2016). Retrieved from github.com: https://github.com/energia/Energia
Instruments, T. (2015). Retrieved from https://dev.ti.com/
VanLaerhoven, T., & Sanjari, S. (2015). Automated Packaging Process Report. Burnaby: SnT
Solutions™.

A.1
Table of Contents
List of Table and Figures ....................................................................................................... 2
Tables ............................................................................................................................... 2
Figures .............................................................................................................................. 2
Description of the System....................................................................................................... 3
Introduction ........................................................................................................................ 3
List of Parts......................................................................................................................... 3
Operation............................................................................................................................ 9
How to Operate the System .................................................................................................... 9
Setting up PC ...................................................................................................................... 9
Starting up “Chocolate Factory” software ............................................................................... 9
Setting up the CRS Controller ............................................................................................... 9
Turning on the CRS controller............................................................................................. 10
Setting up Cognex Camera.................................................................................................. 10
Setting up the Microcontroller ............................................................................................. 10
Add/Modify Chocolate’s Type ............................................................................................ 11
Running the process ........................................................................................................... 11
Troubleshooting................................................................................................................... 12
List of specifications............................................................................................................. 13

A.2
List of Table and Figures
Tables
Table 1: Troubleshooting Procedure.....................................................................................................12
Table 2: List of Parts with the Specifications .......................................................................................13
Figures
Figure 1: Tray For Objects........................................................................................................ 3
Figure 2: Cognex Camera Module............................................................................................. 4
Figure 3: Cognex Camera......................................................................................................... 4
Figure 4: Level Shifter 2 (24V to 3.6V)...................................................................................... 5
Figure 5: White LED ............................................................................................................... 5
Figure 6: Photo-resistor............................................................................................................ 6
Figure 7: Rotary Encoder.......................................................................................................... 6
Figure 8: Microcontroller Launchpad MSP430............................................................................ 7
Figure 9: Motor Driver............................................................................................................. 7
Figure 10: Conveyer Belt.......................................................................................................... 8
Figure 11: Robotic Arm CRS A255 ........................................................................................... 8

A.3
Description of the System
Introduction
Automated Chocolate Sorting Process is designed to classify the type of chocolate, pick them from
the conveyer, and place into the appropriate location using the robotic arm. The goal of the process is
to eliminate as much human labor as possible to increase the profit margin of the company. In
addition, our process can be used to deal with other different types of objects on the factory, for
example to sort white board markers. This document outlines the steps on how to set-up and operate
the system.
List of Parts
Figure 1: Tray For Objects

A.4
Figure 2: Cognex Camera Module
Figure 3: Cognex Camera

A.5
Figure 4: Level Shifter 2 (24V to 3.6V)
Figure 5: White LED

A.6
Figure 6: Photo-resistor
Figure 7: Rotary Encoder

A.7
Figure 8: Microcontroller Launchpad MSP430
Figure 9: Motor Driver

A.8
Figure 10: Conveyer Belt
Figure 11: Robotic Arm CRS A255

A.9
Operation
As chocolate enters on the conveyer belt, it crosses the light gauge sensor. The microcontroller
constantly monitors the light gauge sensor and will send a triggering signal to the camera if the object
had crossed the sensor. Once the controller had triggered the camera, the Cognex camera takes a
picture and the Cognex software analyses the image. As a result of the analysis, the software
determines whether the chocolate had met the predefined criteria i.e. is of a desired type and not
damaged. Also the software determines the coordinates of the chocolate, which are sent to the main
computer. As the conveyer is moving, the rotary encoder sends pulses to the microcontroller, which
sends the current coordinates to the main computer. Once the computer received chocolate’s
coordinates from the camera, the computer will drive the robot to pick the chocolate up when the
chocolate entered the arm’s work area. If there are too many objects to be picked up at the same time,
the computer will send the command to the microcontroller to stop the conveyer belt since the
conveyor belt’s speed is driven by the microcontroller.
How to Operate the System
Setting up PC
1) Make sure you are running Windows 7.
2) Install the InSight Cognex software (In-sight Explorer 4.9.3).
3) Install the Chocolate Factory software.
4) Install the Code Composer Studio to calibrate the microcontroller.
Starting up “Chocolate Factory” software
1) Download and run Chocolate Factory (refer to section “Files”).
2) For the robotic arm, choose your COM port # and set the BAUD rate to be 57600 bits per
second. Click initialize connection.
3) For the camera, choose your COM port # and set the BAUD rate to be 115200 bits per
second. Click initialize connection.
Setting up the CRS Controller
1) Connect the feedback cable to the CRS controller and the Robotic Arm.
DANGER: CONNECT THE GROUND CABLE TO THE FRAME OF THE CRS
CONTROLLER. DO NOT APPLY POWER IF THE GROUND IS NOT CONNECTED.
2) Connect the power cable to the CRS Controller and the Robotic Arm.
DANGER: CONNECT THE GROUND CABLE TO THE FRAME OF THE CRS
CONTROLLER. DO NOT APPLY POWER IF THE GROUND IS NOT CONNECTED.
3) Connect the power cable to the controller.
4) Connect the serial cable to the front panel port on the CRS Controller.

A.10
Note: if the serial cable is not connected specifically to the front panel, the CRS Controller will not
boot.
5) Connect the pendant to the Controller.
At this point your robotic arm, pendant, and PC is connected to the CRS Controller.
Turning on the CRS controller
1) Turn on the CRS controller using the switch.
2) Wait for 10 sec for the controller to boot.
3) Using pendant align the 5 joints to the arrow that indicates the home position, and click
HOME.
Caution: Make sure that the arrows are exactly aligned with the arrows; otherwise, robot can collide
with the conveyer.
4) On the pendant click ESC until you see “Release the Control”. Click YES.
5) In Chocolate Factory, click Enter key in the command line for the robot. Communication, and
you should see “$” appear in the terminal.
Setting up Cognex Camera
1) Connect the serial cable to the camera and the PC.
2) Open Insight Explorer.
3) Click on the Get Connected Tab.
Note: Make sure to be in the EasyBuilder view. If you are on spreadsheet view,
press [ctrl + shift + v] or click on View → EasyBuilder.
4) Click COGNEXInSight → Connect.
5) Open our camera job file (refer to section “files”).
6) Click on the Online button at the top right of the interface.
Setting up the Microcontroller
1) Connect the light gauge sensor to Pin A3.
2) Connect the level shifting circuit to Pin 1.0.
3) Connect the rotary encoder channel A to Pin 1.4, and channel B to Pin 1.5.
4) Launch Code Composer Studio.
5) Click Project → Import Energia Sketch or Example. Import our code for microcontroller
(refer to section “files”).
6) Build, upload, and run our microcontroller code.

A.11
Add/Modify Chocolate’s Type
1) Go to InSight Explorer.
2) Click on the Locate Part Tab.
Note: Make sure to be in the EasyBuilder view. If you are on spreadsheet view,
press [ctrl + shift + v] or click on View → EasyBuilder.
3) Select PatMax Patterns (1-10) and click on add.
4) Place the desired chocolate on the camera’s view.
5) Select appropriate model and search shapes.
6) Place the model shape on top of the chocolate so that it covers only the chocolate.
7) On the settings tab select the desired tolerance thresholds.
At this point you successfully defined a new object, which all passing parts will be compared against.
Running the process
At this point you should have completed the previous steps, and your process is ready to run.
1) Using the switch, turn on the conveyer belt.
2) Place the chocolate on the conveyer, which type is defined.
At this point the robotic arm has to pick it up and place it into the tray.
3) Place undefined object on the conveyer.
At this point the robotic arm should ignore it, and the part will fall into the trash.
Congratulations! You have successfully set-up the Automated Chocolate Sorting Process! You are
now ready to use this product in your industry.

A.12
Troubleshooting
Over the operation of the system you might face certain problems, which can be fixed easily as
described below:
Table 1: Troubleshooting Procedure
Problem Possible Cause Possible Solution
Power to the arm
is not applied
● E-Stop is activated.
● Power is disabled in
software.
● Fuses are burnt
● Release you stop buttons.
● Press “Power On” on the CRS controller.
● In terminal type “armpower 1”.
● Change the fuses
Joints are limped ● Unlimp the joints in
software.
● Fuses are burnt.
● In terminal type “unlimp all”.
● Change the fuses.
Camera doesn’t
trigger
● The room is too
bright.
● System is not online.
● Decrease the brightness in the room.
● Calibrate the critical point for triggering in the
microcontroller’s code.
● Click on the online button in In-Sight Explorer.
Camera
constantly
triggers
● The room is too
dark.
● Increase the brightness in the room.
● Calibrate the critical point for triggering in the
microcontroller’s code.
Serial
communication
is closed
● COM Port settings
are not configured.
● The COM Port is
used by another
software.
● Make sure the BAUD rate is correct (refer to
“How to Operate the System” section).
● Check available COM port # in the Device
Manager (refer to Microsoft).
● Turn off all the software that might be using
your COM port.
● Turn off virtual machine.
● Restart PC.
Objects are
ignored
● This type is not
loaded.
● Refer to Section “Add/Modify Chocolates
Type”.
Program is stuck
in diagnostic
mode
● Controller was
configured to boot
on this mode.
● Type “boot” into the terminal.
If the solution to your problem is not listed, please contact: vasiliy.baryshnikov@gmail.com

A.13
List of specifications
The following table contains all the parts with their specification that we used over the project.
Table 2: List of Parts with the Specifications
Part Parameter Value
Conveyor Surface length
Width
Usable Width
Height
120 inches
18 inches
8 inches
38 inches
Motor Voltage
Gear Ratio
Current
HP
115V
9.3 : 1
2.1 A
0.25 HP
Motor Driver Input Voltage
Output Voltage
Rated Current
HP
120 VAC - 240 VAC
90 VDC - 180 VDC
3A
0.33 HP
Microcontroller LaunchPad Input Voltage
Digital Output Voltage
Analog Output Voltage (PWM)
3.6 VDC
3.6 VDC
3.6 VDC, 500Hz
Rotary Encoder Sick Input Voltage
Resolution
24 VDC
200 Pulse/rev
LED Voltage Drop
Operating Current
3.1V
5mA
Photoresistor Resistance 1k Oms
Camera Module Input Voltage
Digital Output Voltage
120VAC
24VDC
Note: This table is partially based on “Automated Packaging Process” 2015 report by VanLaerhoven
and Sanjar.

B.1
Appendix B
Chocolate Factory Code
(PC Application)

B.2
Table of Contents
Table of Figures:..................................................................................................................................................2
Form Code ...........................................................................................................................................................3
Form Design Code............................................................................................................................................ 10
Form Design...................................................................................................................................................... 18
Table of Figures:
Figure 1: Screen Shot of the Chocolate Factory Application ............................................................................18

B.3
Form Code
'*********************************************************************************
'* This Program is written to:
'* 1) control the CRS Robot Using the Visual Studio environment through the Serial Port
'* a) Pick the objects up and place them on to the tray
'* 2) interface the camera (receive ready coordinates from the In-Sight using the Serial Port
'* 3) theoretically capable to process infinitely many parts (coordinates) to be picked in a short period of time (parallel tasks)
'* 4) interface the microcontroller (receive encoder counts, that will tell the current position of the chocolates)
'*
'*
'* Written by Vasiliy Baryshnikov and Cesar Moreno. 5 May 2016.
'**********************************************************************************
Option Explicit On 'force programmer to declare all variables
'including the libraries and stuff
Imports System
Imports System.Threading
Imports System.IO.Ports
Imports System.ComponentModel
Public Class Form1
Dim myPort As Array 'declaring an array of the COM port list that will be gotten later when configuring the
COM ports
Delegate Sub SetTextCallBack(ByVal [text] As String) 'to prevent threading problems in the receiver side (from online)
'*********************************************************************
Dim QuickStartUP As Boolean = 0 'enables or disables precompiled auto start-up
Dim COMPortRobot As String = "COM1" 'FOR AUTOCONNECT AT START UP
Dim BaudRateRobot As Integer = 57600 'FOR AUTOCONNECT AT START UP
Dim COMPortCamera As String = "COM9" 'FOR AUTOCONNECT AT START UP
Dim BaudRateCamera As Integer = 115200 'FOR AUTOCONNECT AT START UP
Dim COMPortController As String = "COM10" 'FOR AUTOCONNECT AT START UP
Dim BaudRateController As Integer = 9600 'FOR AUTOCONNECT AT START UP
Dim Force As Integer = 100 'declaring the force of the gripper
Dim Speed As Integer = 100 'declaring speed for the robot
Dim TimeSleep As Integer = 1000 'declaring the time the robot should sleep
'*********************************************************************
Dim EncoderCount As UInt32 = 0 'initializing the encoder count (this will be assigned by the terminal where we receive counts from
controller)
Dim CoordinatString As String 'the string of coordinates received by the camera
Dim Coordinate As Double 'the acquired coordinate converted into double (only 1 coordinate because 2nd is the same)
Dim FaulseCoordinate As Double = 0 'we define the number that camera assigns as the coordinate of objects that didn't pass inspection
Dim CoordinateOutOfReach As Double = 6.8 '[inches] we define the maximum value for the coordinate received from Camera that the robot will be
able to handle (Robot's X-axis)
Dim zOffsetFactor As Double = 1.2 '[inches] define the hight of the chocolate (since the robot will be placing chocolates on top of another
chocolate, so it needs offset)
Dim OffsetFactor As Double = 1.2 '[inches] define the distance between two locations on the tray
|o---o|
Dim TrayPartsNum As Integer = 0 'initialise the number of parts placed on to the tray (maximum 4 parts, as specified in the MaxTrayPrts) |o-
--o|
Dim MaxTrayPrts As Integer = 8 ' define the number maximum number of parts that can fit on the tray
Public CompensationTimeBooster As Double = 0.15 'define the time sec by how much the robot will close the gripper earlier (Positive value is
earlier, negative value is later)
Dim CompensationCountBooster As UInt32 = 200 ' 600 for high speed
Dim RobotTaskQueSize As Integer = 0 'initialise the size of the que for the parallel tasks (that will be permitted to control the robot when it
is their tern)
Dim RunningTaskIndex As Integer = 0 'the current running task # index number (will be incremented whenever a new robot task is started)
Dim ComlpetedTaskIndex As Integer = 0 'the competed task # index number (will be incremented whenever the task is completed)
Dim CAMERA_FILE As String = "CameraRecieved.txt" 'the file name of txt file where all the coordinates are saved (wiped clean at the start-up)
Dim ROBOT_FILE As String = "RobotComunicationLog.txt" 'the file name of txt file where all the comunication with robot are saved (Keeps
growing, never erases)
Dim CONTROLLER_FILE As String = "ControllerComunicationLog.txt" 'the file name of txt file where all the comunication with robot are saved
(Keeps growing, never erases)
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
'this subroutine is run on the start-up of the program, and runs only once, so I take advatage of it to initialize and start-up here
everything
myPort = IO.Ports.SerialPort.GetPortNames() 'acquire the list of the availbale PORTs and save them as an array
PortListRobot.Items.AddRange(myPort) 'assign the list of the ports to the list box
PortListCamera.Items.AddRange(myPort) 'assign the list of the ports to the list box
PortListController.Items.AddRange(myPort) 'assign the list of the ports to the list box
If QuickStartUP = True Then
'COMMENT THIS SECTION IF YOU WANT TO ENABLE MANUAL INITIALIZATION BY USER (OR CHANGE QuickStartUP PARAMETER 20 LINES ABOVE)
'(Ctrl + K + Ctrl + C to COMMENT; or Ctrl + K + Ctrl + U to Uncomment)
'*******QUICK_START*******QUICK_START*******QUICK_START*******QUICK_START*******QUICK_START

B.4
'preset the COMPORT parameters to make the proccess faster
SerialPortRobot.PortName = COMPortRobot 'grab the specified by user COM Port and assign it to the serial port configurator
SerialPortRobot.BaudRate = BaudRateRobot 'grab the specified by user BUAD RATE and assign it to the serial port configurator
SerialPortRobot.Open() 'only now we can open the comunication in the serial port
cmdConnectToRobot.Enabled = False 'disable the button once we are connected to prevent problems
PortListRobot.Enabled = False 'disable the button once we are connected to prevent problems
RateListRobot.Enabled = False 'disable the button once we are connected to prevent problems
SerialPortCamera.PortName = COMPortCamera 'grab the serial name from the serial port
SerialPortCamera.BaudRate = BaudRateCamera 'grab the BaudRate and assign it to the serial port setting
SerialPortCamera.Open() 'only now we can open the comunication in the serial port
cmdConnecttoCamera.Enabled = False 'disable the button once we are connected
PortListCamera.Enabled = False 'disable the button once we are connected
RateListCamera.Enabled = False 'disable the button once we are connected
PortListRobot.Text = COMPortRobot 'simply display the setting as selected in the list box
PortListCamera.Text = COMPortCamera 'simply display the setting as selected in the list box
PortListController.Text = COMPortController 'simply display the setting as selected in the list box
RateListRobot.Text = BaudRateRobot 'simply display the setting as selected in the list box
RateListCamera.Text = BaudRateCamera 'simply display the setting as selected in the list box
RateListController.Text = BaudRateController 'simply display the setting as selected in the list box
'*******QUICK_START*******QUICK_START*******QUICK_START*******QUICK_START*******QUICK_START
End If
FileOpen(1, CAMERA_FILE, OpenMode.Output) : FileClose() 'wipe the Camera text file clean at the start-up of the program
FileOpen(1, ROBOT_FILE, OpenMode.Append) 'open the Robot Log file in the append mode (adding things)
PrintLine(1, "**************************************************************") 'keep adding text to the text file
PrintLine(1, (DateAndTime.Now & " " & "Started New Sesion")) 'stated that a new session is started, for the log tracking (time and date
included)
FileClose(1) 'cose the file, so that I can use it somewhere else
FileOpen(2, CONTROLLER_FILE, OpenMode.Append) 'open the Robot Log file in the append mode (adding things)
PrintLine(2, (DateAndTime.Now & " " & "Started New Sesion")) 'stated that a new session is started, for the log tracking (time and date
included)
CheckBoxAutoScrollRobot.Checked = "true" 'configure the Robot Comunication TextBox (terminal) to autoscroll
CheckBoxAutoScrollCamera.Checked = "true" 'configure the Robot Comunication TextBox (terminal) to autoscroll
CheckBoxAutoScrollController.Checked = "true" 'configure the Robot Comunication TextBox (terminal) to autoscroll
End Sub
Private Sub cmdConnectToRobot_Click(sender As Object, e As EventArgs) Handles cmdConnectToRobot.Click
'this subroutine is run if the "connect to robot" button is pressed
SerialPortRobot.PortName = PortListRobot.Text 'grab the specified by user COM Port and assign it to the serial port configurator
SerialPortRobot.BaudRate = RateListRobot.Text 'grab the specified by user BUAD RATE and assign it to the serial port configurator
SerialPortRobot.Open() 'only now we can open the communication in the serial port
cmdConnectToRobot.Enabled = False 'disable the button once we are connected to prevent problems
PortListRobot.Enabled = False 'disable the button once we are connected to prevent problems
RateListRobot.Enabled = False 'disable the button once we are connected to prevent problems
End Sub
Private Sub cmdGripperClose_Click(sender As Object, e As EventArgs) Handles cmdGripperClose.Click
'this subroutine is run if the "Gripper Close" button is pressed
GripperClose(Force, 0)
End Sub
Private Sub cmdGripperOpen_Click(sender As Object, e As EventArgs) Handles cmdGripperOpen.Click
'this subroutine is run if the "Gripper Open" button is pressed
GripperOpen(Force, 0)
End Sub
'*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*
' These functions are called whenever we tell a robot to do something
Public Sub GripperOpen(ByVal [force] As Integer, ByVal [delayTime] As Integer)
SerialPortRobot.Write("go" & " " & [force] & vbCr) ' the command will look like "go 100 "
Threading.Thread.Sleep([delayTime])
End Sub
Private Sub GripperClose(ByVal [force] As Integer, ByVal [delayTime] As Integer)
SerialPortRobot.Write("gc" & " " & [force] & vbCr)
End Sub
Private Sub MoveJ(ByVal [PointName] As String, ByVal [delayTime] As Integer)
SerialPortRobot.Write("move" & " " & [PointName] & vbCr) ' the command will look like "pointname" (joint interpolation)
End Sub
Private Sub MoveL(ByVal [PointName] As String, ByVal [delayTime] As Integer)
SerialPortRobot.Write("moves" & " " & [PointName] & vbCr) ' the command will look like "pointname" (straight-line mode)
Threading.Thread.Sleep(500)
End Sub
Private Sub SleepRobot(ByVal [miliseconds] As Integer)
Threading.Thread.Sleep([miliseconds]) 'waits 1 second
End Sub
Private Sub MoveHome(ByVal [delayTime] As Integer)
SerialPortRobot.Write("ready" & vbCr) ' the command will look like "ready"
End Sub

B.5
Private Sub MoveJ_By_X_World(ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("wx" & " " & [inches] & vbCr) ' the command will look like "wx [x-Coordinate]"
End Sub
Private Sub MoveJ_By_Y_World(ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("wy" & " " & [inches] & vbCr) ' the command will look like "yx [y-Coordinate]"
End Sub
Private Sub MoveJ_By_Z_World(ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("wz" & " " & [inches] & vbCr) ' the command will look like "zx [z-Coordinate]"
End Sub
Private Sub DepartJBy(ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("depart" & " " & [inches] & vbCr) 'moves the centre of the tool up by distance (joint interpolation)
End Sub
Private Sub DepartLBy(ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("departs" & " " & [inches] & vbCr) 'moves the centre of the tool up by distance (straight line mode)
End Sub
Private Sub ApproachJ(ByVal [PointName] As String, ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("appro" & " " & [PointName] & ", " & [inches] & vbCr) ' approaches the point with an offset (joint interpolation)
End Sub
Private Sub ApproachL(ByVal [PointName] As String, ByVal [inches] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("appros" & " " & [PointName] & ", " & [inches] & vbCr) ' approaches the point with an offset (straight line mode)
End Sub
Private Sub RotateJoint(ByVal [JointNum] As Integer, ByVal [degrees] As Double, ByVal [delayTime] As Integer)
SerialPortRobot.Write("joint" & " " & [JointNum] & "," & [degrees] & vbCr) ' approaches the point with an offset (straight line mode)
End Sub
'*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*R*
Private Sub SerialPortRobot_DataReceived(sender As Object, e As SerialDataReceivedEventArgs) Handles SerialPortRobot.DataReceived
'this subroutine is run when the port has data to receive
'This subroutine will print the recieved string on the Strin Box
ReceivedTextRobot(SerialPortRobot.ReadLine()) 'we are passing the text that is gotten from the port to a routine that will display it
End Sub
Private Sub ReceivedTextRobot(ByVal [text] As String)
'this subroutine is run when the port has data to receive and display
If Me.DataReceivedRobotTextBox.InvokeRequired Then ' this statement eliminates froblems (from online)
Dim x As New SetTextCallBack(AddressOf ReceivedTextRobot)
Me.Invoke(x, New Object() {(text)})
Else
Me.DataReceivedRobotTextBox.Text &= [text] 'Display the recieved text in the box under recieved section
FileOpen(1, ROBOT_FILE, OpenMode.Append) 'open the Robot Log file in the append mode (adding things)
PrintLine(1, (DateAndTime.Now & " " & [text])) 'add the line that we recieved via COM PORT (the line also includes the date and
time)
FileClose() 'cose the file, so that I can use it somewhere else
End If
' usually the recieved data does not include the sent data. But sometimes it would display the sent data as well (maybe due to handshaking)
End Sub
Private Sub WriteDataRobotTextBox_KeyPress(sender As Object, e As KeyPressEventArgs) Handles WriteDataRobotTextBox.KeyPress
If Asc(e.KeyChar) = 13 Then ' if user click 'ENTER' than output the text and clear the text box
SerialPortRobot.Write(WriteDataRobotTextBox.Text) 'Write to the PORT the content of the Text Box
WriteDataRobotTextBox.Text = Nothing 'clear the sent text box
End If
End Sub
Private Sub cmdConnecttoCamera_Click(sender As Object, e As EventArgs) Handles cmdConnecttoCamera.Click
SerialPortCamera.PortName = PortListCamera.Text 'grab the serial name from the serial port
SerialPortCamera.BaudRate = RateListCamera.Text 'grab the BaudRate and assign it to the serial port setting
SerialPortCamera.Open() 'only now we can open the comunication in the serial port
cmdConnecttoCamera.Enabled = False 'disable the button once we are connected
PortListCamera.Enabled = False 'disable the button once we are connected
RateListCamera.Enabled = False 'disable the button once we are connected
End Sub
Private Sub SerialPortCamera_DataReceived(sender As Object, e As SerialDataReceivedEventArgs) Handles SerialPortCamera.DataReceived
ReceivedTextCamera(SerialPortCamera.ReadLine()) ''we are passing the text(line) that is gotten from the port to a routine that will display
it
'camera must use 'LF' character as a delimeter(terminator) for this to work
End Sub
Private Sub ReceivedTextCamera(ByVal [text] As String)
If Me.DataReceivedRobotTextBox.InvokeRequired Then 'this statement eliminates froblems (from online)
Dim x As New SetTextCallBack(AddressOf ReceivedTextCamera)
Me.Invoke(x, New Object() {([text])})
Else
CoordinatString = [text] 'we update the global variable with the current gotten string from the cam
Double.TryParse(CoordinatString, Coordinate) 'this converts the coordinate string into the global double which I can use

B.6
'Coordinate = Coordinate - 2.5
MathRobot() 'this sub analyses the position of the object, and will start the DriveRobot sub in it
Me.DataReceivedCameraTextBox.Text &= Coordinate & " " 'Display the recieved text in the box under recieved section
DataStreamToTextFile([text], CAMERA_FILE) 'Write the stream into the text file for further use
'Since I don't recieve a nice string like "1.234567" that is convertable into the double esially, and I get randomly split strings
'I decided to use a text file as a buffer that would put the entire text niclly, and it would be convertable into what I need
End If
End Sub
Private Sub DataStreamToTextFile(ByVal [text] As String, ByVal [fileName] As String)
'this subroutine is run when we need to write simething into the text file
FileOpen(1, [fileName], OpenMode.Append) 'open the cirtain file in the append mode (adding things)
Print(1, [text]) 'keep adding text to the text file
End Sub
Private Sub WriteDataCameraTextBox_KeyPress(sender As Object, e As KeyPressEventArgs) Handles WriteDataCameraTextBox.KeyPress
'if ENTER is pressed while we input something into the box, this subroutine is run
If Asc(e.KeyChar) = 13 Then 'if the button that pressed is ENTER
SerialPortCamera.Write(WriteDataCameraTextBox.Text) 'Write to the PORT the content of the Text Box
WriteDataCameraTextBox.Text = Nothing 'clear the sent text box
End If
End Sub
Private Sub cmdClearHistoryRobot_Click(sender As Object, e As EventArgs) Handles cmdClearHistoryRobot.Click
'this subroutine is run if the user clicked to Clear the terminal history
DataReceivedRobotTextBox.Text = Nothing
End Sub
Private Sub cmdClearHistoryCamera_Click(sender As Object, e As EventArgs) Handles cmdClearHistoryCamera.Click
DataReceivedCameraTextBox.Text = Nothing
End Sub
Private Sub cmdDisconnectFromRobot_Click(sender As Object, e As EventArgs) Handles cmdDisconnectFromRobot.Click
SerialPortRobot.Close() 'close the serial connection
cmdConnectToRobot.Enabled = True 'enable the button once we are connected to prevent problems
PortListRobot.Enabled = True 'enable the button once we are connected to prevent problems
RateListRobot.Enabled = True 'enable the button once we are connected to prevent problems
End Sub
Private Sub cmdDisconnectFromCamera_Click(sender As Object, e As EventArgs) Handles cmdDisconnectFromCamera.Click
SerialPortCamera.Close() 'close the serial connection
cmdConnecttoCamera.Enabled = True 'enable the button once we are connected to prevent problems
PortListCamera.Enabled = True 'enable the button once we are connected to prevent problems
RateListCamera.Enabled = True 'enable the button once we are connected to prevent problems
End Sub
Private Sub MathRobot() 'this sub will process the location of the objects to pick them up
'every time the sub MathRobot is called, this will start-up separate independant procecess (on a new thread)
'disadvantage, I may longer have access to different Forms (eg Form2) of my project from the task running on a different thread
Dim Math = Task.Factory.StartNew(
Sub()
Dim EncoderCountAtCapture As UInt32 = EncoderCount 'we need to capture the encoder counter at the time the camera is trigered as a
reference point
Dim TrayPositionIndex As Integer ' local variable that represents where the chocolate will be placed on the tray
'declaring some variables for the encoder part
Dim Inch_Per_mm As Double = 0.0393700787 'conversion factor = 1"/1mm
Dim Pulse_Per_MM As Double = 1 'number of pulses over 1 mm of objects traveled distance (specs)
Dim Pulse_Per_Rev As Double = 800 'pulse generated by the rot encoder over 1 rev. Technically it is 200 * 4(quadrature) (specs)
Dim Distance_CameraRobot_Inch_MIN As Double = 92 'the minimal distance between the camera and the robot at which the arm can pick
part up (defined by the work object)
Dim Distance_CameraRobot_Inch_MAX As Double = 100 'the maximum distance between the camera and the robot at which the arm can pick
part up (defined by the work object)
'********************************************* 'the START code to be exicuted
Dim EncoderCountToReachRobot As UInt32 = 9250 'this number represent the distance in counts between the camera and the robot
workobject
Dim EncoderCountAtPickUp As UInt32 = EncoderCountToReachRobot + EncoderCountAtCapture 'getting the encoder count value for the part
to enter the workobject of CRS
If ((Coordinate > FaulseCoordinate) And (Coordinate < CoordinateOutOfReach)) Then 'check whether the object passed inspection
(inspection is done in In-Sight Program), if it is not a valid part or out of reach, then don't drive the robot
'inpection is done in the insight program. We recieve "0" as a coordinate if the part didn't meet the required specifications.
'I am putting the tray analysis here so that the object will get assigned its own index for the tray right away
TrayPartsNum = TrayPartsNum + 1 'increase the number of parts that will be placed on to the tray to find the appropriate
position number
If TrayPartsNum > MaxTrayPrts Then : TrayPartsNum = 1 : End If 'if the number of parts on the tray exceeds the maximum allowed,
set TrayPartsNum to 1 (which corresponds to the position number on the tray)

B.7
TrayPositionIndex = TrayPartsNum ' assign the current position number count of the tray to the local variable (index) that will
be passed to the robot function for further work
'at the end we pass cirtain parameter to the subroutine that will drive the robot
DriveRobot(Coordinate, TrayPositionIndex, EncoderCountAtPickUp) 'a new proccess for the robot is created once we jump there,
and will be placed in the que to avoid conflicting proccesses
End If
'********************************************* 'the END code to be exicuted
End Sub) ' the independent math proccess finishes here, in the meantime the DriveRobot prooccess is running
End Sub
Private Sub DriveRobot(ByVal [PartCoordinate] As Double, ByVal [TrayPositionIndex] As Integer, ByVal [EncoderCountAtPickUp] As UInt32)
'this sub will drive the robot to pick and place based on the locations that were passed (somehow) earlier.
'every time the sub DriveRobot is called, this will start-up a separate independant procecess (on a new thread)
'disadvantage, I may longer have access to different Forms (eg Form2) of my project from the task running on a different thread
'the task is pseudo parallel! it is held on old untill the previos same task is completed (introduced a Que concept because I broke the
robot without it)
Dim Robot = Task.Factory.StartNew(
Sub()
RobotTaskQueSize = RobotTaskQueSize + 1 'this indicates that the que size had increased as we started the proccess
RunningTaskIndex = RunningTaskIndex + 1 'this indicates that the index number had been increased
Dim LocalTaskIndex = RunningTaskIndex 'we assign the uniqui index number specifically to this task
Console.WriteLine("Proccess Started: QueSize = " & RobotTaskQueSize & ", Current Index = " & LocalTaskIndex & ", Completed Index =
" & ComlpetedTaskIndex) 'display some info on screen
While (ComlpetedTaskIndex < (LocalTaskIndex - 1)) 'wait untill the task is in the que just before THIS TASK is completed
Thread.Sleep(10) 'check every 10ms for task status in the que (no reason why 10ms)
End While 'Now it's this task's tern to continue with the code (amd nothing will interfear with it)
'Note: I broke the robot because I didn't have this que concept
'********************************************* 'the start code to be executed (imagine that the header of this sub is not there)
Dim ZOffsetTray As Double 'the Y offset to place the parts on the tray (local variable because each proccess will have its own
unique value for this var)
Dim PositionTray As String = "place" 'tells the program which of the four positions to place the wheel
Dim ApproachOffset As Double = 2 'inches, the offset with which the robot departs and approaches (to avoid colision before moving
somewhere)
'assign the respective offset to the object depending on its induvidual index (passed from the math function)
If [TrayPositionIndex] = 1 Then ' | ---o|
PositionTray = "place" ' | --- |
ZOffsetTray = zOffsetFactor * 0
ElseIf [TrayPositionIndex] = 2 Then ' |o--- |
PositionTray = "place2" ' | --- |
ElseIf [TrayPositionIndex] = 3 Then ' | --- |
PositionTray = "place3" ' | ---o|
ElseIf [TrayPositionIndex] = 4 Then ' | --- |
ElseIf [TrayPositionIndex] = 5 Then ' | --- | To be done in the inteliggent way (loop with verticle offset)
PositionTray = "place" ' | ---o|
End If
'brining the robot to the position where it will pick up the parts (conveyer)
GripperOpen(Force, 0)
ApproachJ("origin", ApproachOffset, 0)
MoveJ_By_X_World([PartCoordinate], 0)
SleepRobot(TimeSleep)
MoveJ_By_Z_World(-ApproachOffset, 0)
'wait untill the part is there (here probobly I will be using the real position from the encoder instead of time wise location)
' While ((TimeAtPickUp - CompensationTimeBooster) > DateAndTime.Timer) 'if in the time wise the part did not arrive, wait! (not
considering the time it takes for the robot to grab... etc)
While ((EncoderCountAtPickUp - CompensationCountBooster) > EncoderCount) 'if in the encoder count wise the part did not arrive,
wait! (not considering the time it takes for the robot to grab... etc)
End While 'if the in time wise the part entered the CRS's workj object, continue doing the code below
'Part ENTERED the Work Object
Console.Beep() 'make a sound signal :)
'pick-up the part and bring it to the tray
GripperClose(Force, 0)

B.8
DepartLBy(ApproachOffset, 0)
ApproachJ(PositionTray, ApproachOffset, 0)
MoveJ_By_Z_World(-2.1 + ZOffsetTray, 0)
SleepRobot(2000)
GripperOpen(70, 0)
'brining the robot to the position where it will be fast for it to execute further commands (back to the conveyer)
DepartLBy(ApproachOffset, 0)
ApproachJ("origin", ApproachOffset, 0)
'********************************************* 'the end of the code to be exicured (imagine that the bottom of this sub is not
there)
ComlpetedTaskIndex = ComlpetedTaskIndex + 1 'completed proccess over the index numbers
RobotTaskQueSize = RobotTaskQueSize - 1 'this indicates that the que size had decreased as we finished the proccess
Console.WriteLine("Proccess Finished: QueSize = " & RobotTaskQueSize & ", Current Index = " & LocalTaskIndex & ", Completed Index =
" & ComlpetedTaskIndex) 'display some info on screen
End Sub) ' the independent DriveRobot proccess finishes here, in the mean time the other same proccess might be running and waiting for
there tern
End Sub
Private Sub cmdResetTrayPosition_Click(sender As Object, e As EventArgs) Handles cmdResetTrayPosition.Click
TrayPartsNum = 0 'if we need to force reset the tray position, it can be done by click the button
End Sub
Private Sub DataReceivedRobotTextBox_TextChanged(sender As Object, e As EventArgs) Handles DataReceivedRobotTextBox.TextChanged
'this subroutine is called, when the text is changed in the Robot comunication terminal box
If CheckBoxAutoScrollRobot.Checked = "true" Then 'if the check mark is present (that we want autoscroll enabled)
DataReceivedRobotTextBox.SelectionStart = DataReceivedRobotTextBox.TextLength 'we set the caret position to the value of the last
character's position in the text box (i.e. the lenghth of text)
DataReceivedRobotTextBox.ScrollToCaret() 'scrolls the content of the textbox to the current caret position, which was assigned in the
previous statement
End If
End Sub
Private Sub DataReceivedCameraTextBox_TextChanged(sender As Object, e As EventArgs) Handles DataReceivedCameraTextBox.TextChanged
'this subroutine is called, when the text is changed in the Camera comunication terminal box
If CheckBoxAutoScrollCamera.Checked = "true" Then 'if the check mark is present (that we want autoscroll enabled)
DataReceivedCameraTextBox.SelectionStart = DataReceivedCameraTextBox.TextLength 'we set the caret position to the value of the last
character's position in the text box (i.e. the lenghth of text)
DataReceivedCameraTextBox.ScrollToCaret() 'scrolls the content of the textbox to the current caret position, which was assigned in the
previous statement
End If
End Sub
Private Sub SerialPortController_DataReceived(sender As Object, e As SerialDataReceivedEventArgs) Handles SerialPortController.DataReceived
ReceivedTextController(SerialPortController.ReadLine()) 'we are passing the text that is gotten from the port to a routine that will
display it
End Sub
Private Sub ReceivedTextController(ByVal [text] As String)
If Me.DataReceivedControllerTextBox.InvokeRequired Then ' this statement eliminates froblems (from online)
Dim x As New SetTextCallBack(AddressOf ReceivedTextController)
Me.Invoke(x, New Object() {(text)})
Else
Dim EncoderCountString As String = [text] 'capture the string in a local variable
UInt32.TryParse(EncoderCountString, EncoderCount) 'this converts the encoder count string into the global 32 bit integer which I can
use
Me.DataReceivedControllerTextBox.Text &= [text] 'Display the recieved text in the box under recieved section
FileOpen(2, CONTROLLER_FILE, OpenMode.Append) 'open the Robot Log file in the append mode (adding things)
PrintLine(2, (DateAndTime.Now & " " & [text])) 'add the line that we recieved via COM PORT (the line also includes the date and
time)
End If
End Sub
Private Sub DataReceivedControllerTextBox_TextChanged(sender As Object, e As EventArgs) Handles DataReceivedControllerTextBox.TextChanged
'enables auto scroll
'this subroutine is called, when the text is changed in the Controller comunication terminal box
If DataReceivedControllerTextBox.TextLength > 1000 Then
DataReceivedControllerTextBox.Clear()
End If
If CheckBoxAutoScrollController.Checked = "true" Then 'if the check mark is present (that we want autoscroll enabled)

B.9
DataReceivedControllerTextBox.SelectionStart = DataReceivedControllerTextBox.TextLength 'we set the caret position to the value of the
last character's position in the text box (i.e. the lenghth of text)
DataReceivedControllerTextBox.ScrollToCaret() 'scrolls the content of the textbox to the current caret position, which was assigned in
the previous statement
End If
End Sub
Private Sub WriteDataControllerTextBox_KeyPress(sender As Object, e As KeyPressEventArgs) Handles WriteDataControllerTextBox.KeyPress
'if ENTER is pressed while we input something into the box, this subroutine is run
If Asc(e.KeyChar) = 13 Then 'if the button that pressed is ENTER
SerialPortController.Write(WriteDataControllerTextBox.Text) 'Write to the PORT the content of the Text Box
WriteDataControllerTextBox.Text = Nothing 'clear the sent text box
End If
End Sub
Private Sub cmdConnecttoController_Click(sender As Object, e As EventArgs) Handles cmdConnecttoController.Click
'this subroutine is run if the "connect to controller" button is pressed
SerialPortController.PortName = PortListController.Text 'grab the specified by user COM Port and assign it to the serial port configurator
SerialPortController.BaudRate = RateListController.Text 'grab the specified by user BUAD RATE and assign it to the serial port configurator
SerialPortController.Open() 'only now we can open the comunication in the serial port
cmdConnecttoController.Enabled = False 'disable the button once we are connected to prevent problems
PortListController.Enabled = False 'disable the button once we are connected to prevent problems
RateListController.Enabled = False 'disable the button once we are connected to prevent problems
End Sub
Private Sub cmdClearHistoryController_Click(sender As Object, e As EventArgs) Handles cmdClearHistoryController.Click
DataReceivedControllerTextBox.Text = Nothing
End Sub
End Class

B.10
Form Design Code
<Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()> _
Partial Class Form1
Inherits System.Windows.Forms.Form
'Form overrides dispose to clean up the component list.
<System.Diagnostics.DebuggerNonUserCode()> _
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
Try
If disposing AndAlso components IsNot Nothing Then
components.Dispose()
End If
Finally
MyBase.Dispose(disposing)
End Try
End Sub
'Required by the Windows Form Designer
Private components As System.ComponentModel.IContainer
'NOTE: The following procedure is required by the Windows Form Designer
'It can be modified using the Windows Form Designer.
'Do not modify it using the code editor.
<System.Diagnostics.DebuggerStepThrough()> _
Private Sub InitializeComponent()
Me.components = New System.ComponentModel.Container()
Me.cmdGripperOpen = New System.Windows.Forms.Button()
Me.cmdGripperClose = New System.Windows.Forms.Button()
Me.RateListRobot = New System.Windows.Forms.ComboBox()
Me.PortListRobot = New System.Windows.Forms.ComboBox()
Me.cmdConnectToRobot = New System.Windows.Forms.Button()
Me.SerialPortRobot = New System.IO.Ports.SerialPort(Me.components)
Me.SerialPortCamera = New System.IO.Ports.SerialPort(Me.components)
Me.DataReceivedRobotTextBox = New System.Windows.Forms.RichTextBox()
Me.WriteDataRobotTextBox = New System.Windows.Forms.RichTextBox()
Me.PortListCamera = New System.Windows.Forms.ComboBox()
Me.RateListCamera = New System.Windows.Forms.ComboBox()
Me.cmdConnecttoCamera = New System.Windows.Forms.Button()
Me.DataReceivedCameraTextBox = New System.Windows.Forms.RichTextBox()
Me.WriteDataCameraTextBox = New System.Windows.Forms.RichTextBox()
Me.Label1 = New System.Windows.Forms.Label()
Me.cmdClearHistoryRobot = New System.Windows.Forms.Button()
Me.cmdClearHistoryCamera = New System.Windows.Forms.Button()
Me.cmdDisconnectFromRobot = New System.Windows.Forms.Button()
Me.cmdDisconnectFromCamera = New System.Windows.Forms.Button()
Me.Label2 = New System.Windows.Forms.Label()
Me.CheckBoxAutoScrollRobot = New System.Windows.Forms.CheckBox()
Me.CheckBoxAutoScrollCamera = New System.Windows.Forms.CheckBox()
Me.cmdResetTrayPosition = New System.Windows.Forms.Button()
Me.SerialPortController = New System.IO.Ports.SerialPort(Me.components)
Me.DataReceivedControllerTextBox = New System.Windows.Forms.RichTextBox()
Me.WriteDataControllerTextBox = New System.Windows.Forms.RichTextBox()
Me.cmdDisconnectFromController = New System.Windows.Forms.Button()
Me.cmdClearHistoryController = New System.Windows.Forms.Button()
Me.cmdConnecttoController = New System.Windows.Forms.Button()
Me.RateListController = New System.Windows.Forms.ComboBox()
Me.PortListController = New System.Windows.Forms.ComboBox()
Me.CheckBoxAutoScrollController = New System.Windows.Forms.CheckBox()
Me.SuspendLayout()
'

B.11
'cmdGripperOpen
'
Me.cmdGripperOpen.Location = New System.Drawing.Point(266, 4)
Me.cmdGripperOpen.Name = "cmdGripperOpen"
Me.cmdGripperOpen.Size = New System.Drawing.Size(105, 24)
Me.cmdGripperOpen.TabIndex = 0
Me.cmdGripperOpen.Text = "Gripper Open"
Me.cmdGripperOpen.UseVisualStyleBackColor = True
'
'cmdGripperClose
'
Me.cmdGripperClose.Location = New System.Drawing.Point(266, 35)
Me.cmdGripperClose.Name = "cmdGripperClose"
Me.cmdGripperClose.Size = New System.Drawing.Size(105, 23)
Me.cmdGripperClose.TabIndex = 1
Me.cmdGripperClose.Text = "Gripper Close"
Me.cmdGripperClose.UseVisualStyleBackColor = True
'
'RateListRobot
'
Me.RateListRobot.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.RateListRobot.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.RateListRobot.FormattingEnabled = True
Me.RateListRobot.Items.AddRange(New Object() {"9600", "19200", "38400", "57600", "115200"})
Me.RateListRobot.Location = New System.Drawing.Point(82, 4)
Me.RateListRobot.Name = "RateListRobot"
Me.RateListRobot.Size = New System.Drawing.Size(67, 24)
Me.RateListRobot.TabIndex = 9
'
'PortListRobot
'
Me.PortListRobot.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.PortListRobot.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.PortListRobot.FormattingEnabled = True
Me.PortListRobot.Location = New System.Drawing.Point(12, 4)
Me.PortListRobot.Name = "PortListRobot"
Me.PortListRobot.Size = New System.Drawing.Size(64, 24)
Me.PortListRobot.TabIndex = 10
'
'cmdConnectToRobot
'
Me.cmdConnectToRobot.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdConnectToRobot.Location = New System.Drawing.Point(12, 35)
Me.cmdConnectToRobot.Name = "cmdConnectToRobot"
Me.cmdConnectToRobot.Size = New System.Drawing.Size(137, 24)
Me.cmdConnectToRobot.TabIndex = 11
Me.cmdConnectToRobot.Text = "Connect to Robot"
Me.cmdConnectToRobot.UseVisualStyleBackColor = True
'
'SerialPortRobot
'
'
'SerialPortCamera
'
'
'DataReceivedRobotTextBox
'
Me.DataReceivedRobotTextBox.AccessibleName = ""
Me.DataReceivedRobotTextBox.BackColor = System.Drawing.Color.FromArgb(CType(CType(64, Byte),
Integer), CType(CType(64, Byte), Integer), CType(CType(64, Byte), Integer))

B.12
Me.DataReceivedRobotTextBox.Font = New System.Drawing.Font("Courier New", 9.75!,
System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, CType(0, Byte))
Me.DataReceivedRobotTextBox.ForeColor = System.Drawing.Color.White
Me.DataReceivedRobotTextBox.Location = New System.Drawing.Point(12, 70)
Me.DataReceivedRobotTextBox.Name = "DataReceivedRobotTextBox"
Me.DataReceivedRobotTextBox.ReadOnly = True
Me.DataReceivedRobotTextBox.Size = New System.Drawing.Size(640, 753)
Me.DataReceivedRobotTextBox.TabIndex = 12
Me.DataReceivedRobotTextBox.Text = ""
'
'WriteDataRobotTextBox
'
Me.WriteDataRobotTextBox.AccessibleName = "WriteDataTextBox"
Me.WriteDataRobotTextBox.BackColor = System.Drawing.SystemColors.Info
Me.WriteDataRobotTextBox.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.WriteDataRobotTextBox.Location = New System.Drawing.Point(12, 829)
Me.WriteDataRobotTextBox.Name = "WriteDataRobotTextBox"
Me.WriteDataRobotTextBox.Size = New System.Drawing.Size(640, 31)
Me.WriteDataRobotTextBox.TabIndex = 13
Me.WriteDataRobotTextBox.Text = ""
'
'PortListCamera
'
Me.PortListCamera.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.PortListCamera.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.PortListCamera.FormattingEnabled = True
Me.PortListCamera.Location = New System.Drawing.Point(690, 5)
Me.PortListCamera.Name = "PortListCamera"
Me.PortListCamera.Size = New System.Drawing.Size(60, 24)
Me.PortListCamera.TabIndex = 14
'
'RateListCamera
'
Me.RateListCamera.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.RateListCamera.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.RateListCamera.FormattingEnabled = True
Me.RateListCamera.Items.AddRange(New Object() {"115200"})
Me.RateListCamera.Location = New System.Drawing.Point(756, 5)
Me.RateListCamera.Name = "RateListCamera"
Me.RateListCamera.Size = New System.Drawing.Size(78, 24)
Me.RateListCamera.TabIndex = 15
'
'cmdConnecttoCamera
'
Me.cmdConnecttoCamera.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdConnecttoCamera.Location = New System.Drawing.Point(690, 35)
Me.cmdConnecttoCamera.Name = "cmdConnecttoCamera"
Me.cmdConnecttoCamera.Size = New System.Drawing.Size(144, 28)
Me.cmdConnecttoCamera.TabIndex = 16
Me.cmdConnecttoCamera.Text = "Connect to Camera"
Me.cmdConnecttoCamera.UseVisualStyleBackColor = True
'
'DataReceivedCameraTextBox
'
Me.DataReceivedCameraTextBox.AccessibleName = ""
Me.DataReceivedCameraTextBox.BackColor = System.Drawing.Color.Navy
Me.DataReceivedCameraTextBox.Font = New System.Drawing.Font("Courier New", 9.75!,
Me.DataReceivedCameraTextBox.ForeColor = System.Drawing.SystemColors.Menu
Me.DataReceivedCameraTextBox.Location = New System.Drawing.Point(690, 69)
Me.DataReceivedCameraTextBox.Name = "DataReceivedCameraTextBox"

B.13
Me.DataReceivedCameraTextBox.ReadOnly = True
Me.DataReceivedCameraTextBox.Size = New System.Drawing.Size(392, 753)
Me.DataReceivedCameraTextBox.TabIndex = 17
Me.DataReceivedCameraTextBox.Text = ""
'
'WriteDataCameraTextBox
'
Me.WriteDataCameraTextBox.AccessibleName = "WriteDataTextBox"
Me.WriteDataCameraTextBox.BackColor = System.Drawing.SystemColors.Info
Me.WriteDataCameraTextBox.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.WriteDataCameraTextBox.Location = New System.Drawing.Point(690, 829)
Me.WriteDataCameraTextBox.Name = "WriteDataCameraTextBox"
Me.WriteDataCameraTextBox.Size = New System.Drawing.Size(392, 31)
Me.WriteDataCameraTextBox.TabIndex = 18
Me.WriteDataCameraTextBox.Text = ""
'
'Label1
'
Me.Label1.AutoSize = True
Me.Label1.Location = New System.Drawing.Point(858, 809)
Me.Label1.Name = "Label1"
Me.Label1.Size = New System.Drawing.Size(202, 13)
Me.Label1.TabIndex = 19
Me.Label1.Text = "By Vasiliy Baryshnikov and Cesar Moreno"
'
'cmdClearHistoryRobot
'
Me.cmdClearHistoryRobot.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdClearHistoryRobot.Location = New System.Drawing.Point(155, 4)
Me.cmdClearHistoryRobot.Name = "cmdClearHistoryRobot"
Me.cmdClearHistoryRobot.Size = New System.Drawing.Size(72, 23)
Me.cmdClearHistoryRobot.TabIndex = 21
Me.cmdClearHistoryRobot.Text = "Clear"
Me.cmdClearHistoryRobot.UseVisualStyleBackColor = True
'
'cmdClearHistoryCamera
'
Me.cmdClearHistoryCamera.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdClearHistoryCamera.Location = New System.Drawing.Point(840, 4)
Me.cmdClearHistoryCamera.Name = "cmdClearHistoryCamera"
Me.cmdClearHistoryCamera.Size = New System.Drawing.Size(72, 25)
Me.cmdClearHistoryCamera.TabIndex = 22
Me.cmdClearHistoryCamera.Text = "Clear"
Me.cmdClearHistoryCamera.UseVisualStyleBackColor = True
'
'cmdDisconnectFromRobot
'
Me.cmdDisconnectFromRobot.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdDisconnectFromRobot.Location = New System.Drawing.Point(155, 34)
Me.cmdDisconnectFromRobot.Name = "cmdDisconnectFromRobot"
Me.cmdDisconnectFromRobot.Size = New System.Drawing.Size(105, 24)
Me.cmdDisconnectFromRobot.TabIndex = 24
Me.cmdDisconnectFromRobot.Text = "Disconnect"
Me.cmdDisconnectFromRobot.UseVisualStyleBackColor = True
'
'cmdDisconnectFromCamera
'
Me.cmdDisconnectFromCamera.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdDisconnectFromCamera.Location = New System.Drawing.Point(840, 35)
Me.cmdDisconnectFromCamera.Name = "cmdDisconnectFromCamera"
Me.cmdDisconnectFromCamera.Size = New System.Drawing.Size(121, 28)

B.14
Me.cmdDisconnectFromCamera.TabIndex = 25
Me.cmdDisconnectFromCamera.Text = "Disconnect"
Me.cmdDisconnectFromCamera.UseVisualStyleBackColor = True
'
'Label2
'
Me.Label2.AutoSize = True
Me.Label2.Location = New System.Drawing.Point(437, 810)
Me.Label2.Name = "Label2"
Me.Label2.Size = New System.Drawing.Size(202, 13)
Me.Label2.TabIndex = 27
Me.Label2.Text = "By Vasiliy Baryshnikov and Cesar Moreno"
'
'CheckBoxAutoScrollRobot
'
Me.CheckBoxAutoScrollRobot.AutoSize = True
Me.CheckBoxAutoScrollRobot.Location = New System.Drawing.Point(637, 808)
Me.CheckBoxAutoScrollRobot.Name = "CheckBoxAutoScrollRobot"
Me.CheckBoxAutoScrollRobot.Size = New System.Drawing.Size(15, 14)
Me.CheckBoxAutoScrollRobot.TabIndex = 33
Me.CheckBoxAutoScrollRobot.UseVisualStyleBackColor = True
'
'CheckBoxAutoScrollCamera
'
Me.CheckBoxAutoScrollCamera.AutoSize = True
Me.CheckBoxAutoScrollCamera.Location = New System.Drawing.Point(1067, 808)
Me.CheckBoxAutoScrollCamera.Name = "CheckBoxAutoScrollCamera"
Me.CheckBoxAutoScrollCamera.Size = New System.Drawing.Size(15, 14)
Me.CheckBoxAutoScrollCamera.TabIndex = 34
Me.CheckBoxAutoScrollCamera.UseVisualStyleBackColor = True
'
'cmdResetTrayPosition
'
Me.cmdResetTrayPosition.Font = New System.Drawing.Font("Microsoft YaHei UI", 12.0!,
Me.cmdResetTrayPosition.Location = New System.Drawing.Point(455, 5)
Me.cmdResetTrayPosition.Name = "cmdResetTrayPosition"
Me.cmdResetTrayPosition.Size = New System.Drawing.Size(105, 53)
Me.cmdResetTrayPosition.TabIndex = 32
Me.cmdResetTrayPosition.Text = "Reset Tray Position"
Me.cmdResetTrayPosition.UseVisualStyleBackColor = True
'
'SerialPortController
'
'
'DataReceivedControllerTextBox
'
Me.DataReceivedControllerTextBox.AccessibleName = ""
Me.DataReceivedControllerTextBox.BackColor = System.Drawing.SystemColors.ControlText
Me.DataReceivedControllerTextBox.Font = New System.Drawing.Font("Courier New", 9.75!,
Me.DataReceivedControllerTextBox.ForeColor = System.Drawing.SystemColors.Menu
Me.DataReceivedControllerTextBox.Location = New System.Drawing.Point(1118, 69)
Me.DataReceivedControllerTextBox.Name = "DataReceivedControllerTextBox"
Me.DataReceivedControllerTextBox.ReadOnly = True
Me.DataReceivedControllerTextBox.Size = New System.Drawing.Size(405, 753)
Me.DataReceivedControllerTextBox.TabIndex = 35
Me.DataReceivedControllerTextBox.Text = ""
'
'WriteDataControllerTextBox
'

B.15
Me.WriteDataControllerTextBox.AccessibleName = "WriteDataTextBox"
Me.WriteDataControllerTextBox.BackColor = System.Drawing.SystemColors.Info
Me.WriteDataControllerTextBox.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.WriteDataControllerTextBox.Location = New System.Drawing.Point(1118, 828)
Me.WriteDataControllerTextBox.Name = "WriteDataControllerTextBox"
Me.WriteDataControllerTextBox.Size = New System.Drawing.Size(405, 31)
Me.WriteDataControllerTextBox.TabIndex = 36
Me.WriteDataControllerTextBox.Text = ""
'
'cmdDisconnectFromController
'
Me.cmdDisconnectFromController.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdDisconnectFromController.Location = New System.Drawing.Point(1276, 34)
Me.cmdDisconnectFromController.Name = "cmdDisconnectFromController"
Me.cmdDisconnectFromController.Size = New System.Drawing.Size(121, 28)
Me.cmdDisconnectFromController.TabIndex = 41
Me.cmdDisconnectFromController.Text = "Disconnect"
Me.cmdDisconnectFromController.UseVisualStyleBackColor = True
'
'cmdClearHistoryController
'
Me.cmdClearHistoryController.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdClearHistoryController.Location = New System.Drawing.Point(1276, 3)
Me.cmdClearHistoryController.Name = "cmdClearHistoryController"
Me.cmdClearHistoryController.Size = New System.Drawing.Size(72, 25)
Me.cmdClearHistoryController.TabIndex = 40
Me.cmdClearHistoryController.Text = "Clear"
Me.cmdClearHistoryController.UseVisualStyleBackColor = True
'
'cmdConnecttoController
'
Me.cmdConnecttoController.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.cmdConnecttoController.Location = New System.Drawing.Point(1118, 34)
Me.cmdConnecttoController.Name = "cmdConnecttoController"
Me.cmdConnecttoController.Size = New System.Drawing.Size(152, 28)
Me.cmdConnecttoController.TabIndex = 39
Me.cmdConnecttoController.Text = "Connect to Controller"
Me.cmdConnecttoController.UseVisualStyleBackColor = True
'
'RateListController
'
Me.RateListController.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.RateListController.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.RateListController.FormattingEnabled = True
Me.RateListController.Items.AddRange(New Object() {"9600", "57600"})
Me.RateListController.Location = New System.Drawing.Point(1192, 4)
Me.RateListController.Name = "RateListController"
Me.RateListController.Size = New System.Drawing.Size(78, 24)
Me.RateListController.TabIndex = 38
'
'PortListController
'
Me.PortListController.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList
Me.PortListController.Font = New System.Drawing.Font("Microsoft Sans Serif", 10.0!)
Me.PortListController.FormattingEnabled = True
Me.PortListController.Location = New System.Drawing.Point(1118, 4)
Me.PortListController.Name = "PortListController"
Me.PortListController.Size = New System.Drawing.Size(68, 24)
Me.PortListController.TabIndex = 37
'
'CheckBoxAutoScrollController

B.16
'
Me.CheckBoxAutoScrollController.AutoSize = True
Me.CheckBoxAutoScrollController.Location = New System.Drawing.Point(1508, 809)
Me.CheckBoxAutoScrollController.Name = "CheckBoxAutoScrollController"
Me.CheckBoxAutoScrollController.Size = New System.Drawing.Size(15, 14)
Me.CheckBoxAutoScrollController.TabIndex = 42
Me.CheckBoxAutoScrollController.UseVisualStyleBackColor = True
'
'Form1
'
Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
Me.BackColor = System.Drawing.Color.FromArgb(CType(CType(255, Byte), Integer),
CType(CType(255, Byte), Integer), CType(CType(128, Byte), Integer))
Me.ClientSize = New System.Drawing.Size(1584, 871)
Me.Controls.Add(Me.CheckBoxAutoScrollController)
Me.Controls.Add(Me.cmdDisconnectFromController)
Me.Controls.Add(Me.cmdClearHistoryController)
Me.Controls.Add(Me.cmdConnecttoController)
Me.Controls.Add(Me.RateListController)
Me.Controls.Add(Me.PortListController)
Me.Controls.Add(Me.WriteDataControllerTextBox)
Me.Controls.Add(Me.DataReceivedControllerTextBox)
Me.Controls.Add(Me.CheckBoxAutoScrollCamera)
Me.Controls.Add(Me.CheckBoxAutoScrollRobot)
Me.Controls.Add(Me.cmdResetTrayPosition)
Me.Controls.Add(Me.Label2)
Me.Controls.Add(Me.cmdDisconnectFromCamera)
Me.Controls.Add(Me.cmdDisconnectFromRobot)
Me.Controls.Add(Me.cmdClearHistoryCamera)
Me.Controls.Add(Me.cmdClearHistoryRobot)
Me.Controls.Add(Me.Label1)
Me.Controls.Add(Me.WriteDataCameraTextBox)
Me.Controls.Add(Me.DataReceivedCameraTextBox)
Me.Controls.Add(Me.cmdConnecttoCamera)
Me.Controls.Add(Me.RateListCamera)
Me.Controls.Add(Me.PortListCamera)
Me.Controls.Add(Me.WriteDataRobotTextBox)
Me.Controls.Add(Me.DataReceivedRobotTextBox)
Me.Controls.Add(Me.cmdConnectToRobot)
Me.Controls.Add(Me.PortListRobot)
Me.Controls.Add(Me.RateListRobot)
Me.Controls.Add(Me.cmdGripperClose)
Me.Controls.Add(Me.cmdGripperOpen)
Me.Location = New System.Drawing.Point(-400, -400)
Me.Name = "Form1"
Me.Text = "Chocolate Factory"
Me.ResumeLayout(False)
Me.PerformLayout()
End Sub
Friend WithEvents cmdGripperOpen As Button
Friend WithEvents cmdGripperClose As Button
Friend WithEvents RateListRobot As ComboBox
Friend WithEvents PortListRobot As ComboBox
Friend WithEvents cmdConnectToRobot As Button
Friend WithEvents SerialPortRobot As IO.Ports.SerialPort
Friend WithEvents SerialPortCamera As IO.Ports.SerialPort
Friend WithEvents DataReceivedRobotTextBox As RichTextBox
Friend WithEvents WriteDataRobotTextBox As System.Windows.Forms.RichTextBox

B.17
Friend WithEvents PortListCamera As System.Windows.Forms.ComboBox
Friend WithEvents RateListCamera As System.Windows.Forms.ComboBox
Friend WithEvents cmdConnecttoCamera As System.Windows.Forms.Button
Friend WithEvents DataReceivedCameraTextBox As System.Windows.Forms.RichTextBox
Friend WithEvents WriteDataCameraTextBox As RichTextBox
Friend WithEvents Label1 As Label
Friend WithEvents cmdClearHistoryRobot As Button
Friend WithEvents cmdClearHistoryCamera As Button
Friend WithEvents cmdDisconnectFromRobot As Button
Friend WithEvents cmdDisconnectFromCamera As Button
Friend WithEvents Label2 As Label
Friend WithEvents CheckBoxAutoScrollRobot As CheckBox
Friend WithEvents CheckBoxAutoScrollCamera As CheckBox
Friend WithEvents cmdResetTrayPosition As Button
Friend WithEvents SerialPortController As IO.Ports.SerialPort
Friend WithEvents DataReceivedControllerTextBox As RichTextBox
Friend WithEvents WriteDataControllerTextBox As RichTextBox
Friend WithEvents cmdDisconnectFromController As Button
Friend WithEvents cmdClearHistoryController As Button
Friend WithEvents cmdConnecttoController As Button
Friend WithEvents RateListController As ComboBox
Friend WithEvents PortListController As ComboBox
Friend WithEvents CheckBoxAutoScrollController As CheckBox
End Class

B.18
Form Design
Figure 1: Screen Shot of the Chocolate Factory Application

Appendix C
Camera Code
(Cognex InSight)

D.1
Appendix D
Microcontroller Code
(Launchpad MSP430)

D.2
//***************************************************************************************
//*Encoder and Light Sensor interface:
//*We are using a voltage divider circuit with the photoresistor. Circuit that is power by the 3.2V and GND
//*Reads an analog voltage input on pin A3, prints the result to the serial monitor and triggers the camera.
//*this program was also written to interface the encoder, and send the encoder counts
//*through the serial port to the PC
//Written By Cesar Moreno and Vasiliy Baryshnikov. May 6, 2016.
//***************************************************************************************
#define CRITICAL_SENSOR_VALUE 450 //the value, which we read from the photo resistor, that will trigger the
camera
#define CAMERA_TRIGGER_PIN P1_0 //assign the physical pin 1
#define PHOTO_RESISTOR_VOLTAGE_PIN A3 //assign the physical analog pin 3
//***************************************************************************************
uint8_t volatile edgeA,edgeB;
long int EncoderCount; long int EncoderCountOld; //declare the global vaiables
int ChannelA, ChannelB; //define Channal A,B
//***************************************************************************************
// the setup routine runs once when you press reset:
void setup()
{
Serial.begin(9600); // initialize serial communication at 9600 bits per second:
EncoderCount = 0; EncoderCountOld = 0; //initialise the encoder count to zero at start-up
pinMode(P1_4, INPUT); //configure pin 1.4 (chanelA) to be input
pinMode(P1_5, INPUT); //configure pin 1.5 (chanelB) to be input
pinMode(CAMERA_TRIGGER_PIN,OUTPUT); //configure pin to be output
digitalWrite(CAMERA_TRIGGER_PIN,LOW); //initialise the output to be low
attachInterrupt(P1_4, ChannelA_isr, FALLING); //configure the interupt on channel A to be on falling edgeA (will be
chaned to rising)
attachInterrupt(P1_5, ChannelB_isr, FALLING);//configure the interupt on channel B to be on falling edgeA (will be
chaned to rising)
edgeA = FALLING; //initialise the flag in the edge variable parameter
edgeB = FALLING;//initialise the flag in the edge variable parameter
}
//***************************************************************************************
//***************************************************************************************
void loop()//main infinite loop
{
int sensorValue = analogRead(PHOTO_RESISTOR_VOLTAGE_PIN); // capture the analog input on pin (ADC)
if (sensorValue < CRITICAL_SENSOR_VALUE)// check whether the part had crossed the light sensor (which will change
the resistance value => change the voltage value on the ADC)
{
digitalWrite(CAMERA_TRIGGER_PIN,HIGH); // trigger the camera quick
delay(50); // hold ON for 0.050 sec
digitalWrite(CAMERA_TRIGGER_PIN,LOW);
// delay(250);
}
int delayTimeInterval = 20;
for (int delayTime = 0; delayTime <= 100; delayTime = delayTime + delayTimeInterval)
{
delay(delayTimeInterval); //wait 0.1s umtil the next scan
if (EncoderCount != EncoderCountOld) //making sure I am not sending the count everytime
{
Serial.println(EncoderCount); //send the count number through the serial port
EncoderCountOld = EncoderCount;//Update the old encoder count for further operations
}
}
}
//***************************************************************************************
//*********************************************************************************
void ChannelA_isr() {
ChannelB = digitalRead(P1_5); //capture the channels for further comparison
ChannelA = digitalRead(P1_4);
// EncoderCountOld = EncoderCount;//Update the old encoder count for further operations

D.3
if ((ChannelA) != (ChannelB)) EncoderCount++; //if the interrupted on A, and A and B
are different, increment counter
else EncoderCount--;
if(edgeA == FALLING){ //enable quadrature by allowing to be interrupted on rising and falling edges
attachInterrupt(P1_4, ChannelA_isr, RISING); //reconfigure interrupt to be rising edge for A
edgeA = RISING; //update the flag in the edge variable parameter
} else {
attachInterrupt(P1_4, ChannelA_isr, FALLING); //reconfigure interrupt to be falling edge for A
edgeA = FALLING; //update the flag in the edge variable parameter
}
// Serial.println(EncoderCount); //send the count number through the serial port
}
//*********************************************************************************
//*********************************************************************************
void ChannelB_isr() {
ChannelB = digitalRead(P1_5);//capture the channels for further comparison
ChannelA = digitalRead(P1_4);
// EncoderCountOld = EncoderCount;//Update the old encoder count for further operations
if ((ChannelA) == (ChannelB)) EncoderCount++;//if the interrupted on B, and A and B are same,
increment counter
else EncoderCount--;
if(edgeB == FALLING) { //enable quadrature by allowing to be interrupted on rising and falling edges
attachInterrupt(P1_5, ChannelB_isr, RISING); //reconfigure interrupt to be rising edge for B
edgeB = RISING; //update the flag in the edge variable parameter
} else {
attachInterrupt(P1_5, ChannelB_isr, FALLING); //reconfigure interrupt to be falling edge for B
edgeB = FALLING; //update the flag in the edge variable parameter
}
// Serial.println(EncoderCount); //send the count number through the serial port
}
//*********************************************************************************

Appendix E
Schematic Circuit
Diagrams

Figure 1: Microcontroller MSP430 G2553 Connections
Figure 2: Level Shifter 3.6V to 24V

Figure 3: CIO-1400 Camera Module (In-Sight®1400I/O Expansion Module
Installation and Reference Manual)

Final Project Report

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Final Project Report

Similar to Final Project Report (20)

Final Project Report