2. Table of Content
1. Introduction
2. Networking
3. Signal Processing Scheme
4. Links to Vector Calculator and Networking Android Studio Files
3. 1. Introduction
The assistive technology that is being researched is an Eyelid scanner device. The
device utilizes accelerometer to send x, y, z reading in bits and byte form to a portable
processor such as an Android mobile application. This application must be able to receive
data from the device wirelessly and then processes the data sent. It must be able to
recognize pattern from data and decide what to do next such as sending the processed
data to another device wirelessly.
2. Networking
There are many ways to send and receive data wirelessly between devices but the
best method for our specification is Bluetooth. Android Bluetooth APIs let applications
wirelessly connect to other Bluetooth devices, enabling point-topoint and multipoint
wireless features that allow user to scan for other bluetooth devices, query the local
Bluetooth deveies, establish RFCOMM channels, connect to other devices through
service discovery, manage multiple connections, and transfer data to and from other
devices. The range and speed for transfering data through Bluetooth is depending on the
Android devies’ capability. Most devices’ Bluetooth are capable of transmitting data in a
15 meters range, and 25 Mbps with Bluetooth 3.0.
To create the network for the device to send data to the android application, I
implemented a client-server Transmission Control Protocol (TCP) using Android
Bluetooth API. The application is made in JAVA and XML in Android Studio. All the
codes and explanation are listed below.
4. Since the Eyelid scanner is not yet fabricated, I mimicked the device with an
Android phone. So the connection will be between two android devices through
the application using Android Bluetooth API.
To allow Android Bluetooth API to work, there are some permissions to put in the
AndroidManifest.xml file. They are listed below.
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.BLUETOOTH" />
There are three activities in this application. They are MainActivity, Client, and
Server. Each activity have its own JAVA file and its XML layout file.
MainActivity.java
How to set up Bluetooth in Android Bluetooth API:
mBluetoothAdapter = BluetoothAdapter.getDefaultAdapter();
On the start of the Activity, it check if the device have Bluetooth capability or not.
If the device have Bluetooth Capability, it asks the user to turn the Bluetooth on if
the Bluetooth is off. If the device does not have Bluetooth capability there will be
a message display saying so.
public void onStart(){
super.onStart();
if(mBluetoothAdapter== null){
status.setText("Device does not support Bluetooth");
}else{
if(!mBluetoothAdapter.isEnabled()){
Intent enableIntent = new
Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE);
startActivityForResult(enableIntent, REQUEST_ENABLE_BT);
}
}
}
5. There are two buttons in this activity. They are RUN AS SERVER, and RUN AS
CLIENT. They each were created in the xml layout with a id, and also have a
listener to response to clicks. Once clicked it launch the next activity of the
application either Server activity or Client activity based on what the button
suggested.
XML codes for the buttons:
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/serverID"
android:text="Run as Server"
android:layout_below="@id/textview"
/>
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/clientID"
android:text="Run as Client"
android:layout_below="@id/serverID"
/>
JAVA codes for setting up the two buttons and the two ClickListeners:
Client = (Button) findViewById(R.id.clientID);
Server = (Button) findViewById(R.id.serverID);
Client.setOnClickListener(new View.OnClickListener(){
public void onClick(View V){
Intent intent = new Intent(MainActivity.this, Client.class );
MainActivity.this.startActivity(intent);
}
});
Server.setOnClickListener(new View.OnClickListener(){
public void onClick(View V){
Intent intent = new Intent(MainActivity.this, Server.class );
MainActivity.this.startActivity(intent);
}
});
Layout of MainActivity:
6. Client.java
public void onStart(){
super.onStart();
if(!mBluetoothAdapter.isEnabled()){
textview.setText(" Bluetooth disabled!");
Intent enableIntent = new
Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE);
startActivityForResult(enableIntent,
MainActivity.REQUEST_ENABLE_BT);
}else{
Set<BluetoothDevice> pairedDevices =
mBluetoothAdapter.getBondedDevices();
if(pairedDevices.size() > 0){
adapter.clear();
for(BluetoothDevice device : pairedDevices){
// textView.setText("Bluetooth Device Found");
adapter.add("nDevice: " + device.getName() + " " +
device.getAddress() + "n");
}
}
}
}
On the start of this activity, it check to see if the Bluetooth is turned on or not then
it display the status of the Bluetooth. If the Bluetooth is off, it will ask the user to
7. turn it on. If the Bluetooth is on, it will list all paired Bluetooth devices in an
array adapter. In our case, it will list our Eyelid Scanner.
Code for setting up an array adapter or listView to Bluetooth adapter in JAVA:
listView = (ListView) findViewById(R.id.listview);
adapter = new ArrayAdapter<String>(this,
android.R.layout.simple_list_item_1);
listView.setAdapter(adapter);
XML code for array adapter:
<ListView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/listview"
android:layout_below="@id/clientStatus"
/>
Once we clicked on the device listed, it will attempt to connect to the device.
listView.setOnItemClickListener(new AdapterView.OnItemClickListener() {
@Override
public void onItemClick(AdapterView<?> parent, View view, int
position, long id) { // make listview clickable to connect to particular
device
//MainActivity.mBluetoothAdapter.cancelDiscovery();
Set<BluetoothDevice> pairedDevices =
MainActivity.mBluetoothAdapter.getBondedDevices();
String info = ((TextView) view).getText().toString();
int i = 0;
for (BluetoothDevice listedDevice : pairedDevices) {
if (i == position) {
MainActivity.mmDevice = listedDevice;
adapter.clear();
adapter.add("nDevice: " +
MainActivity.mmDevice.getName() + " " +
MainActivity.mmDevice.getAddress() + "n");
final Thread mConnectThread = new
ConnectThread(MainActivity.mmDevice);
mConnectThread.start();
}
i++;
}
}
});
To connect to a device, the application create a thread called “ConnectThread”
which create a Bluetooth Socket for connection:
8. BluetoothSocket tmp = null;
mmDevice = device;
try{
tmp =
mmDevice.createRfcommSocketToServiceRecord(MainActivity.MY_UUID);
}catch (IOException e){e.printStackTrace();}
mmSocket = tmp;
}
And also calls Android Bluetooth API connect function to try to connect the the
clicked device. The status of the process is update thought a Handler.
Message msgObj = mHandler.obtainMessage();
Bundle b = new Bundle();
try{
String msg= " Attempting to connect";
b.putString("message", msg);
msgObj.setData(b);
mHandler.sendMessage(msgObj);
mmSocket.connect();
}catch (IOException e){
try{
mmSocket.close();
}catch (IOException e1){e1.printStackTrace();}
return;
}
Handler Implementation:
Handler mHandler = new Handler() {
@Override
public void handleMessage(Message msg) {
byte[] writeBuf = (byte[]) msg.obj;
int begin = (int)msg.arg1;
int end = (int)msg.arg2;
String data = msg.getData().getString("message");
if((null!=data)){
textview.append(data);
}else{
textview.append("no data");
}
}
};
Once the device is connected, the application create a new thread called
“ConnectedThread” which manage what’s going on between the connection.
9. ThreadmConnectedThread = new ConnectedThread(mmSocket);
mConnectedThread.start();
In the ConnectedThread, it check to see if there is any data to send or receive. If
there is any data received, it will update the status to the data received.
public class ConnectedThread extends Thread{
public ConnectedThread(BluetoothSocket socket){
mmSocket = socket;
InputStream tmpIn = null;
OutputStream tmpOut = null;
try{
tmpIn = socket.getInputStream();
tmpOut = socket.getOutputStream();
}catch (IOException e){e.printStackTrace();}
mmInStream = tmpIn;
mmOutStream = tmpOut;
}
public void run(){
byte[] buffer = new byte[1024];
int bytes;
while(true){
try{
bytes = mmInStream.read(buffer);
mHandler.obtainMessage(1, bytes, -1,
buffer).sendToTarget();
}catch (IOException e){e.printStackTrace();}
break;
}
}
}
Let backtrack a bit. Once the two devices are connected, the button and textfield
for inputing data to send to the server will appear. We can send the input data by
calling write function of the Android Bluetooth API.
To receive the status of the Bluetooth connection, there is a function in Android
Bluetooth API called BroadcastReceiver. The code for the set up of the reciever:
IntentFilter filter1 = new
IntentFilter(BluetoothDevice.ACTION_ACL_CONNECTED);
IntentFilter filter2 = new
IntentFilter(BluetoothDevice.ACTION_ACL_DISCONNECT_REQUESTED);
IntentFilter filter3 = new
IntentFilter(BluetoothDevice.ACTION_ACL_DISCONNECTED);
this.registerReceiver(mReceiver, filter1);
10. this.registerReceiver(mReceiver, filter2);
this.registerReceiver(mReceiver, filter3);
private final BroadcastReceiver mReceiver = new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
String action = intent.getAction();
BluetoothDevice device =
intent.getParcelableExtra(BluetoothDevice.EXTRA_DEVICE);
if (BluetoothDevice.ACTION_FOUND.equals(action)) {
//Device found
}
else if (BluetoothDevice.ACTION_ACL_CONNECTED.equals(action)) {
//Device is now connected
textview.setText(" Device connected!");
closeConnection.setVisibility(View.VISIBLE);
closeConnection.setOnClickListener(new View.OnClickListener()
{
@Override
public void onClick(View view) {
textview.setText("Attempting to Disconnect");
cancel();
}
});
X.setVisibility(View.VISIBLE);
Y.setVisibility(View.VISIBLE);
Z.setVisibility(View.VISIBLE);
inputData.setVisibility(View.VISIBLE);
send.setVisibility(View.VISIBLE);
datastatus.setVisibility(View.VISIBLE);
send.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
datastatus.setText("sending Data");
byte[] xin = X.getText().toString().getBytes();
byte[] yin = Y.getText().toString().getBytes();
byte[] zin = Z.getText().toString().getBytes();
write(xin);
write(yin);
write(zin);
}
});
}
else if
(MainActivity.mBluetoothAdapter.ACTION_DISCOVERY_FINISHED.equals(action))
{
//Done searching
}
else if
(BluetoothDevice.ACTION_ACL_DISCONNECT_REQUESTED.equals(action)) {
//Device is about to disconnect
}
else if (BluetoothDevice.ACTION_ACL_DISCONNECTED.equals(action)) {
//Device has disconnected
textview.setText(" Device disconnected!");
Set<BluetoothDevice> pairedDevices =
mBluetoothAdapter.getBondedDevices();
if(pairedDevices.size() > 0){
adapter.clear();
12. Server.java
Just like Client.java all the status messages are handle by the handler
implemented and the Bluetooth’s status can be detect through BroadcastReceiver. When
the activity start, there is a button called “BEGIN LISTENING FOR DATA”. When this
button is clicked, the application create an AcceptThread.
beginListening = (Button) findViewById(R.id.beginListening);
beginListening.setOnClickListener(new View.OnClickListener(){
public void onClick(View V){
AcceptThread mAcceptThread = new AcceptThread();
mAcceptThread.start();
}
});
For the devices to connect, they have to have the same UUID, below is how you can set
the UUID up.
BluetoothServerSocket tmp = null;
try{
tmp = mBluetoothAdapter.listenUsingRfcommWithServiceRecord("Meng",
MainActivity.MY_UUID);
}catch(IOException e){
e.printStackTrace();
}
mmServerSocket = tmp;
AcceptThread calls accept function of the Android Bluetooth API that keep listening to
incoming connection.
socket = null;
while(true) {
try {
String msg = "nListening for connection";
b.putString("message", msg);
msgObj.setData(b);
mHandler.sendMessage(msgObj);
socket = mmServerSocket.accept();
} catch (IOException e) {
e.printStackTrace();
break;
}
if (socket != null) {
Thread mConnected = new ConnectedThread(socket);
mConnected.start();
try {
13. mmServerSocket.close();
} catch (IOException e) {
}
break;
}
}
}
When the devices are connected ConnectThread create a ConnectedThread which
manage the connection such as receiving data and sending the data to the handler for
display.
public ConnectedThread(BluetoothSocket Socket){
socket = Socket;
InputStream tmpIn = null;
OutputStream tmpOut = null;
try{
tmpIn = socket.getInputStream();
tmpOut = socket.getOutputStream();
}catch (IOException e){e.printStackTrace();}
mmInStream = tmpIn;
mmOutStream = tmpOut;
}
public void run(){
byte[] buffer = new byte[1024];
int bytes;
while(true){
try{
bytes = mmInStream.read(buffer);
if(buffer != null){
String data = new String(buffer, 0, bytes);
Message msgObj = mHandler.obtainMessage();
Bundle b = new Bundle();
String msg= " nData: " +data;
b.putString("message", msg);
msgObj.setData(b);
mHandler.sendMessage(msgObj);
}else{
Message msgObj = mHandler.obtainMessage();
Bundle b = new Bundle();
String msg= "no Data!";
b.putString("message", msg);
msgObj.setData(b);
mHandler.sendMessage(msgObj);
}
// mHandler.obtainMessage(1, bytes, -1, buffer).sendToTarget();
}catch (IOException e){e.printStackTrace();}
break;
}
}
14. 3. Signal Processing Scheme
We can store the data received in arrays and do some vector computation to determine
the pattern of blinking. Below are the code for simple JAVA vectors computation.
public static float numX, numY, numZ, numX1, numY1, numZ1;
sumX2 = MainActivity.numX + MainActivity.numX1;
sumY2 = MainActivity.numY + MainActivity.numY1;
sumZ2 = MainActivity.numZ + MainActivity.numZ1;
differX = MainActivity.numX - MainActivity.numX1;
differY = MainActivity.numY - MainActivity.numY1;
differZ = MainActivity.numZ - MainActivity.numZ1;
productX = MainActivity.numX * MainActivity.numX1;
productY = MainActivity.numY * MainActivity.numY1;
productZ = MainActivity.numZ * MainActivity.numZ1;
dX = MainActivity.numX / MainActivity.numX1;
dY = MainActivity.numY / MainActivity.numY1;
dZ = MainActivity.numZ / MainActivity.numZ1;
4. Links to Vector Calculator and Networking Android Studio Files
Networking: https://psu.box.com/s/rv0t0rnikj6yshxu2nu2xmdm7vcnzdcl
Vector Computation: https://psu.box.com/s/wdysmm9vf9j8dst7on6fjsbiflxibk2a
