Research Methods EE

Copyright: David Thiel 2009
Research Methods in Electrical
Engineering
Professor David Thiel
Centre for Wireless Monitoring and
Applications
Griffith University, Brisbane Australia

Purpose
• To make attendees familiar with the
process of rigorous research in an
academic environment.
• To encourage attendees to critically
evaluate research papers they read.
• To outline the processes required to
undertake a research project.

Topics
1. Overview of the Research Process
2. Literature Search
3. Report Writing, Data Collection &
Presentation
4. Statistical Analysis of Data and Sampling
5. Making a Presentation
6. Survey Research Methods
7. Review

Topic 1
Overview of the Research Process

What is Research?
• Discovery of new things that have been
independently verified by other
professionals.
• Something new to humanity (not just new
to you or your group).

Good & Bad Research Examples
• Case 1 A high school research paper
• Case 2 A good idea
• Case 3 Tested outcomes for a new idea

The Scientific Method
An idea
Discovery
Independent verification:
literature, experiment,
numerical model,
analytical model, etc
Prior
knowledge
Submit
Report,
Thesis,
Journal
or
Conference
Paper
Assessors
Independent verification:
literature,
numerical model,
analytical model, etc
The Outcome is Recognised
as a Major contribution
to the field

The Research Community
• All use the same scientific method.
• All follow the same ethical principles.
• All use the same language and terms.
• All provide information to the world-wide
community reported in a full and open
manner.
• All acknowledge the previous work of
others.

Publications and Referencing
• The archival literature (must be printed
somewhere and unalterable).
• Must be reviewed by independent
professionals before publication.
• Must be complete so others can reproduce
the results.
• These three form the basic validity test!

Types of Publication
• Scientific papers (refereed journal and
conference papers)
• Trade articles
• Newspaper articles
• Infomercials
• Advertisements
You must only rely on refereed papers in
accredited journals and conferences.

How can you tell?
• Length of title
• References (and their quality)
• Author’s name and affiliation
• Evidence that the paper has been reviewed and
revised.
• Date of submission & date of publication.
• The paper includes a review of previously
published work.
• Conclusion contains a critical reflection on the
contents of the article.

Activity
• Use http://scholar.google.co.id/ and enter
the key words from the paper you have.
• Did you find it?
• What else did you find?

“Next step” research
• Incremental advance compared to
paradigm shift.
• Lateral translation research.

Topic 2
Literature Search

Literature Review
• Who has done what and how?
• What is their plan for “further work”?
• Have they reported more recent work in a
conference?
• What opportunities are available for
confirming the results of others and
expanding their results and conclusions?

Key Words
• Essential for searching the literature.
• Must be both general and specific.

Publication delays
• Check your paper and see the submission
date and the publication date.
• This delay may mean that this team has
moved forward with their research.
Following their suggestions for further
work might have you gazumped.
• Conferences often have a 6 month delay
between abstract submission and the
conference presentation.

Planning for an outcome
• What is convincing “proof”?
• What is the evidence you will provide?
– Data
– Sampling techniques
– Accuracy.
• Who is interested in this research?
• Where will you release (publish/present)
your research results?

Anticipating problems
• Team planning meetings
– Circulate outcomes immediately following the
meeting
– Action items
• Equipment calibration
• Reliable power
• Preventing Data loss

Publication of Data
• Internal report?
• Choosing a conference
• Choosing a journal

Journal rankings
• Impact factor
• Half life
• Citations (Google, ISI Thomson Web of
Knowledge, Scopus, etc)
http://scholar.google.co.id/
• Weaknesses of the ranking systems
• H index – The number of papers that have
more than that number of citations fpr
person.

Research Planning
• Concurrent Engineering
– Assembling the equipment
– Arranging access to the site
– Writing the paper draft
– Choosing the journal
• Concurrent Research

Using the right language
• Definition of terms (standards, standard usage,
standard methods of analysis).
• Standard Measurement Procedures
• Standard values (eg copper conductivity)
• Spelling (US English or UK English?), Lexicon
and naming conventions.
• Key words in publications
• This is vital for accurate electronic searching of
indexes.

Searching the Web
• Google scholar http://scholar.google.co.id/
• Journals and publisher’s indexes
– IEEE Xplore digital library
http://ieeexplore.ieee.org/Xplore/dynhome.jsp
– Elsevier
http://www.elsevier.com/wps/find/journal_brow
se.cws_home
– and many more.

IP Searching
• Patents http://www.uspto.gov/
http://www.wipo.int/pctdb/en/search-
adv.jsp
• PCT Applications
http://www.wipo.int/pctdb/en/
• Country Based Searching
http://www.wipo.int/ipdl/en/resources/links.j
sp

Activity
• Find some scientific terms in your paper,
and check the definition. (Why not
wikipedia?)
• Key word searches, key word selection.
• Definition of terms.

Topic 3
Report Writing

The title
• 10-15 words is most common.
• Must be sufficiently specific.

The Abstract – an example
• High speed electronic beam switching is a
desirable feature of smart antennas.

desirable feature of smart antennas. Most
smart antennas are too large for most
applications and require significant power
during normal operations.

during normal operations. A thirteen
element switched parasitic antenna was
optimised for gain, speed and beam
coverage.

during normal operations. A thirteen
element switched parasitic antenna was
optimised for gain, speed and beam
coverage. Antenna characteristics were
determined at 1.8 GHz by finite element
modelling and measurements on a
prototype.

desirable feature of smart antennas. Most smart
antennas are too large for most applications and
require significant power during normal
operations. A thirteen element switched
parasitic antenna was optimised for gain, speed
and beam coverage. Antenna characteristics
were determined at 1.8 GHz by finite element
modelling and measurements on a prototype.
The antenna had a gain of +9.8 dBi, a footprint
of less than one half wavelength squared and
was switched ion less than 100 ms.

desirable feature of smart antennas. Most smart
antennas are too large for most applications and
require significant power during normal
operations. A thirteen element switched
parasitic antenna was optimised for gain, speed
and beam coverage. Antenna characteristics
were determined at 1.8 GHz by finite element
modelling and measurements on a prototype.
The antenna had a gain of +9.8 dBi, a footprint
of less than one half wavelength squared and
was switched ion less than 100 ms. This is a
better performance compared to previous
antennas.

The Abstract – a general guide
• 2 sentences on the wider field – context
and significance.
• 2 sentences on the research method
• 2 sentences on the results and
conclusions.

Scientific writing style
Do’s and Don’ts
• Past tense
• Third person
• Usually timing of events is not included
unless it is essential to data collection.
• Sections and subsections (one level? two
level? three level?).
• Quotes from other authors – not common!

Creating equations
• There are standard symbols for quantities (eg
V=IR).
• There are standard forms for scalar symbols
(often lower case, italics, not-bold) and vector
symbols (upper-case, bold).
• The symbols must be the same font on every
occasion used in the equations and in the main
text.
• All symbols must be defined.
• MS Equation editor allows for equation creation.
• There are standard upper-case and lower-case
type settings.

Data Collection & Presentation

Types of Data
• Quantitative data (numerical)
– Integers (eg animal counts, packets received,
bit error rate)
– Non-integers (eg analog sensor output)
• Qualitative data (descriptive words)
• Binary data (yes/no, success/failure,
present/absent etc)
• Scalar information (1D, 2D, 3D, nD)
• Vector information (1D, 2D, 3D, nD)

Quantitative Data
• Kelvin’s First Law of Measurement: “The
measurement must not alter the event
being measured”.
– Microwave current measurements?
– The impedance of an antenna?

Data Presentation
• Plots (2D and 3D), histograms, pie charts, tables of
numbers.
• Printed papers usually black and white (lines
distinguished by dots, dashes, ellipse, legend etc)
• Colour in power point slides and web publishing.
• For comparison plot more than one data set on the same
graph using the same scale.
• Images and flow charts.
• Interpolation and extrapolation.
• Curve fitting (covered in later lectures)
• Contour plots.

Plotting and analysis tools
• MS EXCEL (Chart Wizard - 4 steps) -
demonstration
• Matlab (plot, subplot, contour, quiver, etc)

Graphing Guidelines
• Always plot discrete points clearly.
• Do not join points unless you have a
continuous mathematical function.
• To compare data plot several lines on the
same axes.
• Consider including error bars on all points

0
0.5
1
1.5
2
2.5
3
3.5
0 2 4 6 8 10 12
Time (secs)
Voltage
(mV)
0
0.5
1
1.5
2
2.5
3
3.5
0 2 4 6 8 10 12
Time (secs)
Voltage
(mV)
0
0.5
1
1.5
2
2.5
3
3.5
0 2 4 6 8 10 12
Time (secs)
Voltage
(mV)
X X

1 2 3 4 5 6 7
1
2
3
4
5
6
7
8
0
2
4
6
8
0
2
4
6
8
10
20
30
40
50
1 2 3 4 5 6 7
1
2
3
4
5
6
7
8
0
2
4
6
8
0
2
4
6
8
10
20
30
40
50
contourf
surf
image
mesh
Matlab scalar 2D plots

quiver
Matlab vector 2D plots
1 2 3 4 5 6 7 8 9 10 11
1
2
3
4
5
6
7
8
East-west (metres)
North-south
(metres)

Qualitative Data
• This can be a challenge as everyone will
use a different description.
• One approach is to convert qualitative
data to quantitative data (eg rate from very
bad to very good on a score of 1 to 10).

Decision Matrix
Vehicle Cost Size Warranty
Delivery
time Comfort
Total
Score
Mazda 3 6 8 7 8 8 37
Mazda 2 8 6 7 7 6 34
Ford
Focus 6 7 7 8 7 35
Honda 6 6 5 6 5 28
Toyota
Camry 4 8 6 7 8 33
VW 2 6 5 3 7 23

Decision Matrix - Histogram
0
5
10
15
20
25
30
35
40
C
o
s
t
S
i
z
e
W
a
r
r
a
n
t
y
D
e
l
i
v
e
r
y
t
i
m
e
C
o
m
f
o
r
t
T
o
t
a
l
S
c
o
r
e
Mazda 3
Mazda 2
Ford Focus
Honda
Toyota Camry
VW
Score
Survey Questions

Data Collection
• Asking the right questions without leading
the person (survey instruments -
questionaires).
• Use redundant questions that always need
a positive response (discussed in a later
lecture).
• Survey results (Is 35% return good
enough?).

Flow Charts (MS Word)
Initiate equipment
Yes/No?
Stop process

Activity
• Plotting analysis using MS eXcel.
• Flow chart using MS word.

Topic 5
Statistical Analysis and Sampling
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North

Normal Distribution
From: http://mathbits.com/MathBits/TISection/Statistics2/normaldistribution.htm

Experimental error?
• How does this compare with your results?
• Is your result significant statistically?

Linear correlation
• Need to fit a line to your data? Quote the
linear correlation coefficient (linear
regression)
y = 0.1199x + 0.2876
R2
= 0.9498
0
0.5
1
1.5
2
2.5
3
0 5 10 15 20
Voltage
Sample

Sampling
• Population – every possible candidate.
• Sample population – a small number of
candidates selected from the population.
• It is impossible to know from an
examination of your sample alone, if your
sample is representative of the whole
population.

Examples:
• In Australia the total population over 18 years votes in
an election.
• Before the election, the press like to take a small
sample the population to estimate the likely outcome of
an election. This is called “polling”.
• They hope that the sample is representative of the
entire population.
• How do they select a representative sample for a
telephone poll?
– Post code?
– Telephone book?
– In the street or shopping centre?
– etc

All samples may be biased
• Why?
– Age?
– Shyness/openness?
– Work times (shift workers)?
– etc

Example
• 6 people live in a single house
• We want to randomly select 2 to get an idea of
the use of mobile phones in the house.
• To do this we could:
visit at 10am on a week day.
visit at 3pm on a week day.
telephone at 8pm on a week day.
visit on Saturday morning at 10am.
Visit on Sunday afternoon at 3pm.
– etc

We ask the question:
• How do you rate your use of a mobile
phone on a scale of 1 to 10?
– 10 means very continuously (>20 hours per
week)
– 1 means never (<30 minutes per week)

We have the following opinions
• Mary stays at home, goes shopping and drives
children to school at 8am and pick up at 3pm.
• Fred drives to work for night shift. Leaves at
7pm and comes home at 6am.
• Asif is a 9am – 5pm office worker who rides the
train.
• Sri is a part time sales person drives around the
city from 10am to 2 pm.
• Chen cycles to University 9am and back at
3pm.
• Rocco is retired and stays in the house all day.
4
2
5
8
7
1
Average value is 4.5/10

How many possibilities?
• If we select 2 people from the total population of n
people we have P combinations where
• ! indicates factorial where 5! = 5x4x3x2x1.
• For a population of 6 we have 15 possibilities.
!
)!
2
(
!
2
n
n
P



There are 15 different combinations
• Lowest result from a sample of two people
would be Rocco and Fred (2 and 1) –
Mean is 1.5/10.
• Highest sample of two would be Sri and
Chen (7 and 8) – Mean is 7.5/10.
• 5 combinations lie between 4 and 5

Compromise required
• The greater the need for a very accurate
result, the smaller the chance of fulfilling
this, even with the best method of
approach.

Sampling Strategies
• Clustered Sampling: Select a sample from only those
parts of the population which are relevant; eg chose only
those people who use the road at peak hour.
• Stratified Sampling: Select a sample proportionally to
those who are likely to use the road at peak hour and
those that don’t. (4/6 use at peak hour and 2/6 don’t, so
use a sample of 3, two who travel at peak hour and one
that does not)
• Destructive Sampling: If the sample is destroyed by
sampling (i.e. their mind is changed), then clearly you
should not sample all people.

Chassis strength testing
• A production line of note book computers
produces 2000 units per day.
• The company is required to strength-test
to failure15 samples every day.
• How do we select those samples?

The Monte-Carlo Method
• A random sampling technique to define the
effect of a large number of parameters on
an outcome. (Usually between 0.1% and
1% of total population).
• Usually applied to complex systems
described by mathematics.
• One randomly selects the parameters and
calculates the outcome.
• Used in optimisation.

Random Sampling
• How can I choose a team of 6 people
randomly from this class?
– Family name?
– Student number?
– Seating location in the class?
– Every third person?
• Every person must have an equal
probability of being chosen.

Random Numbers
1 0.5175 0.2455 0.9670 0.7566 0.3222
6 0.3234 0.0239 0.0048 0.6207 0.3796
11 0.4670 0.0300 0.3014 0.6453 0.6414
16 0.3208 0.8862 0.4546 0.3273 0.6023
21 0.0936 0.8864 0.8905 0.1542 0.0377
26 0.8704 0.9132 0.8435 0.1844 0.3351
31 0.4451 0.5474 0.2504 0.4552 0.0782
36 0.1478 0.1726 0.7339 0.5332 0.5440
41 0.6520 0.4870 0.8396 0.1624 0.4911
46 0.9420 0.8144 0.4230 0.9258 0.2879
51 0.8824 0.9366 0.7085 0.4091 0.2527
56 0.6609 0.5831 0.4059 0.0312 0.4393
61 0.2039 0.5489 0.5263 0.1673 0.6586
66 0.1703 0.4718 0.5256 0.5651 0.3256
71 0.0161 0.7533 0.0915 0.9854 0.0017
76 0.1654 0.3323 0.4037 0.1403 0.9727
81 0.1091 0.1725 0.7821 0.3336 0.1009
86 0.3612 0.5130 0.2648 0.3091 0.3184
91 0.5611 0.3804 0.3079 0.3543 0.9555
96 0.9638 0.8282 0.1850 0.1629 0.3493
Excel function
=rand()

Sample Rate
• Number of samples per second.
• In a digital recording sensor system this might
be obvious initially, but there may be
“overheads” when you need time to send and/or
store data.
• In an analog system this is regulated by the filter
response (eg mechanical needle, DMM update
speed, noise reduction filter).
• Over-sampling and under-sampling.
• Nyquist sampling (twice the maximum frequency
of interest).

Topic 5
Making a Presentation

Preparing a Power Point
Presentation
• Maximum number of slides – one per
minute!
• Optimal number of slides – one per 2
minutes
• Use slides as a reminder of what you will
say.
• During your presentation, do not read what
is on the slides.
• 100 words maximum on each slide.

Preparing a Power Point
Presentation
• Font size? (large!)
• Graphs? (large!)
• Colours? (clearly distinguishable, high contrast,
minimal background colour – not dark)
• Movies? (check on the presentation computer
before your talk – usually they don’t work!)
• Pictures? (not too dark)
• Lighting? (Keep the room lights up so you can
see the audience)

Images
• You MUST acknowledge the source of
image if it is not yours including
– MS word image library (in this presentation)
– Pictures taken from web sites
– Pictures taken from colleagues
– Graphs taken from papers etc

Organisation: 10 minute talk
• Title slide (Name and affiliation) 1
• Outline slide (Major sections) 1
• Introduction (Wider research context) 1
• Main text (method, apparatus, results) 4-6
• Conclusions 1
• References 1

Nervous?
• Hints for overcoming nervousness:
• Memorise the first 2-3 sentences (opening
sentences).
• Make sure you have key words on your
power point to trigger your memory.
• Do not start speaking until the title slide is
visible to the audience.

Being Polite! Before you speak
• Introduce yourself to the session chair
before the session starts.
• Load your presentation before the session
starts.
• Wait for the chair to introduce you before
you speak.
• Switch off your mobile telephone.

Being Polite! During your talk
• Thank the chairperson for the introduction.
• Speak clearly
• Pretend you are talking to the back row of seats
in the room (project your voice).
• Acknowledge your co-authors in Slide 1.
• Rigidly stick to the allocated presentation time.
• Clearly indicate the presentation is finished by a
slide and say “thank you” to the audience.
• Do not invite questions from the audience. (This
is the role of the chair person)

Being Polite! After your talk
• Go quickly back to your seat.
• Do not discuss your paper with others
during the next talk.
• If necessary, leave the room (politely – do
not slam the door).
• Once the session is complete, thank the
chair person.

Why References?
• For scientific rigour.
• In case someone in the audience has
made a major contribution to the field.
• So the audience can follow up on your
previous publications.

Topic 6
Survey Research Methods

• This is about how to prepare and analyse
a survey (questionaire)

“Sick building” Survey
• The research question:
• Do you think that working in this building is
making you feel sick?

Designing a Survey
• Role of the researcher
– Develop the research plan
– Design the survey instrument
– Select the sample population
– Issue/distribute the survey
– Prompt the sample population for responses
– Analyse the data
– Generate conclusions

Who are the stake-holders
• Selecting the sample population
– Who are the stake-holders?
– What’s in it for them? (No interest can mean no
completion)
• Random selection from a large population
• Inclusion –
– Those that are keen to participate will respond
– Are they a biased sample?
• Exclusion
– Will people be offended if they are not asked to
respond?

Who are the stake-holders
• You must be able to defend your sample
population selection

Anonymous Responses
• Arguments for “yes” – Anonymous
– Sample population might be less influenced by who is
asking the questions
– Respondents might be less concerned about others
learning of their opinions
• Arguments for “no” – Non-anonymous
– Who will you send the results to?
– Who will you send the reward (chocolates) to?
– How do you know who to follow up about returning
the survey?

Confidentiality
• You need to ensure that confidentiality is
assured before the survey is sent out.
• Consider using an independent third party
to administer the survey.
• I have been asked to complete a survey
which asked for sufficient personal
information to be identified uniquely.
• How will you report “free” comments?

Feedback
• It is assumed that your sample population
(and the full population) will want access
to the results.
• You must explain how will this be done at
the beginning of the survey.

Sample Time lines
• Week 1:Pre-survey letter of introduction
(Wider research context and brief research
plan)
• Week 2:Survey send out
• Week 3:Mid-survey reminder letter
• Week 4:Last minute final reminder
• Week 6:Post-survey analysis report
completed

Rating system – 5 point scale
• Strongly disagree 1
• Disagree 2
• Neutral 3
• Agree 4
• Strongly agree 5
• Neutral allows respondents to “sit on the
fence”

Rating system – 4 point scale
• Strongly disagree 1
• Disagree 2
• Agree 3
• Strongly agree 4
• This forces respondents to show positive
or negative attitudes.

Topics for “Sick building” survey
• General personal well being
• Lighting
• Ventilation
• Noise and vibration
• Odour
• Electromagnetic radiation
• Security
• Demographics of respondents

Hints for writing questions
• Keep is very simple – avoid jargon
• Use one concept per question – avoid multiple concepts
• Keep wording positive – avoid negative words and
phrases, double negatives
• The first question should be the “over-all question” –
Never place a controversial question at the beginning.
• Place demographics questions at the end –
Demographics at the beginning can raise suspicions.
• Keep related questions together – Difficult for the
respondent to remain coherent
• Use three questions per topic – Do not over question,
don’t waste people’s time.

Statement wording
• I don’t feel well most of the time (negative wording).
• I enjoy good health.
• I am satisfied with the ventilation and the lighting
environment (double-barrelled question).
• I am satisfied with the ventilation.
• I am satisfied with the lighting.
• The University does not do a bad job of keeping us
informed about work place health and safety issues.
(double negative)
• The University does a good job of keeping us informed
about work place health and safety issues.
• Many students feel ill as soon as they walk into the
building. (projecting the feelings of others).
• Students enjoy working in this building.

Judgemental versus Observational
• This work environment is just as good as
other places where I have worked.
• I am happy with this work environment.
• The University listens and acts on student
and staff concerns about the building
environment.
• I am satisfied with the University’s
response to student concerns about the
building environment.

Judgemental versus Observational
• This work environment is just as good as
other places where I have worked.
• I am happy with this work environment.
• What if you asked both statements to be
rated?
• The conclusions would be different

Reverse scoring
• Q10: I am not happy with this work
environment. (1 – 5)
• Q35: I am happy with this work environment. (1
– 5)
• You would need to reverse score Q10 for proper
statistics.
• The dangers include:
– Donkey vote gives confusion (What do you do if you
get 5 for both?)
– Was the question misread?
– Was the respondent annoyed?

Sample Open ended questions
and comments
• Please identify at least three things that
cause you concern in this work
environment.
• Please identify at least three things that
you like about this work environment.

Reporting
• Calculate averages and statistics for each
theme.
• Construct a Histogram and report the mean
value
• E.g. 80% rated the noise environment neutral or
better.
• Or: 20% indicated that the noise environment
was not good.
• Report selective quotes on open questions.

Missing Data
• Did the respondent simply forget one
question?
• Maybe the question was not relevant to
that person?
• Was the question too personal?
• Was the question confusing? Could it have
been scored as a 1 for one interpretation
and a 5 using another interpretation.

Accuracy and Reliability
• On a 5 point scale there are 5 possible
answers.
• Your mean value for the sample population
can be expressed to several decimal
places.
• How many places are significant?
• Return to Normal Distribution statistics
based on z score.

References
• Connolly, P.M. & Connolly, K.G., 2004,
Employee opinion questionaires, Pfeiffer.
• Rosenfeld, P., Edwards, J.E., & Thomas,
M.D., (eds), 1993, Improving
organizational surveys, SAGE Pub.
• Images from MS Word Clip Art.

Review

1. The Research Process
• Independent verification of results.
• Designing the experiment for outcomes
• Journal rankings

2. Literature Search
• Using the web etc

3. Report writing, Data Collection
& Presentation
• Abstract
• Referencing
• Equations
• Figures
• Conclusions and Further work
• Qualitative and quantitative data
• Plotting techniques for multi-dimensional data

4. Statistical Analysis and Sampling
• Regression analysis
• How to select a random sample.

5. Making a Presentation

6. Survey research methods
• How to create and analyse a survey.

Why this presentation?
• To develop an understanding of the
scientific environment in which research is
conducted.

Student Evaluation of Course
• Survey!

Research Methods EE

Recommended

Recommended

More Related Content

Similar to Research Methods EE

Similar to Research Methods EE (20)

Recently uploaded

Recently uploaded (20)

Research Methods EE