The Art and Science of Applied Test Development. This is the fifth in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
Applied Psych Test Design: Part F--Psychometric/technical statistical analysi...Kevin McGrew
The Art and Science of Applied Test Development. This is the fifth in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
Part I: Beyond the CHC tipping point: Back to the futureKevin McGrew
An overview of the CHC (Cattell-Horn-Carroll) theory of intelligence within a historical and "waves of interpretation" context. Presents idea that CHC has reached the "tipping point" in school psychology..and...this is allowing assessment practitioners to realize past attempts to engage in individual strength and weakness interpretation of CHC based test profiles
Applied Psych Test Design: Part F--Psychometric/technical statistical analysi...Kevin McGrew
The Art and Science of Applied Test Development. This is the fifth in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
Part I: Beyond the CHC tipping point: Back to the futureKevin McGrew
An overview of the CHC (Cattell-Horn-Carroll) theory of intelligence within a historical and "waves of interpretation" context. Presents idea that CHC has reached the "tipping point" in school psychology..and...this is allowing assessment practitioners to realize past attempts to engage in individual strength and weakness interpretation of CHC based test profiles
Recently I was required to provide a brief run down of psychometric tests and their applications. There's more than I thought. Hopefully someone else might find this powerpoint useful too.
Applied Psych Test Design: Part A--Planning, development frameworks & domain/...Kevin McGrew
The Art and Science of Applied Test Development. This is the first in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
In this presentation I will show a set of important topics about Software Engineering Empirical Studies that can be useful for increasing quality on your thesis and monographs in general. You can read this presentation and to think about how to do a good experimentation by apply its objectives, validation methods, questions, answers expected, define metrics and measuring it.I will exhibit how the researchers selected the data for avoid case studies in a biased way using a GQM methodology to sort the study in a simpler view as well.
Applied Psych Test Design: Part B - Test and Item DevelopmentKevin McGrew
The Art and Science of Applied Test Development. This is the second in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
Recently I was required to provide a brief run down of psychometric tests and their applications. There's more than I thought. Hopefully someone else might find this powerpoint useful too.
Applied Psych Test Design: Part A--Planning, development frameworks & domain/...Kevin McGrew
The Art and Science of Applied Test Development. This is the first in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
In this presentation I will show a set of important topics about Software Engineering Empirical Studies that can be useful for increasing quality on your thesis and monographs in general. You can read this presentation and to think about how to do a good experimentation by apply its objectives, validation methods, questions, answers expected, define metrics and measuring it.I will exhibit how the researchers selected the data for avoid case studies in a biased way using a GQM methodology to sort the study in a simpler view as well.
Applied Psych Test Design: Part B - Test and Item DevelopmentKevin McGrew
The Art and Science of Applied Test Development. This is the second in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.
The Model of Achievement Competence Motivation (MACM) Part E: Crossing the R...Kevin McGrew
The Model of Achievement Competence Motivation (MACM) is a series of slide modules. This is the fifth (Part E) in the series. It is brief...only 11 slides. The modules will serve as supplemental materials to "The Model of Achievement Competence Motivation (MACM)--Standing on the shoulders of giants" (McGrew, in press, 2021 - in a forthcoming special issue on motivation in the Canadian Journal of School Psychology)
The Model of Achievement Competence Motivation (MACM): Part D: The volition ...Kevin McGrew
The Model of Achievement Competence Motivation (MACM) is a series of slide modules. This is the fourth (Part D) in the series. The modules will serve as supplemental materials to "The Model of Achievement Competence Motivation (MACM)--Standing on the shoulders of giants" (McGrew, in press, 2021 - in a forthcoming special issue on motivation in the Canadian Journal of School Psychology)
The Model of Achievement Competence Motivation (MACM) Part C: The motivation...Kevin McGrew
The Model of Achievement Competence Motivation (MACM) is a series of slide modules. This is the third (Part C) in the series. The modules will serve as supplemental materials to "The Model of Achievement Competence Motivation (MACM)--Standing on the shoulders of giants" (McGrew, in press, 2021 - in a forthcoming special issue on motivation in the Canadian Journal of School Psychology)
The Model of Achievement Competence Motivation (MACM): Part B - An overview ...Kevin McGrew
The Model of Achievement Competence Motivation (MACM) is a series of slide modules. This is the second (Part B) in the series. The modules will serve as supplemental materials to "The Model of Achievement Competence Motivation (MACM)--Standing on the shoulders of giants" (McGrew, in press, 2021 - in a forthcoming special issue on motivation in the Canadian Journal of School Psychology)
The Model of Achievement Competence Motivation (MACM): Part A Introduction o...Kevin McGrew
The Model of Achievement Competence Motivation (MACM) is a series of slide modules. This is the first (Part A) in the series. The modules will serve as supplemental materials to "The Model of Achievement Competence Motivation (MACM)--Standing on the shoulders of giants" (McGrew, in press, 2021 - in a forthcoming special issue on motivation in the Canadian Journal of School Psychology)
The WJ IV Cognitive GIA in iintellectual disability (ID) assessmentKevin McGrew
This is a brief presentation that explains why the WJ IV (and WJ III) GIA IQ score is an appropriate and valid indicator of general intelligence that can be used in possible intellectual disability (ID) determinations
The Evolution of the Cattell-Horn-Carrol (CHC) Theory of Intelligence: Schne...Kevin McGrew
This presentation is based on Schneider, W. J., & McGrew, K. S. (in press). The Cattell-Horn-Carroll Theory of Cognitive Abilities. This presentation includes a portion of key material to be published in a forthcoming CHC update/revision chapter-->In D. P. Flanagan & Erin M .McDonough (Eds.), Contemporary intellectual assessment: Theories, tests and issues (4thed.,) New York: Guilford Press.
This is only a small amount of the chapter. Also, I have inserted some new material related to test interpretation that is not included in the to-be-published chapter. The tentative date for publication of the Flanagan book is spring 2018.
Beyond cognitive abilities: An integrative model of learning-related persona...Kevin McGrew
For centuries educational psychologists have highlighted the importance of "non-cognitive" variables in school learning. The presentation is a "big picture" overview of how cognitive abilities and non-cognitive factors can be integrated into an over-arching conceptual framework. The presentation also illustrates how the big picture framework can be used to conceptualize a number of contemporary "buzz word" initiatives related to building 21st century educationally important skills (social-emotional learning, critical thinking, creativity, complex problem solving, etc.)
What about executive functions and CHC theory: New research for discussionKevin McGrew
This module contains a subset of slides that were only briefly touched on as part of a larger "Beyond CHC" presentation at the Australian Psychological Society (APS) 2016 Annual Congress. Time was limited. Thus, the complete subset of slides are presented here for FYI and discussion purposes.
CHC model of inteligence revised (v2.4). Has Glr been incorrectly conceptual...Kevin McGrew
This presentation contains a historical overview of the derivation of the Glr ability domain in contemporary CHC theory. It then presents new data, as well as historical conclusions of the CHC masters, that makes a strong case for replacing the stratum II broad ability domain of Glr with two separate broad ability domains of Gl (learning efficiency) and Gr (retrieval fluency). How to obtain WJ IV scores for these two broad abilities is presented, as well as other possible Gl and Gr tests indicators from the CHC cross-battery literature.
"intelligent" intelligence testing: Why do some individuals obtain markedly ...Kevin McGrew
This is the second in a series. Please view the first ("intelligent" intelligence testing: Evaluating wihtin CHC domain test score differences) to better appreciate this module
Data and theory-based hypotheses for evaluating differences between scores on the different WJ IV tests of Gwm
A presentation on deciding when the scores from two tests, which are from the same CHC domain (e.g., Gwm), and which may have the same narrow CHC classifications, are different enough to warrant clinical interpretation.
How to evaulate the unusualness (base rate) of WJ IV cluster or test score di...Kevin McGrew
The WJ IV provides two primary methods for comparing tests or cluster scores. One is based on a predictive model (the variation and comparison procedures) and the other allows comparisons of SEM confidence bands, which takes into account each measures reliability. A third method for comparing scores, one that takes into account the correlation between compared measures (ability cohesion model) is not provided, but is frequently used by assessment professionals. The three types of score comparison methods are described and new information, via a "rule of thumb" summary slide and nomograph, are provided to allow WJ IV users to evaluate scores via all three methods.
The WJ IV and Beyond CHC Theory: Kevin McGrew's NASP mini-skills workshopKevin McGrew
This presentation represents the slides Dr. Kevin McGrew presented at his WJ IV and Beyond CHC theory mini-skills workshop at the 2015 NASP convention in Orlando, Florida. The show includes more and newer slides than were presented at the live session.
The WJ IV Measurement of Auditory Processing (Ga)Kevin McGrew
The WJ IV Cognitive and Oral Language include new measures of auditory processing (Ga) that are much more cognitively complex auditory measures of intelligence. This short presentation provides an overview of the WJ IV Ga tests and presents evidence supporting the importance of Ga as a major component of human intelligence.
Overview of the WJ IV Cognitive Battery: GIA and CHC ClustersKevin McGrew
This slideshow provides an overview of the composition of the WJ IV Cognitive clusters. It outlines the design principles used to assign tests to the GIA and CHC clusters, and also presents summary statistics as per the primary design principles used in constructing the COG clusters.
CHC Theory Codebook 1: Cognitive definitionsKevin McGrew
A presentation of the most up-to-date CHC broad and narrow ability definitions as adapted from McGrew & Schneider (2012) and McGrew, LaForte and Schrank (2014). One of two. See CHC Codebook 2 for additional information
Welocme to ViralQR, your best QR code generator.ViralQR
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Securing your Kubernetes cluster_ a step-by-step guide to success !
Applied Psych Test Design: Part G: Psychometric/technical statistical analysis: External
1. The Art and Science of Test Development—Part G
Psychometric/technical statistical analysis: External
Kevin S. McGrew, PhD.
Educational Psychologist
Research Director
Woodcock-Muñoz Foundation
The basic structure and content of this presentation is grounded extensively on the test
development procedures developed by Dr. Richard Woodcock
2. “In god we trust….all others must show data”
(unknown source)
Test authors and
publishers have
standards-based
responsibility to provide
supporting psychometric
technical information re:
tests and battery
Typically in the form of a series of
technical chapters in manual or a
separate technical manual
4. External evidence is
Theoretical Domain - CHC g focused on
relations between
test battery
variables (measures
or latent constructs)
and other
external (outside of
Gf Gv Glr Gs battery)
constructs,
measures, or
criteria
Gc Gsm Ga
Measurement or empirical domain
5. External Stage of Test Development
Purpose Examine the external relations among the focal construct (i.e.,
intelligence or cognitive abilities) and other constructs and/or
subject characteristics
Questions asked Do the focal constructs and observed measures “fit” within a
network of expected construct relations (i.e., the nomological
network)
Method and concepts • Group differentiation
• Structural equation modeling
• Correlation of observed measures with other measures
• Multitrait-Multimethod matrix
Characteristics of • Focal constructs vary in theorized ways with other constructs
strong test validity • Measures of the constructs differentiate existing groups that
program are known to differ on the constructs
• Measures of focal constructs correlate with other validated
measures of the same constructs
• Theory-based hypotheses are supported, particularly when
compared to rival hypotheses
6. External Stage of Test Development
Purpose Examine the external relations among the focal construct (i.e.,
intelligence or cognitive abilities) and other constructs and/or
subject characteristics
Questions asked Do the focal constructs and observed measures “fit” within a
network of expected construct relations (i.e., the nomological
network)
Method and concepts • Correlation of observed measures with other measures
Characteristics of • Measures of focal constructs correlate with other validated
strong test validity measures of the same constructs
program
7. Concurrent external validity example:
WJ III GIA clusters correlations with other IQ
battery full scale scores
Provide evidence at select key age groups (related to intended age range
and purpose of battery) in normal samples
8. Concurrent external validity example:
WJ III Achievement (reading, math, writing) cluster correlations
with measures from other (external) ach. batteries
Provide evidence at select key age groups (related to intended age range and purpose of battery) in normal samples
9. Concurrent external validity example:
Comparative predictive validity (of achievement)
Comparisons of correlations (across reading, math, written language, and total
achievement domains) of the average WJ III GIA and Predicted Achievement score
options and full scale scores from other (external) major intelligence batteries
Other
Battery WJ III WJ III WJ III
Total (Full Pred. GIA- GIA-
Scale) Score Ach. Extended Standard
DAS .41 -- .52 .47
WPPSI-R .37 -- .52 .47
WISC-III .50 .68 .67 .63
WAIS-III .39 .56 -- .56
KAIT .53 .56 -- .56
Provide evidence at select key age groups (related to intended age range
and purpose of battery) in normal samples
10. External Stage of Test Development
Purpose Examine the external relations among the focal construct (i.e.,
intelligence or cognitive abilities) and other constructs and/or
subject characteristics
Questions asked Do the focal constructs and observed measures “fit” within a
network of expected construct relations (i.e., the nomological
network)
Method and concepts • Correlation of observed measures with other measures
Characteristics of • Focal constructs vary in theorized ways with other constructs
strong test validity • Measures of focal constructs correlate with other validated
program measures of the same constructs
11. • Focal constructs vary in theorized ways with other constructs
• Measures correlate with other validated measures of the same constructs
(select illustrative examples—concurrent external validity correlations)
?
12. • Focal constructs
vary in theorized
ways with other
constructs
• Measures
correlate with
other validated
measures of the
same constructs
(select illustrative
example—
exploratory factor
analysis of select
WJ III and WISC-III
tests)
13. • Focal constructs vary in theorized ways with other constructs
• Measures correlate with other validated measures of the same constructs
(select illustrative example—WJ III Block Rotation [Gv-Vz] correlation
with WISC-III tests in grade 3-5 sample)
WJ III
BLKROT
WISC-III Tests
Information 0.27
Coding 0.08
Similarities 0.29
Picture Arangment 0.14
Arithmetic 0.09
Block Design 0.38
Vocabulary 0.23
Object Assembly 0.31
Comprehension 0.15
Symbol Search 0.23
Digit Symbol 0.08
Note: Absolute magnitude of correlations artificially low due to sample range restriction.
Important observation is relative magnitude of correlations
14. • Focal constructs
vary in theorized
ways with other
constructs
• Measures
correlate with
other validated
measures of the
same constructs
(select illustrative
example—
confirmatory factor
analysis of select
WJ III and WISC-III
tests)
Phelps et al. (2005)
WISC-III/WJ III cross-
battery (joint) CFA
15. Phelps et al. (2005) WISC-III/WJ III cross-battery (joint) CFA
17. External Stage of Test Development
Purpose Examine the external relations among the focal construct (i.e.,
intelligence or cognitive abilities) and other constructs and/or
subject characteristics
Questions asked Do the focal constructs and observed measures “fit” within a
network of expected construct relations (i.e., the nomological
network)
Method and concepts • Structural equation modeling
Characteristics of • Theory-based hypotheses are supported, particularly when
strong test validity compared to rival hypotheses
program
20. Structural equation modeling external validity evidence example
f1
r17
Visual Matching .82 r20
.78Mem for Sentences
r18 .64 .44 Mem .69
Decision Speed Gs Span r21
.73 Mem for Words
r19
.62
Cross Out
.46 f9
r22
.62 Aud Working Mem
Work .67
Mem r23
Numbers Reversed
f2
r1
Block Rotation .40
r2 .44 f5 r11
.89
Spatial Relations Verbal Comp
Gv
3
.35
.9
r3
Picture Recognition .78 Oral Comp r12
Gc
.8
f3 .79
5 .87
r4
Memory for Names .52 General Information r13
r5 .48
Retrieval Fluency .93 g f6
.07
r14
r6 .69 Glr .94 .63
Analysis-Synthesis
DR: Vis-Aud Lrng 8
.74
.83
.7 f7 Concept Formation
r15
Gf
.84
r7
Vis-Aud Learning .63
f4
r16
r8 Numerical Reas
Sound Blending .64
.19
r9 .49
.36
Incomplete Words Ga
r10
.45
Sound Patterns .27
.96 Word Attack
r24
WA
Ages 6-8
21. External Stage of Test Development
Purpose Examine the external relations among the focal construct (i.e.,
intelligence or cognitive abilities) and other constructs and/or
subject characteristics
Questions asked Do the focal constructs and observed measures “fit” within a
network of expected construct relations (i.e., the nomological
network)
Method and concepts • Group differentiation
Characteristics of • Measures of the constructs differentiate existing groups that
strong test validity are known to differ on the constructs
program
28. (Note: The following information is almost identical to that presented in Part F—Internal
psychometric/statistical analysis)
Lack of rigor and quality control in all prior/earlier stages will “rattle through the data” and
rear its ugly head when performing the final statistical analysis, especially multivariate
validity analyses (SEM, DF, multiple regression, EFA, CFA)
Shorts cuts in prior stages will “bite you in in the ____” as you attempt to perform final
statistical analysis
Data screening, data screening, data screening!!!!……. prior to do performing final statistical
analysis
• Compute extensive descriptive statistical analysis for all variables (e.g., histograms,
scatterplots, box-whisker plots, etc.)
• More than means and SD’s. Also calculate median, skew, kurtosis, n-tiles, etc.
Deliberately planned and sophisticated “front end” data collection short-cuts (e.g., matrix
sampling) introduce an extreme level of “back end” complexity to routine
statistical/psychometric analysis
Know your limits, level of expertise, and skills. Even those with extensive test development
experience often need access to trusted measurement/statistical consultants
(cont. next slide)
29. Published statistics/psychometric information needs to be based on final publication
length tests
• Often need to use test-length correction formula’s (e.g., KR-21) for test reliabilities
• Correlations between short /and or long norming versions of a test and other tests,
that differ in test length (number of items) from publication length test, may need special
adjustments/corrections.
Back up, back up, back up!!!!!!!!!! Don’t let a dead hard drive or computer destroy your
work and progress. Do it constantly. Build redundancy into your files and people skill
sets
Sad fact: Majority of test users do NOT pay attention to the fancy and special
psychometric/statistical analysis you report in technical chapters or manuals. Be
prepared for post-publication education via other methods.
Post-manual publication technical reports of special/sophisticated analyses are good
when publication time-line pressures dictate making difficult decisions.
30. Most test developers are stuck in a methodological rut. There is much that
can be learned about the internal and external validity of a test battery using
lesser-used statistical methods.
• Multidimensional scaling (MDS); cluster analysis, CART
(classification and regression tree analysis), MARS (multivariate
applied regression splines)
Use of curve smoothing procedures to better estimate population
parameters from statistical analyses across age groups.
Multiple group CFA (planned incomplete data) reference variable validity
designs and methods (Jack McArdle).
31. End of Part G
Additional steps in test development process will be
presented in subsequent modules as they are developed