Aviation Selection Test Battery (ASTB)      22 Nov 10 – 1100 Central Time  Audio Conferencing Dial In Line: 1-866-711-1482...
Overview          Background on ASTB          Current ASTB administration rules          APEX web-based test delivery p...
Question # 1                                 A Testing All Officers Eligibility. What is the purpose of      B Developed ...
Aviation Selection Test Battery (ASTB)        Primary tool used to select student pilots and         student flight offic...
ASTB Administration    Approximately 10,000 ASTBs administered every year    243 permanent custody locations:           ...
Question # 2 The ASTB tests which   A Spatial Apperception  of the following         Test  categories?            B Navig...
Current Structure & Content        The current battery (Forms 3, 4, & 5) consists of six         multiple choice subtests...
ASTB Score Components        Officer Aptitude Rating (OAR)           • MST, RCT, MCT           • Predicts academic perfor...
ASTB Minimums        USN Pilot:             4/5 (AQR/PFAR)        USN FO:               4/5 (AQR/FOFAR)        USMC Pil...
ASTB Score and Flight School Grades                          SNA AQR and Ground School Grades                             ...
ASTB & Attrition                                                                                         -                ...
ASTB: The Bottom Line        The current ASTB is calculated to save the U.S.         Government $38 Million in training c...
ASTB Administration       Continued success of the ASTB depends on:          1. Security of test materials          2. Adh...
ASTB Security        Per OPNAVINST 1532.1, custodians of ASTB testing material         shall:          • Prohibit access ...
ASTB Security  Loss or Compromise of the ASTB Materials    (1) NAVCRUITDIST personnel shall immediately contact NOMI      ...
Current ASTB Retest Rules                                                                         Student AQR Means by Ret...
ASTB Retest Rules        Examinees cannot replace parts of their scores           • Retests are All or None        An il...
Test Merges        If an examinee takes the OAR (first three subtests of the         ASTB) only, then decides s/he wants ...
ASTB Scoring        All ASTB scoring conducted by NOMI        Mail completed tests to NOMI           • Include your e-ma...
APEX.NET – Web-Based ASTB        APEX is a web-based platform for secure online test         administration        Deplo...
How does APEX work?          System requirements are minimal          Requires a high-speed internet           connectio...
 Release of ASTB-E08/29/12                     22
Future Score Components   NOMI delivered a major ASTB revision     • ASTB-E   Current ASTB yields the following scores: ...
Future Structure & Content    ASTB-E will consist of seven sections:      • (1) Math Skills Test (MST)      • (2) Reading...
Naval Aviation Trait Facet Inventory (NATFI)  NOMI has identified the trait scores that lead to success in aviation   tra...
What do these answers mean?         Examinees establish a pattern of responses          regarding each trait.           •...
Performance-Based Measure (PBM)                             •     Presents a unique string of numbers & letters Measure o...
Biographical Inventory        A questionnaire of background experiences relevant to         aviation        Self-proctor...
 ASTB-E Administration Rules08/29/12                               29
ASTB-E Administration     Parts of the ASTB-E can be administered in booklet      format (for when computers are unavaila...
ASTB-E Administration Rules        Lifetime two-attempt limit at ASTB-E          • Previous attempts at Forms 3, 4, & 5 d...
ASTB-E Administration Rules        180 day delay required before ASTB-E retest        OAR Merges are still permissible w...
BI-RV Administration        Administration time for MST, MCT, RCT, ANI, NATFI,         & PBM is approximately 2:45       ...
ASTB-E Merge Scenario 1           • (1) Math Skills Test (MST)           • (2) Reading Comprehension Test        John Smit...
ASTB-E Merge Scenario 2                                                   Joe Sixpack takes           (1) Math Skills Test...
ASTB-E Retest/Merge Scenario 3                                                      Jake Scully takes Full           (1) M...
ASTB-E Resume Scenario 4           (1) Math Skills Test (MST)                                                 Jane Goodall...
ASTB-E Retest Scenario 5                                               Bob Dooley takes the first six           (1) Math S...
The Gouge        If this seems confusing, just use APEX for all ASTB-E         administrations.        APEX will not all...
Computer-Adaptive Tests                    Computer Adaptive Tests   Computer adaptive tests (CATs) tailor items given to...
Will the ASTB-E Improve Diversity?      Existing ASTB subtests: similar to the SAT,       ACT, MCAT, etc.        • Cognit...
Fake-Resistant Engine        The engine is software that delivers adaptive forced-         choice personality tests      ...
Examinees will lie if they can. Historically, the biggest problem with personality  measurements is that they can be fake...
 The ASTB-E engine uses a new capability: forced-choice items where each  statement is equally “desirable”.   • Not previ...
How do new ASTB-E measures improve                              prediction? Cognitive ability can predict a big  chunk of...
ASTB Information Web Site        http://www.med.navy.mil/sites/navmedmpte/nomi/nami        Hosted on NOMI website for In...
Summary Background on ASTB Current ASTB administration rules APEX web-based test delivery platform Release of ASTB-E  ...
Questions?                          Contact:                       NCC Carlin                    Jason.carlin@navy.mil    ...
Upcoming SlideShare
Loading in …5
×

Astb homework 1

4,890 views

Published on

Published in: Business, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,890
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Welcome to N7’s Webinar PQS Training Series. I am NCC Carlin from N7 Training Department, National Training Team. I will be providing training on the ASTB. This training will last about 1 hour. Before we start, we have some attendee rules for you: Put your phone on mute – if you do not have a mute button press *6 to toggle mute on and off. Do not put your phone on hold – we will hear you background music and it will be disruptive to attendees. If you have a question, ask it when it pops in your mind. We will be asking you questions; when the slide displays, I will bring up an electronic popup box to record you answers. You can move the popup box by putting your cursor on the top line and dragging it to another location. Please use the Status Symbols on the Attendee List to indicate if you have to step away from your computer. If Senior calls on you and you don’t answer, we don’t want to have to flunk you for this course. For those of you who are enrolled, a Training Syllabus will be emailed to all NRD XOs/CTs for inclusion in your training jacket. If you attended with another person or a conference group, pls send a muster sheet to sandra.ogden@navy.mil to ensure you get a training syllabus. One of the perks of attending a PQS webinar is you get to sign off the discussion portion of your PQS. Copies of this training can be request copies from me at NRC N72; contact information is on the last slide.
  • a. Testing Overview . The Aviation Selection Test Battery (ASTB) was developed by statistical methods to predict the success of students in aviation officer training programs. The Navy, Marine Corps, and Coast Guard use the complete test battery as a primary selection instrument for aviation programs.
  • The ASTB is available both in hardcopy and online formats. The online version is available via the APEX.NET platform and is only for use at NAVCRUITDIST and NORS locations meeting minimum connectivity requirements.    Applicants for Officer Candidate School, as well as for various direct appointment programs, will be administered applicable portions of the ASTB either via hardcopy testing or online testing through APEX.NET.    The ASTB should be given in its entirety to all applicants who indicate an interest in aviation and intelligence programs.
  • The ASTB is divided into six parts: Math Skills Test (MST), Reading Comprehension Test (RCT), Mechanical Comprehension Test (MCT), Spatial Apperception Test (SAT), Aviation & Nautical Information Test (ANIT), Aviation Supplemental Test (AST). From these six parts, four scores are derived: Academic Qualification Rating (AQR), Pilot Flight Aptitude Rating (PFAR), Flight Officer Aptitude Rating (FOFAR), and Officer Aptitude Rating (OAR).   
  • (3) The Officer Aptitude Rating (OAR) portion of the test, the MST, RCT and MCT sections, may be given to individuals who are not interested/not qualified for aviation and intelligence programs. This score is applicable only to various non-aviation designators as specified in the Program Authorizations, however, Intelligence Program applicants are required to take the entire test. An applicant who has taken the MST, RCT and MCT only to obtain an OAR score may subsequently take the SAT, ANIT and AST within 90 days of taking the MST, RCT and MCT to obtain a complete set of scores. This is a merged test and is considered to be a single set of ASTB scores. Scores are derived from combinations of the subtests AQR, PFAR, FOFAR – standardized and reported as stanines (range from 1-9) OAR – t-scores ranging from 20 to 80
  • Analyses describe aviation training exits for the interval 2004 through Dec 2008
  • d. ASTB Administration . The NOMI (Code 341) generates all procedures governing the administration, handling, accountability, scoring and safeguards for the ASTB.    (1) All personnel responsible for ASTB test administration and custody shall have read and be thoroughly familiar with the references for administration, handling, accountability, and safeguards of the ASTB. These references can be obtained from NOMI (Code 0341) at commercial 850-452-2257x1060, or DSN 922-2257x1060. The references for these procedures are:     Examiner’s Manual for The United States Navy and Marine Corps Aviation Selection Tests (NOMI P-5098C) and      APEX.NET Version 1.1 Examiner’s Guide (NOMI P-5098C Part II).    (2) Handling of all testing materials, completed answer sheets and APEX.NET Login ID and passwords shall be limited to personnel designated in writing as either Test Control Officers, (Exhibit 040201) , Officer Programs Test Administrators, (Exhibit 040202) or personnel authorized sub-custody of ASTB material, (Exhibit 040203). Personnel authorized custody or sub-custody use of APEX.NET by their NAVCRUITDIST Commanding Officer should contact NOMI (Code 0341) at commercial 850-452-2257x1060, or DSN 922-2257x1060 to request Login IDs and passwords for APEX.NET.
  • e. ASTB Accountability . Those activities authorized by NOMI to hold and administer testing materials have certain specific responsibilities. Any designated individual accepting custody of the testing materials specifically accepts the responsibility for the following:    Security . Test booklets and filled in test answer sheets are classified "FOR OFFICIAL USE ONLY". These materials shall be handled in all respects as to transmission, use, stowage, and control of dissemination in accordance with current security instructions.    (2) Maintenance of an adequate supply of testing materials and mailing materials. (3) Exact conformity to the procedures.    (4) Observance of the limitations concerning discussion of test results with examinees. f. Sub-Custody of ASTB   (1) Shall be designated in writing by the NAVCRUITDIST Commanding Officer only after the TCO has verified the adequacy of security provisions, testing facilities, and test administration personnel.    (2) Sub-custody of ASTB testing materials to non-recruiting activities, including ROTC units, is prohibited. These activities shall request ASTB testing materials through NOMI.  g. Handling of ASTB Materials   (1) Only personnel designated in writing by the Commanding Officer are authorized to handle testing materials and completed answer sheets.    (2) At a minimum, all testing materials including APEX.NET Login ID and password (if written down) shall be stowed in a safe or cabinets (preferably metal) secured by combination locks. Where feasible, cabinets should be in secured spaces.    (3) Hard copy ASTB test material may be used during travel, however a system of sub-custody shall be established for all testing materials. Documentation of testing material taken on road trips shall be made in a Test Control Log, and such material shall be subject to all routine and specific inventory requirements. Appropriate security measures shall be taken to preclude loss or compromise. Testing material shall not be stowed in unattended automobiles unless the automobile is locked and testing material is placed out of view, preferably in the trunk. Testing material will be kept in a distinctive container or folder to distinguish it from other papers.   (4) Test booklets shall be inventoried as follows:    (a) Test booklets used for a testing session shall be inventoried and page checked by the Test Administrator before and after that testing session.     (b) Test booklets shall be inventoried (but not page checked) when the Test Administrator or TCO opens the safe/cabinet for purposes other than test administration.    (c) A monthly inventory and page check of all test booklets shall be conducted at least once during each calendar month. Inventories of individual testing booklets used for a test session do not meet this requirement. See Article 040201.    (d) Results of the above inventories, including notation of sub-custody material, shall be recorded in a testing materials log and signed by the test administrator conducting the inventory. The monthly inventory and page check shall be specifically marked as such in the Test Control Log. h. Applicant Counseling Limitations   (1) Officer programs recruiting, processing, and testing personnel are prohibited from disclosing any information concerning the tests other than a general overview.    (2) Recruiters may advise applicants to review standard study guides for military officer candidate programs, which are available in most libraries and commercial bookstores.
  • Loss or Compromise of the ASTB    (1) Upon discovery of loss or compromise of ASTB materials or APEX.NET Login IDs or passwords at Recruiting Command activities, NAVCRUITDIST personnel shall immediately contact NOMI (Code 0341), commercial 850-452-2257x1060, or DSN 922-2257x1060, and NAVCRUITCOM (N311).   (2) Commanding Officers shall direct an investigation of the circumstances surrounding the loss or compromise of testing material, and upon completion, provide a report to NAVCRUITCOM (N311) and (00IG) via the appropriate chain of command.
  • Retest means to test a person for a second time with either a portion or complete ASTB examination. Any retest must be administered using a version other than that used for the previous test.    The first retest is authorized after 30 days has elapsed from the date of the initial test (testing to occur no earlier than the 31st day).   A second (and final) retest is authorized no earlier than 90 days from the last test (testing to occur no earlier than the 91st day). A lifetime limit of three ASTB administrations is mandated, a third retest is prohibited.    Note: Only ASTB forms three, four, and five count towards the lifetime three-test limit. Previous administrations of ASTB forms (one and two) do not count toward the lifetime three-test limit.    (4) In the case of a merged test, the next legal test date is determined by the 30 day or 90 day period, as appropriate, counted from the date of the last portion taken.     (5) Applicants who have taken the entire test and who are being retested must take the entire test battery again even though they may be satisfied with their performance on certain sections. An applicant’s official score will be derived from his or her most recent legal test.     (6) In order to determine retest eligibility, the examiner must query the applicant as to whether he/she has ever taken this test before, and if so, when and for what program. If any doubt exists concerning an applicant’s eligibility to re-take the ASTB, contact Naval Operational Medicine Institute (NOMI) (Code 341) before testing the individual. If an illegal testing is discovered, the test sheet shall be forwarded to NOMI (Code 341) immediately under a separate cover from normal test answer sheet transmittals.
  • Scoring the ASTB   (1) Local scoring of the ASTB and OAR is prohibited.    (2) Completed answer sheets shall be forwarded to NOMI per the above references.    (3) For ASTB tests administered via APEX.NET, immediate official scoring is available and can be printed for the applicant and the command.    (4) The official score sheet from NOMI, or the Commanding Officer’s Letter printed from APEX.NET, shall be included in the officer application as required by the appropriate officer application checklist.
  • * Recall we gave these tests to ~500 student aviators and plan to administer it to a sample of fleet JOs and PXOs.
  • Developed to primarily predict attrition The BI first appeared in the original Aviation Selection Test Battery (ASTB), implemented in 1942 BI was a biodata instrument developed using industry standards at each revision It was revised in 1953 ,1971, and 1994 The 1994 version was developed over the period 1986-1994 to accompany an ASTB Revision completed in 1992 1994 version was used from 1994-2002 Two scoring keys developed (i.e., two sets of ‘right’ answers) and keyed as the: Pilot Biographical Inventory (PBI) & Flight Officer Bio Inventory (FOBI) Empirically keyed PBI had a reported 1994 biserial r = .10 with training attrition PBI & FOBI validity coefficients steadily declined through 2001 PBI zero-order r = .05 in 2001 Arnold (2001) found the PBI had zero incremental validity for the prediction of attrition beyond that yielded by AQR & PFAR Insufficient justification for its retention in a selection instrument resulting in Adverse Impact per 41 CFR 60.3 Why did this happen? Empirically keyed instrument validities tend to decrease over time Test administrators had an interest in test outcome Information about the test available online gradually increased Changes in the applicant and SNA populations over time Differences in scores by accession source provided a first clue as to what factors underlie PBI/FOBI scores Examination of item scoring and response frequencies confirmed that the 1994 BI forms could have been replaced by a single item: Did you attend the Naval Academy?
  • In summary, we have covered in depth these items of discussion. ---Explain SLIDE--- By taking part in this training, I hope you have a better understanding of the items that we covered today. .
  • Here is our contact information. If you need any other training needs or just have questions, please feel free to contact us at N7 where your training and development is our top priority. Thank you for your participation and you all have an outstanding day!
  • Astb homework 1

    1. 1. Aviation Selection Test Battery (ASTB) 22 Nov 10 – 1100 Central Time Audio Conferencing Dial In Line: 1-866-711-1482 Participant Passcode: 7418049 Please, be considerate of other participants…Use your MUTE button (*6 if you don’t have a Mute button)
    2. 2. Overview  Background on ASTB  Current ASTB administration rules  APEX web-based test delivery platform  Release of ASTB-E • New content • New administration rules08/29/12 2
    3. 3. Question # 1 A Testing All Officers Eligibility. What is the purpose of B Developed by statistical methods to predict the success of students in aviation ASTB? officer training programs. C Testing for applicants to join the JAG Program. D. Identifying the trends of all Officer candidates. Answer: B 3
    4. 4. Aviation Selection Test Battery (ASTB)  Primary tool used to select student pilots and student flight officers for the USN, USMC, and USCG  Sections of the battery are also used to select candidates for OCS  ASTB was first implemented in 1942, with revisions in 1953, 1971, and 1992  In July 2004, three new forms (Forms 3, 4, and 5) were released to address a test compromise08/29/12 4
    5. 5. ASTB Administration  Approximately 10,000 ASTBs administered every year  243 permanent custody locations: • OSOs, NRDs, NROTCs, Military Institutes, selected NAS and MCAS in CONUS and overseas  Available in paper/pencil and secure web-based format via APEX platform  On-line delivery system for the ASTB (APEX) is in use at over 100 sites CONUS and OCONUS08/29/12 5
    6. 6. Question # 2 The ASTB tests which A Spatial Apperception of the following Test categories? B Navigational Skills C Vision D Essay Writing Answer: A 6
    7. 7. Current Structure & Content  The current battery (Forms 3, 4, & 5) consists of six multiple choice subtests • Math Skills Test • Reading Comprehension Test • Mechanical Comprehension Test • Spatial Apperception Test • Aviation & Nautical Information Test • Aviation Supplemental Test08/29/12 7
    8. 8. ASTB Score Components  Officer Aptitude Rating (OAR) • MST, RCT, MCT • Predicts academic performance in OCS • Range 20-80 (higher is better)  Academic Qualifications Rating (AQR) • Predictor of performance in academic portions of API and ground school and attrition • Range 1-9 (higher is better)  Pilot & Flight Officer Flight Aptitude Ratings (PFAR & FOFAR) • Predictors of flight grades in primary flight training and attrition • Range 1-9 (higher is better)08/29/12 8
    9. 9. ASTB Minimums  USN Pilot: 4/5 (AQR/PFAR)  USN FO: 4/5 (AQR/FOFAR)  USMC Pilot: 4/6 (AQR/PFAR)  USMC ECMO/RIO: 4/6 (AQR/FOFAR)  USCG Pilot: 4/5 (AQR/PFAR) Minimums specified in PAs 106, 107, 150D, & 150E08/29/12 9
    10. 10. ASTB Score and Flight School Grades SNA AQR and Ground School Grades SNA PFAR and Flight Grades 80 80 70 70 3 4 Primary Flight NSS 60 4 60 5 5 API NSS 6 50 6 50 7 7 8 40 8 40 9 9 30 30 20 20 USN USMC USN USMC Service Service SNFO AQR and Ground School Grades SNFO FOFAR and Flight Grades 80 80 70 70 3 4 Primary Flight NSS 60 4 60 5 5 API NSS 6 50 6 50 7 7 8 40 8 40 9 9 30 30 20 20 USN USMC USN USMC Service Service NOTE: Error bars represent one standard deviation.08/29/12 10
    11. 11. ASTB & Attrition - Current ASTB does an AQR and SNA Attrition 45% 40% 35% excellent job of 30% predicting % Attrition 25% USN whether USN USNAs & 20% USMC 15% USNFOs will attrite from 10% 5% training 0% 3 4 5 6 7 8 9 AQR PFAR and SNA Attrition FOFAR and Attrition for SNFOs 35% 35% 30% 30% 25% 25% % Attrition % Attrition 20% 20% USN USN USMC USMC 15% 15% 10% 10% 5% 5% 0% 0% 4 5 6 7 8 9 4 5 6 7 8 9 PFAR FOFAR08/29/12 11
    12. 12. ASTB: The Bottom Line  The current ASTB is calculated to save the U.S. Government $38 Million in training costs annually08/29/12 12
    13. 13. ASTB Administration Continued success of the ASTB depends on: 1. Security of test materials 2. Adherence to test procedures 3. Timely processing of completed tests08/29/12 13
    14. 14. ASTB Security  Per OPNAVINST 1532.1, custodians of ASTB testing material shall: • Prohibit access to test materials by personnel who may later wish to take the test • Prevent disclosure of any information about the test and test materials • Perform strict accounting of all test materials before and after each test administration, including page checks of all booklets • NEVER give information to applicants about specific items on any subtests, or right/wrong answers for any test questions08/29/12 14
    15. 15. ASTB Security Loss or Compromise of the ASTB Materials   (1) NAVCRUITDIST personnel shall immediately contact NOMI (Code 0341), commercial 850-452-2257x1060, or DSN 922-2257x1060, and NAVCRUITCOM (N311).    (2) Commanding Officers (a.) Direct an investigation of the circumstances surrounding the loss or compromise of testing material (b.) Provide a report to NAVCRUITCOM (N311) and (00IG) via the appropriate chain of command.08/29/12 15
    16. 16. Current ASTB Retest Rules Student AQR Means by Retakes  Limit of 3 administrations • One shot at forms 3, 4, & 5 100 9 90 8 80 5.91 7  Only the most recent score 70 % Attrition 5.32 60 4.97 4.96 4.93 6 50 5 counts 40 30 3.80 4 3 20  Mandatory delay between 10 0 21.6 26.9 33.2 30.4 57.1 60 2 1 retests 1 N=5954 2 N=2494 3 N=413 4 N=92 5 N=14 6 N=5 Test Count & Frequency • 30 days between first and second test • 90 days between second and third test  No waivers on retest intervals, so older examinees should plan accordingly08/29/12 16
    17. 17. ASTB Retest Rules  Examinees cannot replace parts of their scores • Retests are All or None  An illegal test (wrong form/too early) counts against an examinee’s limit even if not the examinee’s fault  Score waiver requests should go to PERS-43508/29/12 17
    18. 18. Test Merges  If an examinee takes the OAR (first three subtests of the ASTB) only, then decides s/he wants to be an aviator:  Within 90 days, examinee can take the 3 remaining subtests • does not count as an extra test against examinee’s limit • If OAR given on paper, 2nd half web-based admin requires assistance from NOMI  30/90 day retest clock starts on day of the 2nd half administration08/29/12 18
    19. 19. ASTB Scoring  All ASTB scoring conducted by NOMI  Mail completed tests to NOMI • Include your e-mail address on the transmittal form so that the scores can be returned to you as quickly as possible.  Web-based ASTBs are scored automatically and immediately • Scores available for display and printing as soon as the test is complete08/29/12 19
    20. 20. APEX.NET – Web-Based ASTB  APEX is a web-based platform for secure online test administration  Deployable to all permanent ASTB test sites  System administrator issues user ID, user password, and workstation ID  APEX also available on a temporary basis to non- permanent sites • Successfully used OCONUS & Shipboard08/29/12 20
    21. 21. How does APEX work?  System requirements are minimal  Requires a high-speed internet connection  Installation requires admin user privileges  Installs a small piece of client software on the local machine  Examiner account granted upon verification of requestor ID  The application runs on the server  Test content is downloaded one item or test segment at a time, and erased upon completion of that item or segment  Official score letters can be printed immediately upon test completion08/29/12 21
    22. 22.  Release of ASTB-E08/29/12 22
    23. 23. Future Score Components  NOMI delivered a major ASTB revision • ASTB-E  Current ASTB yields the following scores: Range: • Academic Qualification Rating (AQR) 1-9 • Pilot Flight Aptitude Rating (PFAR) 1-9 • Flight Officer Flight Aptitude Rating (FOFAR) 1-9 • Officer Aptitude Rating (OAR) 20-80  ASTB-E adds these new components: • Pilot Aviation Fit (PAF) 1-9 • Flight Officer Aviation Fit (FOAF) 1-908/29/12 23
    24. 24. Future Structure & Content ASTB-E will consist of seven sections: • (1) Math Skills Test (MST) • (2) Reading Comprehension Test (RCT) • 3) Mechanical Comprehension Test (MCT) • (4) Aviation & Nautical Information Test (ANI) New • (5) Naval Aviation Trait Facet Inventory (NATFI) Content • (6) Performance Based Measures (PBM) • (7) Biographical Inventory with Response Verification (BI-RV)08/29/12 24
    25. 25. Naval Aviation Trait Facet Inventory (NATFI)  NOMI has identified the trait scores that lead to success in aviation training, and is currently validating profiles of aviators mid-career, and upon selection for command for use in PAF/FOAF scoring.  The NATFI is a forced-choice personality assessment, consisting of 88 statement pairs: • Some consist of two flattering statements  I am confident in my ability to learn new skills.  I firmly believe that I am capable of excelling in a wide variety of situations. • Some consist of two unflattering statements  I am more of a follower than a leader.  I am almost always running behind schedule.08/29/12 25
    26. 26. What do these answers mean?  Examinees establish a pattern of responses regarding each trait. • No single answer on this test tells us who the examinee really is. • The set of answers tells us where s/he stands on all the traits we target.  Training success data tells us which traits predict success in training, and which are consistent with success in the fleet  Screen out candidates who have dangerously low scores.  Select candidates with those traits and you’ll get lower attrition08/29/12 and more career-level success 26
    27. 27. Performance-Based Measure (PBM) • Presents a unique string of numbers & letters Measure of dexterity simultaneously in each ear and multi-tasking • Examinee focuses on the numbers heard in the skills designated ear “R 8 N S M A 2 G B 7 F “Y L 3 S R 4 F Z 9 X F L 6” 8 G” Tracking Tasks Dichotic Listening Left Right Ear Ear Tracking Tasks08/29/12 27
    28. 28. Biographical Inventory  A questionnaire of background experiences relevant to aviation  Self-proctored online  BI will include Response Verification • Will reduce faking • Will eliminate typo errors as an explanation for incorrect info about experiences  Sample Item:  If you selected a - d, indicate:  1. What was your involvement in  What year(s)? _________ high school sports?  What school(s)? _________  a. Captain of a team  b. Member of a team  c. Participant in an individual sport  d. Worked in a supportive role (e.g., equipment mgr)  e. No involvement in high school sports08/29/12 28
    29. 29.  ASTB-E Administration Rules08/29/12 29
    30. 30. ASTB-E Administration  Parts of the ASTB-E can be administered in booklet format (for when computers are unavailable)  The final two sections must be administered online (1) Math Skills Test (MST) top (2) Reading Comprehension Test (RCT) OAR (3) Mechanical Comprehension Test (MCT) APEX eb-enabled (4) Aviation & Nautical Information Test (ANI) Paper or (5) Naval Aviation Trait Facet Inventory (NATFI) Only (6) Performance Based Measures (PBM) APEX (7) Biographical Inventory with Response Verification (BI-RV)08/29/12 30
    31. 31. ASTB-E Administration Rules  Lifetime two-attempt limit at ASTB-E • Previous attempts at Forms 3, 4, & 5 do not count toward this limit • Forms 3, 4, & 5 will be deactivated on the day that ASTB-E goes operational • Form 3, 4, & 5 scores generated before ASTB-E release will remain valid for a lifetime • Form 3, 4, & 5 scores generated on or after ASTB- E release date will not be valid08/29/12 31
    32. 32. ASTB-E Administration Rules  180 day delay required before ASTB-E retest  OAR Merges are still permissible within 90 days of valid OAR administration  Paper-to-Web Test Resumes are also legal within 90 days of valid paper exam date  Paper booklets will be issued in two forms: Forms 6 &7 • Web-based ASTB-E content has no form numbers • Subtests are administered in adaptive format. • Each examinee gets questions tailored to his/her ability level.08/29/12 32
    33. 33. BI-RV Administration  Administration time for MST, MCT, RCT, ANI, NATFI, & PBM is approximately 2:45  BI-RV may take examinees an additional :45 to 2:00  BI-RV will be self-proctored • During registration, system will generate a letter for the examinee with a username & password for accessing the BI-RV • Examinee has 90 days from pw issue to complete the BI-RV on any computer  Complete ASTB-E scores will not be generated until the BI-RV is completed08/29/12 33
    34. 34. ASTB-E Merge Scenario 1 • (1) Math Skills Test (MST) • (2) Reading Comprehension Test John Smith OAR (RCT) takes OAR on day 0 • (3) Mechanical Comprehension Test (MCT) Day 90, decides he wants to be an aviator • (4) Aviation & Nautical Information - He must complete Test (ANI) ANI, NATFI, & PBM • (5) Naval Aviation Trait Facet that day - Issued a BI-RV pw, Inventory (NATFI) and has 90 more days • (6) Performance Based Measures (i.e., until day 180) to (PBM) complete it • (7) Biographical Inventory with Response Verification (BI-RV)08/29/12 34
    35. 35. ASTB-E Merge Scenario 2 Joe Sixpack takes (1) Math Skills Test (MST) OAR on day 0 (2) Reading Comprehension Test OAR (RCT) If on Day 22, he (3) Mechanical Comprehension Test decides he wants to be (MCT) an aviator, how many (4) Aviation & Nautical Information days does he have to Test (ANI) complete the ANI, (5) Naval Aviation Trait Facet NATFI, & PBM? Inventory (NATFI) (6) Performance Based Measures If he takes ANI, NATFI, (PBM) & PBM on Day 45, how many days does (7) Biographical Inventory with he have to complete Response Verification (BI-RV) the BI-RV?08/29/12 35
    36. 36. ASTB-E Retest/Merge Scenario 3 Jake Scully takes Full (1) Math Skills Test (MST) ASTB-E on Day 0OAR (2) Reading Comprehension Test (RCT) (3) Mechanical Comprehension Test He retakes OAR on (MCT) Day 181. Is this legal? (4) Aviation & Nautical Information Day 268, decides he Test (ANI) wants to be an aviator (5) Naval Aviation Trait Facet Inventory after all (NATFI) - By what day must he (6) Performance Based Measures (PBM) complete ANI, NATFI, & PBM ? (7) Biographical Inventory with If he takes ANI, NATFI, Response Verification (BI-RV) & PBM on Day 269 - By what day must he complete the BI-RV?08/29/12 36
    37. 37. ASTB-E Resume Scenario 4 (1) Math Skills Test (MST) Jane Goodall wants to be an (2) Reading Comprehension Test aviator, but has no access to (RCT) OAR APEX. (3) Mechanical Comprehension Test Her examiner gives her the (MCT) ASTB-E booklet Form 6 (4) Aviation & Nautical Information (MST, RCT, MCT, ANI, Test (ANI) NATFI) on Day 0. (5) Naval Aviation Trait Facet Inventory (NATFI) Her scantron is received by NOMI & scored on Day 17. (6) Performance Based Measures (PBM) By what day does she have (7) Biographical Inventory with to complete the PBM? Response Verification (BI-RV) If she completes the PBM on Day 78, how long does she have to complete the BI-RV?08/29/12 37
    38. 38. ASTB-E Retest Scenario 5 Bob Dooley takes the first six (1) Math Skills Test (MST) ASTB-E subtests online on DayOAR 0, and is issued a BI-RV pw. (2) Reading Comprehension Test (RCT) He completes the BI-RV on Day (3) Mechanical Comprehension 88. Test (MCT) This becomes his completion (4) Aviation & Nautical date of record. Information Test (ANI) When is he eligible to reattempt (5) Naval Aviation Trait Facet the ASTB-E? Inventory (NATFI) He attempts ASTB-E Booklet (6) Performance Based Measures Form 7 on Day 193. (PBM) Upon receipt by NOMI, this is (7) Biographical Inventory with recorded as an illegal retest, Response Verification (BI- and Bob is eligible for no more RV) attempts. His Day 88 score remains his ASTB-E score of08/29/12 record. 38
    39. 39. The Gouge  If this seems confusing, just use APEX for all ASTB-E administrations.  APEX will not allow you to give an illegal exam.  APEX will allow you to look up an examinee’s complete test history, including paper tests attempted.08/29/12 39
    40. 40. Computer-Adaptive Tests Computer Adaptive Tests  Computer adaptive tests (CATs) tailor items given to an examinee to that examinee’s ability level • CAT provides more accurate measurement using fewer questions  Four existing subtests will be administered in CAT format in the revised ASTB  CAT also provides improved test ICC Plots for 3-PL For m 3 1.0 security – no two test forms are 0.9 0.8 alike 0.7 P(U=1|THETA) 0.6 0.5 0.4 0.3 0.2 0.1 0.0 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 308/29/12 THETA 40
    41. 41. Will the ASTB-E Improve Diversity?  Existing ASTB subtests: similar to the SAT, ACT, MCAT, etc. • Cognitive ability tests have Compensatory Scoring historically been the best predictors of job/task performance • Have been used because they are easy to measure  Title VII protected groups score lower on these cognitive ability tests  Title VII protected groups DO NOT score as low on personality & psychomotor tests • These are also good predictors of job/task performance • These have historically been difficult to measure  Adding these measures to the ASTB (not previously possible) is expected to improve minority pass rates without reducing predictive power08/29/12 41
    42. 42. Fake-Resistant Engine  The engine is software that delivers adaptive forced- choice personality tests • The statements you see later depend on what you Personality Traits said earlier • You see two statements at a time, and must choose the one that describes you better • The statements are always equally “desirable”  Faking answers on this engine is as hard as counting into a seven-deck shoe  The next 2 slides illustrate what’s so new about this08/29/12 42
    43. 43. Examinees will lie if they can. Historically, the biggest problem with personality measurements is that they can be faked: These Likert-scale types of items frequently have obviously “correct” answers. Historically, the biggest problem with personality measurements is that they can be faked: Personality Traits Aggressivenes Adaptability s  These Likert-scale types of items frequently have obviously “correct” answers.08/29/12 43
    44. 44.  The ASTB-E engine uses a new capability: forced-choice items where each statement is equally “desirable”. • Not previously possible due to math limitations of existing models Work Motivation Personality Traits Confidence Unrestraint Warmth08/29/12 44
    45. 45. How do new ASTB-E measures improve prediction? Cognitive ability can predict a big chunk of training performance by itself, which is what the ASTB already gives us Psychomotor ability also predicts a Cognitive lot of training variability, but note Ability the overlap with cog ability Training• This area represents the extra Psychomotor Performance predictive power we get from Ability psychomotor ability – good, but not Aviation- enough. Relevant Traits• An aviation trait test captures a Biographical great deal of training performance, Experiences with minimal overlap from other measures – which is what you want.08/29/12 45
    46. 46. ASTB Information Web Site  http://www.med.navy.mil/sites/navmedmpte/nomi/nami  Hosted on NOMI website for Internet access and linking by CNRC, ROTC, USNA etc.  Provides access to: • ASTB Study Guide w/ Sample Items • APEX Users Guide w/ Screen Shots • ASTB & APEX FAQ list08/29/12 46
    47. 47. Summary Background on ASTB Current ASTB administration rules APEX web-based test delivery platform Release of ASTB-E •New content •New administration rules08/29/12 47
    48. 48. Questions? Contact: NCC Carlin Jason.carlin@navy.mil (901)874-9297 For copies of this Webinar Training send email to: Jason.carlin@navy.mil08/29/12 48

    ×