http://ccn.aacnjournals.org/cgi/external_ref?link_type=PERMISSIONDIRECT
Personal use only. For copyright permission inform...
http://ccn.aacnjournals.org CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 49
Jean C. Toth is an associate professor of card...
The BKATs have been periodically
updated to maintain current rele-
vance, and thus new versions have
beencreated.Asnewvers...
BKATs were used. These included all
of the following versions, which
were in use from 1994 to 2004 (the
year in parenthese...
the nurse educators expected their
nurses to have after orientation.
Changes Made on the Basis of the
BKATs In the initial...
Almost 90% of the nurses in the
replication study who used the BKATs
in orientation added content. This
finding is differe...
Upcoming SlideShare
Loading in …5
×

Tools (BKATs) for Critical Care Nursing and Effects on Staff ...

3,184 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,184
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Tools (BKATs) for Critical Care Nursing and Effects on Staff ...

  1. 1. http://ccn.aacnjournals.org/cgi/external_ref?link_type=PERMISSIONDIRECT Personal use only. For copyright permission information: http://ccn.aacnjournals.orgPublished online © 2006 American Association of Critical-Care Nurses Crit Care Nurse. 2006;26: 49-53 Jean C. Toth Tools (BKATs) for Critical Care Nursing and Effects on Staff Nurses Follow-up Survey 10 Years Later: Use of the Basic Knowledge Assessment http://ccn.aacnjournals.org/subscriptions Subscription information http://ccn.aacnjournals.org/misc/ifora.shtml Information for authors http://www.editorialmanager.com/ccn Submit a manuscript http://ccn.aacnjournals.org/subscriptions/etoc.shtml Email alerts 949-362-2049. Copyright 2006 by AACN. All rights reserved. 101 Columbia, Aliso Viejo, CA 92656. Telephone: 949-362-2000. Fax: Group,Association of Critical-Care Nurses, published bi-monthly by The InnoVision Critical Care Nurse is the official peer-reviewed clinical journal of the American by on October 25, 2010ccn.aacnjournals.orgDownloaded from
  2. 2. http://ccn.aacnjournals.org CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 49 Jean C. Toth is an associate professor of cardiovascular nursing at The Catholic University of America in Washington, DC. shown that this knowledge is beyond that required for licensure as a regis- tered nurse and is essential before entry into critical care nursing prac- tice.2,3 Staff nurses new to critical care typically acquire much of this knowledge during their orientation programs.4(p5) The initial development of the BKAT and its content, validity, and reliability have been described.2 The knowledge of critical care nurses has been monitored since the advent of critical care units, and such monitoring is a hallmark of profes- sional nursing.5 Because the BKATs are widely used, with more than 6000 copies requested from nurse educators/managers working in critical care units and mailed out to them since 1984, it is important for critical care nurses—particularly educators,preceptors,andmanagers —to learn about a test they may want to use in their units. The BKATs The number of questions on the BKATsrangesfrom81to100,depend- ing on which BKAT is used. Content for each BKAT was determined by using a review of the literature, clini- cal expertise, and panels of experts. Construct validity is supported through the panels, learning theory and experimental interventions, replication studies, known group differences, and internal consistency of items.3,5-8 The Cronbach coefficient α was used to measure internal con- sistency reliability for the BKATs and ranges from .81 to .90,2,7 exceeding the minimum recommended relia- bility for classroom tests of .70.9(p421) Jean C. Toth, RN, CNS, DNSc, BCC The Basic Knowledge Assessment Tools (BKATs) are a series of 5 dif- ferent paper-and-pencil tests used to measure basic knowledge in critical care nursing (Table 1). Basic knowl- edge in critical care nursing is defined as the body of knowledge used by critical care nurses to provide safe nursing care to patients hospitalized in critical care units.1,2 Research with undergraduate baccalaureate nursing students and new graduates has Follow-up Survey 10 Years Later: Use of the Basic Knowledge Assessment Tools (BKATs) for Critical Care Nursing and Effects on Staff Nurses StaffDevelopment Author Corresponding author: Jean C. Toth, RN, CNS, DNSc, BCC, School of Nursing, The Catholic University of America, 620 Michigan Ave, NE, Washington, DC 20064 (e-mail: toth@cua.edu). To purchase electronic or print reprints, contact The InnoVision Group, 101 Columbia, Aliso Viejo, CA 92656. Phone, (800) 809-2273 or (949) 362-2050 (ext 532); fax, (949) 362-2049; e-mail, reprints@aacn.org. Table 1 Most recent Basic Knowledge Assessment Tools (BKATs) available Name of test BKAT7 BKAT-5S (BKAT-6S available Fall 2006) PEDS-BKAT5 NICU-BKAT3 ED-BKAT Critical care unit intended for Adult intensive care unit Telemetry/progressive care unit Pediatric intensive care unit Neonatal intensive care unit Adult emergency department A copy of the BKAT can be obtained by writing to Jean C. Toth, RN, CNS, DNSc, BCC, School of Nursing, The Catholic University of America, 620 Michigan Ave, NE, Washington, DC 20064 (e-mail: toth@cua.edu). by on October 25, 2010ccn.aacnjournals.orgDownloaded from
  3. 3. The BKATs have been periodically updated to maintain current rele- vance, and thus new versions have beencreated.Asnewversionsbecome available, older versions are retired, although copies of older versions may still be used. Critical Care Nurses’ Comments on the BKAT Over the years, much correspon- dence about the BKATs has been received from critical care nurses in the United States and in other coun- tries throughout the world. Table 2 lists the comments received from staff nurses and nurse educators about the BKATs. Critical Care Nurses’ Support for BKATs All of the BKATs that are cur- rently available (Table 1) were developed with widespread partici- pation of critical care nurses at all levels of nursing education through- out the United States. This partici- pation included memberships on panels of experts, suggestions for changes in questions and/or answers, and data collection for reliability testing. Use of the BKATs The results of a survey in 1993 indicated that the primary use of the BKAT was in orientation programs conducted in critical care units and that scores on the version of the BKAT usedatthetime(BKAT-3)hadresulted in changes in those programs.1 Other uses include research to test different teaching methods for orientation classes and as a means to identify content for in-service education programs for currently employed critical care nurses.1 In addition, the BKAT has been used for validation of basic knowledge of nurses with previous experience in critical care nursing, so that they could be excused from part or all of orientation classes that present con- tent that they already know. This validation may also mean that a nurse who is new to the unit could be advanced to precepted practice with a senior staff nurse.1 The BKAT Replication Survey Purpose of the Survey The purpose of this study was to replicate survey results published in Critical Care Nurse in 1993.1 The pur- pose of that survey was the same as the purpose of this replication study, that is, to describe how the BKAT is being used and to describe how the BKAT has affected critical care orientation programs.1 The original publication1 of survey results included a recommendation that the survey be repeated so that findings could be compared over time. That comparison is reported here. Methods Instrumentation The same 50- item questionnaire known as the BKAT Utilization Questionnaire (UTool), developed by Dennis1 was used to answer the research ques- tions: (1) How is the BKAT being used and (2) How has critical care orientation been affected? The UTool asked the nurse subjects why they ordered a BKAT (the intended use) and how they used the BKAT (the actual use). Sample and Sampling Technique The population consisted of 300 nurses working in critical care units who had requested and had been mailed a copy of one or more of the BKATs during a period of 20 months. These nurses were con- tacted by mail 6 to 20 months after receiving the BKAT(s). Both older and more recent versions of the 50 CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 http://ccn.aacnjournals.org Table 2 Recent comments from critical care nurses about the Basic Knowledge Assessment Tools (BKATs) Comment [I’m a staff nurse]…and want to obtain a copy of the BKAT test for adult critical care nursing. I have the BKAT-5S (Telemetry) and BKAT-6 (Critical Care)… are these the most current versions? Do you have any competency tools for medical-surgical nursing? I would like to ask permission to use…3 of your BKAT assessment tools…the ICU, telemetry and NICU tools…for assessment purposes during nursing orientation for RNs at our facility. I am a critical care nurse educator…in Castro Valley, California. I came across a copy of the BKAT-5 assessment tool for critical care. I was wondering if there are any other versions of this exam… Please advise…about purchasing the BKAT-6 for adults, peds, and neonates. I am a critical care educator [in] a community hospital in NY. Can you tell me if there is a newer version of the BKAT…and how it is being used in practice?…I have found this to be an excellent evaluation tool. State Texas Illinois Missouri California China New York Date May 2006 June 2005 June 2005 June 2005 April 2006 July 2005 StaffDevelopment by on October 25, 2010ccn.aacnjournals.orgDownloaded from
  4. 4. BKATs were used. These included all of the following versions, which were in use from 1994 to 2004 (the year in parentheses is the year of release for use): • the BKAT for the Adult Inten- siveCareUnit(ICU),Version4(1989), Version 5 (1996), and Version 6 (2001); • the BKAT for Progressive/Inter- mediate Care, Version 4-S (1996) and Version 5-S (2001); and • the Pediatric ICU BKAT (PEDS- BKAT), Version 4 (1996). Fifteen questionnaires were returned undelivered. Of the other 285 ques- tionnaires mailed, the 139 (48.8%) that were completed and returned became the sample. Limitation The main limitation of both the original and replication surveys was that only approximately half of the potential subjects partici- pated. Assumptions The assumptions were the same as for the 1993 survey: • the nurses would answer the UTool honestly, and • the time frame selected to con- tact subjects after they received the BKAT would be long enough to eval- uate its use. Design An ex post facto, 1-group design was again used. Procedure Data were collected on a 1-time-only participation basis. All subjects received a cover letter with an introduction to the survey, the promise of anonymity and confi- dentiality, and an explanation of benefits and the right not to partici- pate. Informed consent was consid- ered obtained if the nurse returned the completed UTool. Each nurse was sent a preaddressed, stamped return envelope. Results Description of the Sample All but 9 of the 139 subjects or 93.5% said that they had used one or more of the BKATs. As in the initial survey, the clinical areas in which these nurses most frequently had respon- sibility were fairly evenly distributed: ICU, 22.3%; ICU-coronary care unit, 16.2%; surgical ICU, 14.5%; coronary care unit, 14.2%; other types of criti- cal care units, 21.9%; and medical ICU, 10.9%. Other types of units included step-down/telemetry units, emergency departments for adults and children, pediatric ICUs, and postanesthesia recovery units. Unlike the original study, in this study a new clinical area of nursing agency emerged that represented 4.6% of the “other” category. Table3comparesthesamplefrom the initial survey with the sample from this replication survey. It shows that the type of institutions in which the nurses in the 2 samples worked remained stable. As reflected in changes in the nursing job market in the past 10 years, more in-service educators and fewer clinical special- ists were administering the BKATs. Also, not unexpected, the frequency of CCRN certification, often tied to the clinical specialist position, had decreased in the replication sample. A new finding was that a larger per- centage of subjects in the replication survey had heard about the BKATs from colleagues rather than from the literature, although publications6,10 about the BKATs had continued. This change reflects networking within critical care nursing. Appropriate to the increase in use of information technology, a new category of Inter- net emerged as a way that nurses in http://ccn.aacnjournals.org CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 51 Table 3 Characteristics of the samples responding to the initial survey and the replication survey Variable Type of institution* Community hospital University hospital Veterans hospital Other Position† Clinical specialist In-service educator Faculty/school of nursing Head nurse/manager Other CCRN certification Yes No Heard about the Basic Knowledge Assessment Tools‡ Journal Colleague(s) Internet Other Frequency 34 10 2 8 20 17 6 6 5 39 15 23 20 0 6 Frequency 97 23 4 14 24 58 1 33 17 63 76 26 71 11 17 % 70.3 16.7 2.9 10.1 18.0 43.6 0.8 24.8 12.8 45.3 54.7 20.8 56.8 8.8 13.6 % 63.0 18.5 3.7 14.8 37.0 31.5 11.1 11.1 9.3 72.2 27.8 46.9 40.9 0 12.2 *Missing 1 value for the replication survey (n=138). †Missing 6 values for the replication survey (n=133). ‡Missing 5 values for original survey (n=49) and 14 values for the replication survey (n=125). Initial survey (N=54) Replication survey (N=139) by on October 25, 2010ccn.aacnjournals.orgDownloaded from
  5. 5. the nurse educators expected their nurses to have after orientation. Changes Made on the Basis of the BKATs In the initial study, 61.1% of the nurses used the BKAT in their orientation program. In the replica- tion survey, this number was only slightly higher at 69.8% of the nurses. These nurses were then asked if they modified their orientation on the basis of the results from the BKAT. Whereas a little over half said yes in the initial survey, two thirds of respondents in the replication sur- vey agreed that they modified their critical care orientation on the basis of the BKATs. Table 5 shows that more than twice as many nurses added content in the replication study as in the ini- tial study. Content was deleted on the basis of the BKATs twice as often in the replication survey as com- pared with the initial survey. the replication study learned about the BKATs. Uses of the BKATs Table 4 compares why the nurses had requested a copy of a BKAT, that is, the intended use, and the actual use of the BKATs between the 2 surveys. The percent- age of yes responses in the replication survey was approximately twice as high in all categories (1.5- to 2.6-fold increase) for the intended use. The actual use was much higher or showed a 2.5- to 5.5-fold increase from the original survey for all categories except research. Thus, not only did the nurses in the follow-up survey request copies of a BKAT, but 62% to 75% used the tests for the intended purpose. Table 4 also illustrates that the intended and actual use of the BKATs in the replication survey for the placement of nurses with previous critical care experience (opting out of part or all of the usual orientation) increased 1.9-fold and 2.5-fold respectively. Likewise, evaluation of current staff showed a 1.8-fold and 3.7-fold increase in intended and actual use. Scores on the BKATs Responses of the nurses in both the initial and replication surveys indicated that scores were reviewed with almost all the staff nurses taking the BKATs: 93.8% and 94.8%, respectively. Also, a question about scores on the UTool was, “Following orientation, the BKAT reflects which level of knowledge that you expect [your nurses to have].” The Figure shows that the majority of responses of the 87 subjects in the replication survey (58.6%) were simi- lar to the majority of responses of the 25 subjects in the initial survey (52.0%) or that the BKAT reflected a higher level of knowledge than what 52 CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 http://ccn.aacnjournals.org Table 4 How the Basic Knowledge Assessment Tools are used Percentage of respondents Variable Orientation Pretest Pretest and posttest Evaluation of the program Placement of nurses with previous critical care experience Evaluation of current staff Research Initial survey (N=54) Intended use 42.6 39.6 22.2 37.0 38.9 5.6 Actual use 26.4 24.5 11.3 28.3 17.0 5.6 Intended use 81.3 65.8 58.8 71.4 71.1 8.3 Actual use 74.7 62.9 62.3 70.6 62.3 5.1 Replication survey (N=139) Changes in level of knowledge on the Basic Knowledge Assessment Tools after ori- entation: comparison of expectations in the initial survey (n=25, 29 missing values) and in the replication survey (n=87, 52 missing values). %ofrespondents Initial survey Replication survey 60 50 40 30 20 10 0 Same Higher Lower StaffDevelopment by on October 25, 2010ccn.aacnjournals.orgDownloaded from
  6. 6. Almost 90% of the nurses in the replication study who used the BKATs in orientation added content. This finding is different from that in the initialsurvey,inwhichonly40%ofthe nurses added content. Discussion The findings from this survey show an increased use of the BKATs for staff nurses to opt out of part or all of the formal orientation classes. Such opting out is not only cost- effective for the institution but also recognizes the knowledge that staff nurses bring to a new clinical setting. Use of the BKAT in research has remained low and may indicate that the BKATs are primarily linked to the knowledge of staff nurses and not to how staff nurses learn best, for example, research. Conclusions On the basis of the results of this study, the following conclusions are made in relation to the use of the BKATs during the past 10 years: • the percentage of orientation programs in critical care units that use the BKATs has increased, • a larger percentage of staff nurses are being required to take a BKATnotassociatedwithorientation, • more newly hired staff nurses with previous experience are being asked to take a BKAT during orien- tation in order to validate basic knowledge, • critical care nurses working in community hospitals remain the largest group of users of the BKATs, • changes in orientation based on the BKATs have increased, espe- cially changes related to the addi- tion of content, and • the reported use of BKATs in nursing research is low. Recommendations On the basis of the findings of this survey, I recommend the following: • the level of difficulty of the BKATs should be reevaluated, and • the results of these evaluations should be communicated to nurses who request a copy of any BKAT. References 1. Toth JC, Dennis MM. The Basic Knowledge Assessment Tool (BKAT) for critical care nursing: its use and effect on orientation programs. Crit Care Nurse. April 1993;13:98- 105. 2. Toth JC, Ritchey KA. New from nursing research: the Basic Knowledge Assessment Tool (BKAT) for critical care nursing. Heart Lung. 1984;13:272-279. 3. Toth JC. Evaluating the use of the Basic Knowledge Assessment Tool (BKAT) in crit- ical care nursing with baccalaureate nursing students. Image J Nurs Sch. 1984;16:67-71. 4. Catalano JT. The development of a profes- sion. In: Catalano JT, ed. Nursing Now! 4th ed. Philadelphia, Pa: FA Davis; 2006:1-22. 5. Runton NG, Toth JC. Introducing the Basic Knowledge Assessment Tool for pediatric critical care nursing (PEDS-BKAT). Crit Care Nurse. June 1998;18:67-72. 6. Toth JC. Comparing basic knowledge in crit- ical care nursing between nurses from the United States and nurses from other coun- tries. Am J Crit Care. 2003;12:41-46. 7. Toth JC. The Basic Knowledge Assessment Tool (BKAT-7S) for Telemetry/Progressive Care Nursing, Version Five (7-S). Washington, DC: The BKAT Office, The Catholic University of America; 2006. 8. Toth JC. Basic Knowledge Assessment Tool for critical care nursing, version four (BKAT-4): validity, reliability, and replica- tion. Crit Care Nurse. June 1994;14:111-117. 9. Polit DF, Beck DT. Nursing Research: Princi- ples and Methods. 7th ed. Philadelphia, Pa: Lippincott Williams & Wilkins; 2004. 10. Riitta-Liisa A, Ritmala-Castren M, Leino- Kilpi H, Suominen T. Biological and physio- logical knowledge and skills of graduating Finnish nursing students to practice in intensive care. Nurse Educ Today. 2004;24:293-300. http://ccn.aacnjournals.org CRITICALCARENURSE Vol 26, No. 4, AUGUST 2006 53 Table 5 Changes in orientation made on the basis of the Basic Knowledge Assess- ment Tools Percentage of respondents Variable Modified orientation† Yes No Added content‡ Yes No Deleted content§ Yes No Initial survey* (N=54) Frequency 18 14 13 19 7 25 % 56.2 43.8 40.6 59.4 21.9 78.1 Frequency 63 32 41 5 14 17 % 66.3 33.7 89.1 10.9 45.2 54.8 Replication survey (N= 139) *Missing 22 values for the initial survey (n=32). †Missing 44 values for the replication survey (n=95). ‡Missing 93 values for the replication survey (n=46). §Missing 108 values for the replication survey (n=31). by on October 25, 2010ccn.aacnjournals.orgDownloaded from

×