Hoping to pass: A multi-site study examines how a one-time intervention affects high-stakes test scores

Abstract

<p><strong>Purpose: </strong>The purpose of this subsection of a multisite trial is to test the effect of a one-time hope intervention on high-stakes program-exit test scores among final semester baccalaureate nursing students, compared to a relaxation intervention. There are two research questions posed for this preliminary report. The first question utilized all participants&rsquo; responses to examine changes in hope scores, and how they differed by group. The second question related only to the final semester students; what are the between group differences in scores on the standardized exam (SE)?</p> <p>Student success on testing is a clear concern. Nursing education programs throughout the world are tasked with providing didactic content and clinical experiences which produce nurses who practice in a safe and effective manner while advancing global health. To accomplish those goals competencies are established by regulatory bodies, and an expected level of knowledge is required to meet licensure requirements (Hudspeth, 2012). In the United States (US), the National Council Licensure Exam for Registered Nurses (NCLEX-RN^&reg;) &ldquo;&hellip; measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse&rdquo; (National Council of State Boards of Nursing^&reg;, 2012, p. 1). Nursing programs in the US have adopted the use of specialty and exit examinations to predict success within a nursing program, as well as the likelihood of a student passing the NCLEX-RN^&reg;.</p> <p>High-stakes testing is defined as a test or series of tests with outcomes that have important consequences for examinees (National Council on Measurement in Education, 2012). These exams are thought to represent having attained adequate knowledge of the subject at hand. Up to 30% of schools in the US require students to pass high-stakes exams as a condition of graduation (National League for Nursing, 2012; Santo, Frander, &amp; Hawkins, 2013).</p> <p>Nursing students understand the importance of, and the potential for, grave consequences if required scores are not obtained. Fear of failure may create unfavorable learning conditions as students are driven by this pressure to succeed. For some, this may lead them to study only to achieve the required passing score on the exam. This approach is unlikely to produce the firmly embedded knowledge needed to connect the dots (Dunn, 2014), and even if they succeed in passing, without a deep understanding of core didactic material, the new graduate may function poorly when clinical judgment and decision-making are required in nuanced healthcare situations.</p> <p>The significance of the situation is demonstrated when nursing programs base progress and completion on SE scores; thus, exams are perceived as stressful to the point of being high-stakes in nature which may create these unfavorable learning conditions. There may be ways to decrease test anxiety and enhance learning by supporting selected psychological aspects related to learning. Hope (the belief in one&rsquo;s ability to achieve desired goals), and achievement goal orientation (why one desires to succeed) are associated with decreased stress, as well as increased academic achievement and higher graduation rates (Feldman &amp; Dreher, 2012). However, this has not been examined as a way to increase high-stakes SE scores or predict nursing student success.</p> <p>A review of the literature provides an interesting base for consideration. Academic progress and completion of a BSN program requires students to demonstrate competency in clinical skills and didactic knowledge. Didactic competencies are often tested via end-of-course and end-of-program SEs (Lauer &amp; Yoho, 2013; NLN, 2012). These tests are normed on national samples. Exams are perceived as high-stakes in nature and students may develop positive and negative ways of studying and coping to achieve success. Regrettably, poor coping mechanisms and ineffective learning pathways result in inadequate mastery of material, increased anxiety, maladaptive learning behaviors, and decreased overall performance (Dunn, 2014).</p> <p>Recent research links hope to success of nursing students (Bressler, Bressler, &amp; Bressler, 2010; Chang, 1998; Dunn, 2014). In a recent study of BSN students a positive association between academic and non-academic factors (mastery and performance, exam scores, and specific types of goal orientation) was found (March &amp; Robinson, 2015). Fortunately, recent research in psychosocial education interventions suggests that a one-time hope intervention may influence how students approach learning (Feldman &amp; Dreher, 2012).</p> <p>With over two decades of research, hope theory is a well-established model to explain outcomes among students in education, sociology, and psychology (Ames, 1992; Cavazos Vela et al., 2014; Snyder et al., 1991). Hope is composed of two inter-related types of goal-directed cognition: agency thinking and pathways thinking (Snyder, et al., 1991). Agency thinking consists of &ldquo;the thoughts that people have regarding their ability to begin and continue movement on selected pathways&rdquo; toward their goals (Snyder, Michael, &amp; Cheavens, 1999). Pathway thinking consists of planning-related cognitions. In short, a pathway is a plan, route, or strategy for accomplishing a goal. Because some pathways may not succeed, high-hope people are theorized to have high agency and to act, in part, to produce multiple pathways in order to circumvent possible obstacles (Snyder, 2002). Higher levels of hope are linked to academic success, such as retention, progression, and graduation (Cavazos Vela et al., 2014; Feldman &amp; Dreher, 2012). Thus, hope theory may predict a variety of academic, social, and emotional outcomes.</p> <p><strong>Methods: </strong>This randomized control trial recruited participants from a traditional baccalaureate nursing program in the southern US. A power analysis for the entire study is available upon request, but does not relate to this subsection of the study. Human subject&rsquo;s approval was obtained. Once randomized to group the students experienced a one-time 90-minute intervention; either hopeful thinking or guided relaxation. Pre- and immediate post-intervention measures were obtained at that time. Additional post-intervention measures prior to the final HESI exam were obtained and will be reported separately. Instruments for this subset of inquiry included the Goal-Specific Hope Scale (GSHS) and the SE score; end of program Health Education System, Inc. (HESI) exam. Hope was measured utilizing four items from the GSHS that assessed agency and four items that assessed pathways. Each item was scored using an 8-point Likert-type scale ranging from 1 (definitely false) to 8 (definitely true). The HESI exam assessed student competency related to the achievement of course and curricular outcomes.</p> <p><strong>Results: </strong>To answer the first question; what are the changes in hope, and how do they differ by group we used a 2 time x 2 group ANOVA to examine scores on the GSHS (n=134). Scores improved significantly from baseline to post-intervention for those in the Hope group [F(1, 70) = 36.7,&nbsp;<em>p&nbsp;</em>&lt; .001], as well as for the Relaxation Group [F(1, 62) = 25.3,&nbsp;<em>p&nbsp;</em>&lt; .001]. The 2 time x 2 group interaction was not significant [F(1, 132) = 2.7,&nbsp;<em>p&nbsp;</em>= .10].</p> <p>To answer the second question; what are the between group differences in scores on the SE, a one-way ANOVA compared group scores (n=45). For students in the final semester the scores on the exit SE did not differ. However, there was a slight score benefit for Hope Intervention students (Difference = 25.8), yet this did not achieved statistical significance over the scores for the Relaxation Intervention [F(1, 43) = 0.61,&nbsp;<em>p&nbsp;</em>= .44]. Next, we examined the difference in mean scores as &ldquo;benefit to pass rates&rdquo; (achieved cut score of 850). This produced a meaningful trend for the Hope group (see Table 1); however, given the small sample the pattern was not significant [c²(45) = .73,&nbsp;<em>p&nbsp;</em>= .39].</p> <table border="0"><caption>Table 1: Benefit to Pass: Cut Score above 850</caption> <tbody> <tr> <td>Score</td> <td>Hope</td> <td>Relaxation</td> </tr> <tr> <td>Above 850</td> <td>66.7% (14)&nbsp;</td> <td>54.2% (13)</td> </tr> <tr> <td>Below 850</td> <td>33.3% (7)</td> <td>45.8% (11)</td> </tr> </tbody> </table> <p><strong>Conclusion: </strong>The preliminary results that goal specific hope increased are promising. Although the greater increase was seen in the Hope group, it was not statistically different from the Relaxation group. We believe that once data collection is complete this may reach statistical significance. In addition, we did not find a statistical difference between groups on the SE scores. The fact that a distinct trend towards a difference is promising, and additional data will be needed to determine if this occurs. The study is ongoing and this poster only presents preliminary findings from one of the two sites involved in the overall study.</p> <p>We are encourage by the results to date and if the findings indicate that a one-time intervention could change hope and therefore impact progression through a nursing program it may be possible to deploy this strategy to many nursing students. A future area of inquiry is to examine whether or not re-dosing may be needed, and if it would be effective.</p>