« Back to Results

The Dissagregation of Value-added Learning Scores in Economic and Personal Finance Education

Paper Session

Friday, Jan. 5, 2018 12:30 PM - 2:15 PM

Marriott Philadelphia Downtown, Meeting Room 415
Hosted By: National Association of Economic Educators
  • Chair: Carlos J. Asarta, University of Delaware

Adjusting for Guessing and Applying a Statistical Test to the Dissagregation of Value-added Learning Scores

Ben O. Smith
,
University of Nebraska-Omaha
Jamie Wagner
,
University of Nebraska-Omaha

Abstract

In 2016, Walstad and Wagner developed a procedure to split pre- and post-test responses into four learning types: positive, negative, retained, and zero learning. This disaggregation is not only useful in academic studies, it also provides valuable insight to the practitioner: an instructor would take different mitigating actions in response to zero versus negative learning. However, the original disaggregation is sensitive to student guessing. This paper extends the original work by accounting for guessing and provides adjusted estimators using the existing disaggregated values. Further, a statistical test is provided for the adjusted learning type estimates. Using this test, an instructor can determine if a difference in positive (or negative) learning is the result of a true change in learning or 'white noise.'

Pretest-posttest Measurement of the Economic Knowledge of Undergraduates – Estimating Guessing Effects

William B. Walstad
,
University of Nebraska-Lincoln
Susanne Schmidt
,
Johannes Gutenberg University Mainz
Olga Zlatkin-Troitschanskaia
,
Johannes Gutenberg University Mainz
Roland Happ
,
Johannes Gutenberg University Mainz

Abstract

Learning economics often is discussed in terms of students’ end-of-course achievement as indicated by a test score or course grade. This stock of knowledge gives information, how much students know at a given time. However, not only the learning outcome is important but also the change patterns over time. In this presentation, we measured students’ economics knowledge and understanding at the beginning (pretest) and at the end (posttest) of a university course. In this value-added approach, learning is measured as the change of knowledge and understanding over a period of time. To measure learning, we administered items from the TEL (2013) and from the TUCE (2007). To ensure that all participants have the same entry conditions to the test, we only sampled students attending an economics introductory course covering economics principles as well as basics of microeconomics and macroeconomics. Thus, undergraduate students of business and economics were assessed over the course of their first year. To investigate the change of economic knowledge and understanding, we followed Walstad and Wagner’s (2016) approach to analyze four types of learning (positive, negative, retained, and zero learning). As pointed out by Smith and Wagner (2017), measuring these four types in the context of multiple-choice items is vulnerable for having a closer look on guessing. Considering this, the estimates for the four types of economic learning were adjusted to the expected number of correct answers. The presentation shows that the approaches of Walstad and Wagner (2016) and Smith and Wagner (2017) have given new insights in learning patterns and analyzing change of knowledge and understanding which is important for researchers and lecturers not only in economics education.

Lesson Format and Types of Economic Learning: Long-form Versus Short-form Online Learning Modules

Diego Méndez-Carbajo
,
Illinois Wesleyan University
Scott A. Wolla
,
Federal Reserve Bank of St. Louis

Abstract

This paper describes the differences in types of student learning (positive, retained, negative, and zero)–adjusted for guessing—associated with differences in the format of online learning resources. We study a set of economic education online learning modules focused on unemployment and produced by the Federal Reserve Bank of St. Louis: a long-form module vis-à-vis several short-form modules derived from it. We find evidence of decreased negative learning in the short-form modules as well as marked differences in retained and positive learning associated with the length of the online module. Our preliminary findings seem to suggest that instructors should select lesson formats according to the type of economic learning they wish to promote.

Learning Outcomes in a High School Personal Finance Course: Evidence From Disaggregation of Value-added Test Scores

Andrew T. Hill
,
Federal Reserve Bank of Philadelphia
Carlos J. Asarta
,
University of Delaware
Jamie Wagner
,
University of Nebraska-Omaha

Abstract

In this paper, we use the unique "Keys to Financial Success" dataset to investigate the effects of a high school personal finance course on students' personal finance knowledge. We disaggregate students' pretest and posttest scores and, adjusting for guessing, study the effects of student characteristics on positive, retained, negative, and zero learning outcomes in semester-long "Keys" courses offered in Delaware, New Jersey, and Pennsylvania high schools.
Discussant(s)
Diego Méndez-Carbajo
,
Illinois Wesleyan University
Scott A. Wolla
,
Federal Reserve Bank of St. Louis
Jamie Wagner
,
University of Nebraska-Omaha
Susanne Schmidt
,
Johannes Gutenberg University Mainz
JEL Classifications
  • A2 - Economic Education and Teaching of Economics