The final unit for our Introduction to Music Education Research course asks us to consider six different studies related to our Unit 2 Research Proposal to build an annotated bibliography and a critique. In some of the course modules, it seems there is some ambiguity in regards to the order of assignments in previous versions of this course. In a previous year, the annotated bibliography and critique may have occurred before the research proposal. Regardless, this unit is framed within the context of The Elementary and Secondary Education Act (ESEA, 1965), its most recent reauthorizations more commonly known as No Child Left Behind (NCLB, 2001) and the Every Student Succeeds Acts (ESSA, 2015), and what our professors are calling the research ecosystem.
ESEA, NCLB, and ESSA
The Elementary and Secondary Education Act was passed by President Lyndon B. Johnson in 1965 as part of his “War on Poverty.” It established Title I funding for schools and districts with a high percentage of low-income families to close the skills gap in reading, writing, and mathematics between students from low-income and middle class families. Our class specifically looked at the provisions within the NCLB revision for what constituted “scientifically-based research”:
Used empirical methods
Involved rigorous and adequate data analyses
Relied on measurements or observational methods that provide reliable and valid data
Used experimental or quasi-experimental design
Allowed for replicability
Underwent expert scrutiny
Education researchers responded with collaborative papers attempting to set standards for the quality of evidence in their research. We were asked to read one such paper, Advancing Scientific Research in Education from the National Academies of Sciences, Engineering, and Medicine. The paper puts forth thirteen recommendations for federal agencies, professional associations, research journals, schools of education, and peer review panels for advancing scientific research in education. These different groups mentioned within the paper are parts of what our professors call the research ecosystem.
Each of our weekly Live Classrooms for this course has functioned as a combination lecture/discussion. For Foundations of Music Education I: Philosophy and History, our Live Classrooms were all discussion-based with a prompt given beforehand that we were asked to prepare. For Introduction to Music Education Research, we were given topics and some idea of what would be discussed, but we have not been required to prepare responses ahead of time. Topics have included:
Introductions & What is Research?
Argument as a component of the research enterprise, what else is there?
Research Questions, Research Problems, and Problems of Practice.
Theory and Theoretical Frameworks in Education Research
The 30,000-foot View of Research in Music education: areas of inquiry, theoretical perspectives, theories, research designs and methodologies.
Evaluating Scholarly Research: validity, generalizability, ….
Connecting research to Education Policy and Practice.
When looking at the 30,000-foot view of research in music education, we were asked to think of music education research as an ecosystem—sites where research occurs, participants in the research, and contexts for research problems and questions. This ecosystem includes (as mentioned above) universities, labs, professional associations, scholarly journals, and national and state music & arts organizations.
Scholarly Disciplines vs. Professional Practices
We also discussed how university settings tend to divide knowledge up into three categories: the Arts & Humanities, the Natural Sciences, and the Social Sciences. Within each of these categories are scholarly disciplines like the Performing Arts, Philosophy, History (Arts & Humanities); Physics, Biology, Chemistry (Natural Sciences); Economics, Sociology, Anthropology, Political Science (Social Sciences). Disciplines have developed research methods and designs to help their researchers investigate the world. Our professors differentiated scholarly disciplines from professional practices (like education, medicine or law) which are inter- and multi-disciplinary. They used the definition that professional practitioners “apply knowledge and insight acquired from amultitude of disciplines in professional relationships with people” (Unit 3, Introduction Video).
Our professors also outlined several families of research methodologies and a few examples of each that are used in education research. The families are listed below in bold with their examples indented beneath in italics. Definitions are taken from Research Design: Qualitative, Quantitative, and Mixed Methods Approaches by Creswell & Creswell unless otherwise noted.
Quantitative: tests “objective theories by examining the relationship among variables” (p. 4)
Experimental: systematic manipulation of 1 or more variables in order to evaluate how the manipulation impacts outcome(s) of interest (p. 147)
Quasi-experimental: form of experimental research where individuals are not randomly assigned to groups (p. 166)
Correlational: describe and measure the degree/association/relationship between two or more variables or sets of scores (p. 12)
Qualitative: explores and understands “the meaning individuals or groups ascribe to a social or human problem” (p. 4)
Basic qualitative: “investigates people's reports of their subjective opinions, attitudes, beliefs, or reflections on their experiences, of things in the outer world” (Generic Qualitative Research in Psychology, The Qualitative Report, 20(2), p. 78)
Ethnography: study of “the shared patterns of behaviors, language, and actions of an intact cultural group in a natural setting over a prolonged period of time” (p. 13)
Case Study: developing “an in-depth analysis of a case… bounded by time and activity” (p. 14)
Action/Design-based: “a disciplined process of inquiry conducted by and for those taking the action. The primary reason for engaging in action research is to assist the “actor” in improving and/or refining his or her actions.” (Guiding School Improvement with Action Research, ASCD)
Conceptual/Philosophical: “is generally used by philosophers and thinkers to develop new concepts or to reinterpret existing ones“ (Conceptual vs. Empirical Research, Reseapro)
Historical: “collect, verify, and synthesize evidence from the past to establish facts that defend or refute a hypothesis” (Types of Research Designs, University of Southern California)
Music/Arts-Based, Aesthetic: focus on music/arts/aesthetic “contents and issues” or “research shaped by” music/arts/aesthic “concepts and approaches” (Music in Qualitative Research, The SAGE Encyclopedia of Qualitative Research Methods)
In my previous post on note-taking applications, I mentioned I would briefly review how using the app Agenda worked with this assignment. One of my biggest annoyances is its auto-formatting of hyperlinks. Every reference I had for this assignment had a URL for the study. Whenever Agenda sees a URL, it auto-formats it into a hyperlink. This wouldn’t be so bad if 1) it didn’t shorten the URL to do so, and 2) it didn’t continue to try and auto-format it after I have removed the hyperlink.
Otherwise, I love Agenda. It syncs between our desktop iMac, my laptop, my iPad, and my iPhone. I can view, organize, and tag the notes in the ways I would like. The only drawback I have found is the auto-formatting I noted above.
There are two pieces to this final assignment: an annotated bibliography of 4 sources and a critique of 2 different sources. Here are (roughly edited) the prompts:
Please select 4 empirical articles (2 quantitative, 2 qualitative) that relate to your topic for the upcoming research problem assignment. Please keep in mind that empirical studies are those where data have been collected and analyzed to answer research questions. This means that literature reviews, philosophical articles, theoretical articles, and professional articles are not appropriate for this assignment. When selecting articles use ONLY the education and music education journals listed in the Syllabus: Course Materials page. Please provide the following information (as available) for each, and use the following template (even though it may vary somewhat from APA in terms of the headings).
Full citation, in APA6 format
Phenomenon of interest (what is being studied)
Theoretical framework (the theory, conceptual lens, or perspective that guides the study)
Research problem (a scholarly argument that articulates what new knowledge is to be created by the study, often presented as tension between the theory and practice)
Research questions (might be explicitly stated or inferred by the purpose statement)
Method (e.g., "surveys," "interviews")
Findings (Evidence based upon a data analysis process)
Conclusions (2-3 sentences, written in your own words)
How this article might relate or inform your research problem. Be specific and make clear connections.
Choose one study that primarily employs the values and language of "quantitative inquiry," and 1 study that primarily employ the values and language of "qualitative inquiry." Write a critique of each study, remembering that a critique is different than an annotated bibliography. The two articles you select for this assignment should be different than the ones you used for the annotated bibliography writing assignment. Limit your critique to 500 words each.
As my Unit 2 Research Proposal was on looking at music education in maker-centered learning experiences, I found that there were not many empirical studies within the education and music education journals listed in the prompt. After compiling a list of journals outside of those listed that contained peer-reviewed, empirical research related to my topic, I received approval from my professor to move outside of the prescribed list.
Calabrese, B. A., & Tan, E. (2018). A longitudinal study of equity-oriented STEM-rich making among youth from historically marginalized communities. American Educational Research Journal, 55(4), 761-800. https://doi.org/10.3102/0002831218758668
Phenomenon of Interest: Equity-oriented STEM-rich maker projects for youth from historically marginalized communities
Cultural views of learning and development, placing human interaction and activity at the center of analysis (Vossoughi & Gutiérrez, 2014; Engeström & Sannino, 2010).
There are “no cultureless or neutral” ways of being in the world (Bang et al., 2013).
Culture is dynamic yet made up of routine practices in which people engage, not merely membership in a group (Gutiérrez & Rogoff, 2003).
Learning is a relational activity in terms of power, place, and time (Leander, Philips, & Taylor, 2010; Gutiérrez, 2008).
Anthropological view or relationality (Eckert, 2016)
Making is a dynamic multipractice where practices are reauthored or remixed from a wide range of experiences (Kafai et al., 2014).
Intersectionality — how do interconnected systems of power impact the marginalized in society (Crenshaw, 1991; Nash 2008).
Research Problem: Makerspaces are popping up all across the country and being included into school STEM settings. The authors identify a need for knowledge and practices within maker culture to ensure maker-centered learning experiences are equitable and consequential for students from historically marginalized communities. Specifically, they identify a lack of empirical studies analyzing how youth engage in STEM-rich making experiences, especially for students from marginalized communities.
Research Questions: Explicitly stated and quoted verbatim:
How does community engagement as a part of STEM-rich making impact what, how, and why youth make?
How do youths’ making practices and projects, as enacted through community engagement, contribute to the making culture in their community making spaces in equitable and consequential ways? (Calbrese & Tan, 2018, p. 763)
Method: The authors designed a longitudinal critical ethnographic study in the qualitative tradition. Researchers collaborated with two community-centered makerspace programs in Michigan and North Carolina from 2013 to 2017, providing youths with STEM-rich making experiences. Data was collected from field notes from multiple researchers; midyear and end-of-year “artifact interviews” with participating youths; interviews with maker space mentors, club leaders, involved community members, and parents; and informal weekly conversations with a subset of youth. There were 48 youths who participated in the study, completing 41 projects
Youth projected their community culture onto their making. They also highlighted injustices and violence they have experienced.
Community engagement in the experience legitimized co-making, supporting equity-oriented goals like making new tools and wisdom accessible to the community to negotiate and rewrite injustice, reorganizing traditional hierarchies for making, and creating more opportunities to be recognized.
Youths viewed their design work as confronting problems in their lives like sexism, racism, and classism, problems that emerged from their own communities.
Maker-centered learning opportunities that engage community honor youths’ backgrounds while encouraging agency.
Maker-centered learning opportunities that engage community provide a platform for integrating questions of social justice and equity.
Co-making within the community disrupts normative power dynamics among adults, youth, and context, creating opportunity for equity.
Relation to my study: I am interested in how music education intersects with maker-centered learning experiences. By embedding maker learning within the community, there are many opportunities for music to intersect. I am curious how this integrates with readings we did in our previous course on music education philosophy and our upcoming course on community music.
Crenshaw, K. (1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6),1241–1299.
Eckert, J. (2016). Beyond Agatha Christie: Relationality and critique in anthropological theory. Anthropological Theory, 16(2–3), 241–248.
Engeström, Y., & Sannino, A. (2010). Studies of expansive learning: Foundations, findings and future challenges. Educational Research Review, 5(1), 1–24.
Gutiérrez, K. D. (2008). Developing a sociocritical literacy in the third space. Reading Research Quarterly, 43(2), 148–164.
Gutiérrez, K. D., & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. Educational Researcher, 32(5), 19–25.
Kafai, Y. B., Fields, D. A., & Searle, K. A. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review, 84(4), 532–556.
Leander, K., Phillips, N., & Taylor, K. (2010). The changing social spaces of learning: Mapping new mobilities. Review of Research in Education, 34, 329–394.
Nash, J. C. (2008). Re-thinking intersectionality. Feminist Review, 89(1), 1–15.
Vossoughi, S., & Gutiérrez, K. (2014). Studying movement, hybridity, and change: Toward a multi-sited sensibility for research on learning across contexts and borders. Teachers College Record, 116(14), 603–632
Chen, Y., & Wu, Can. (2017). The hot spot transformation in the research evolution of maker. Scientometrics, 113(3), 1307-1324. https://doi.org/10.1007/s11192-017-2542-4
Phenomenon of Interest: Evolution of research of maker culture
Theoretical Framework: No theories were explicitly stated. The authors did use constructionism (Harel & Papert, 1991) as a keyword to collect literature.
Research Problem: The volume of literature about the maker culture is rapidly expanding. There is a need to identify the structure of its intellectual landscape to draw out themes of research topics, contributors, and contexts. CiteSpace had not been used to analyze maker literature prior to this study.
Research Questions: Explicitly stated and quoted verbatim:
Does the research evolution of maker study take on some periodic features?
How many research hot spots and knowledge groups can be marked off?
What is the transformation trends of maker study? (Chen & Wu, 2017, p. 1309)
Method: The authors used the quantitative method of scientometrics to measure and analyze scientific literature on maker culture. They used the Institute for Scientific Information’s Web of Science (ISI WoS) to collect literature and the scientometric software CiteSpace to visualize bibliographic records. They reviewed 836 bibliographic records on maker culture to identify themes, knowledge base clusters, frontiers, and major contributors in maker research. Keywords used to collect literature from ISI WoS were: “craft,” “pro-am,” “do it yourself (or DIY),” “maker,” “commons-based peer production,” “physical computing,” and “constructionism.” Records that were considered duplicates or book reviews were removed. CiteSpace was then used to identify major clusters and key studies making significant contributions to research development, to analyze frontline research in the maker field through categories and keywords, and to map contributing countries, organizations, and authors.
Maker study in journal articles and conference proceedings begins to dramatically increase beginning in 2003.
CiteSpace identified 263 knowledge clusters with close relationships and connections within the clusters.
Seven of the top eight most-cited articles occur within the largest knowledge base cluster of “maker movement.”
The research of “maker” is studied mainly in the fields of computer science, engineering, and education.
Countries, institutions, and scholars are ranked in terms of the number of studies and published proceedings between 1972 and 2016.
The largest knowledge base clusters of maker study are reflected in the themes “maker movement,” “DIY culture,” and “craftsman hero”
The research of “maker” is studied mainly from the fields of computer science, engineering, and education. The main categories covered by journal articles were art, business and econimics, information science, and library science.
Major contributors hail mainly from the United States, the University of California Berkeley, the Georgia Institute of Technology, and the Massachusetts Institute of Technology.
Relation to my study: This quantitative study provides a broad overview of the research already undertaken in maker culture. It helps point me towards specific literature for me to review as well as potentially identifying gaps where my interest in music education within makerspaces could fit.
Harel, I., & Papert, S. (1991). Constructionism. New York: Ablex Publishing Corporation.
Kafai, Y. B., Fields, D. A., & Searle, K. A. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review, 84(4), 532-556, 563-565. https://doi.org/10.17763/haer.84.4.46m7372370214783
Phenomenon of Interest: Electronic textiles as a maker activity
Theoretical Framework: None explicitly stated, however the authors do mention technological transparency (e.g.,Hargittai, 2010), aesthetics in education (Dewey, 1980; Vygotsky, 2004), and constructionism (Papert, 1991)
Research Problem: As maker activities continue to grow in popularity, the authors identify two issues: it is a challenge getting women involved in STEM fields, specifically computing, and current models of school rarely combine academic skills and hands-on work.
Research Questions: Explicitly stated and quoted verbatim:
How does designing with e-textile materials promote transparency of technology, a better understanding of how technology works?
What role do aesthetic considerations play in students’ making of e-textiles?
And in what ways do e-textile activities complicate students’ gendered perspectives on computing and engineering? (Kafai, Fields, & Searle; p. 535)
Method: The authors did not explicitly state a specific research design, but the methods used fit the qualitative tradition. They explored three e-textile workshops in an urban, public, science and technology magnet high school in the Mid-Atlantic region in partnership with a local science museum. The researchers developed e-textile making activities, met students once per week for two hours over four weeks, examined students’ design processes and final products, observed activities and conversations, and interviewed students at the conclusion of each workshop.
The inner workings of circuits and programming are made more transparent by infusing crafting into the domains of electronics and coding. Crafting is an integral part of learning about circuits and programming with e-textiles.
Students who incorporated aesthetic elements in their designs felt more ownership of their work and created more challenging projects. This attention to aesthetics motivated the richest learning because students were deeply invested in their projects and intrinsically motivated to learn more complex methods in order to achieve their desired results.
Norms of gendered domains (males and technology, girls and crafting) began to shift by the conclusion of the workshops.
Conclusions: E-textiles are successful at opening doors to students who are traditionally excluded from technical subjects because of the presence of transparency, aesthetics, and gender when making with e-textiles. E-textiles have the potential to create a more inclusive model of computing. They also unify material (hands-on) and digital (minds-on) learning in authentic and aesthetic ways.
Relation to my study: Music, as an aesthetic experience, can also be integrated with maker-centered learning experiences. This study shows ways in which the arts can be authentically included in a STEM-based making activity. I am curious how the making of musical instruments or compositions could be authentic integrations of music and maker-centered learning.
Dewey, J. (1980). Art as experience. New York: Berkley. (Originally published 1934)
Hargittai, E. (2010). Digital na(t)ives? Variation in Internet skills and uses among members of the “Net Generation.” Sociological Inquiry, 80(1), 92-113.
Papert, S. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionisim(pp. 1-11). Cambridge, MA: Massachusetts Institute of Technology Press.
Vygotsky, L. S. (2004). Imagination and creativity in childhood. Journal of Russian and East European Psychology, 42(1), 7-97.
Noh, Y. (2017). A study of the effects of library creative zone programs on creative thinking abilities. Journal of Librarianship and Information Science, 49(4), 380-396. https://doi.org/10.1177/0961000616650933
Phenomenon of Interest: Creative thinking abilities index after engaging in a public library creative zone for six months
Theoretical Framework: While a theoretical framework was not explicitly mentioned, the researchers use the Torrance Tests of Creative Thinking (TTCT): Thinking Creatively with Word (Torrance, 1966, 1993). The tests have a history in the study of measuring creativity (Guilford, 1967; Amabile, 1983, 1996; Csikszentmihayi, 1988, 1999; Osborn, 1953; Torrance, 1972; Isaksen et al., 1994) and are comprised of both verbal and figural components. he verbal test, which was used in this study, measures fluency, flexibility, and originality through six different activities entitled questioning, assuming causes, assuming results, improving the work, unique use of cardboard box, and imagining. Torrance defined creative thinking abilities as “the process of being sensitive to knowledge gaps, identifying the difficulties, exploring and guessing the resolutions or generating a hypothesis against the problem and verifying it to modify it if necessary” (Torrance, 1974, p. 8; 2002, p. 7).
Research Problem: Makerspaces—creative zones—provide the opportunity to engage in what the author calls “infinite creative activity” (Noh, 2017, p. 380). As institutions in the United States have implemented makerspaces, the South Korean government has called for similar creative zones. The authors identify that there were no studies evaluating the change in creative thinking abilities of people who work in an infinite creative space.
Research Question: Explicitly stated and quoted verbatim:
Do participants’ fluency, flexibility, originality, and overall creative thinking abilities improve after participation in the program? (p. 385)
Method: The researcher used a quantitative design in applying the Torrance Tests of Creative Thinking (TTCT) to evaluate creative thinking abilities in participants’ language. Specifically, the study used a one-group pre-test/post-test design (Fraenkel & Wallen, 1996). The participants made an environment-related science picture book through a series of five stages incorporating lectures and workshops into twenty-one separate two-hour activities for a total of forty-two hours. The pre-test was taken during the first stage, and the post-test was taken during the final activity in the fifth stage.
The TTCT verbal tool consists of six activities: questioning, assuming causes, assuming results, improving the work, unique use, and imagining. The first four activities ask participants to interpret a picture: writing down questions about the picture, listing causes for what is occurring in a picture, listing results from what is occurring in the picture, and listing ways to improve what is happening in the picture. The fifth activity asks participants to list possible uses of an empty cardboard box. The sixth activity asks participants to list consequences of an impossible, imagined scenario. The authors had extremely detailed instructions for scoring each activity of the TTCT for fluency, flexibility, originality, and overall creative thinking abilities. The results were verified using t verification,
Participants’ fluency score was improved with a significant difference by participating in the program (average of 88.22 increasing to 106.94, t= -5.35, p= .000)
Participants’ flexibility score was improved with a significant difference by participating in the program (average of 95.17 increasing to 110.17, t= -3.64, p= .002)
Participants’ originality score was improved with a significant difference by participating in the program (average of 94.17 increasing to 109.89, t= -4.50, p= .000)
Participants’ average creative thinking ability score improved with a significant difference by participating in the program (average of 92.44 increasing to 109.1, t= -4.77, p= .000)
There were no significant differences for fluency, flexibility, originality, or average cognitive thinking abilities for factors by gender.
A creative thinking program can improve participants’ fluency, flexibility, originality, and overall creative thinking abilities.
The infinite creative space is a place to share library and community resources for creation.
Results from this study can aid libraries in designing programs that promote creative thinking abilities.
Relation to my study: The TTCT provides a quantitative method for measuring the change in participants’ fluency, flexibility, originality, and average creative thinking abilities. This could be used to measure the success of implementing music activities in maker-centered learning experiences.
Amabile, T. M. (1983). The social psychology of creativity. New York: Springer.
Amabile, T. M. (1996). Creativity in context: Update to the social psychology of creativity. Boulder, CO: Westview.
Csikszentmihalyi, M. (1988). Society, culture, person: A systems view of creativity. In: R. J. Sternberg (Ed.)The Nature of Creativity(pp. 325-329). New York: Cambridge University Press.
Csikszentmihalyi M (1999) Implications of a systems perspective for the study of creativity. In: R. J. Sternberg (Ed.), Handbook of human creativity(pp. 273-296). New York: Cambridge University Press.
Fraenkel, J. R., & Wallen, N. E. (1996). How to design and evaluate research in education. (Vol. 3). New York: McGraw-Hill.
Guilford, J. P. (1967). The nature of human intelligence. New York: McGraw-Hill.
Isaksen, S. G., Dorval, K. B., & Treffinger, D. J. (1994). Creative approaches to problem solving. Dubugus, IO: Kendall/ Hunt.
Osborn, A. F. (1953). Applied imagination. New York: Scribner.
Torrance, E. P. (1966). The Torrance Tests of Creative Thinking: Norms-Technical manual research edition – Verbal tests, forms a and b – Figural tests, forms a and b. Princeton, NJ: Personnel Press.
Torrance, E. P. (1972). Predictive validity of Torrance Test of Creative Thinking. Journal of Creative Behaviour 6(4), 236-252.
Torrance, E. P. (1974). The Torrance Tests of Creative Thinking: Norms-Technical manual research edition – Verbal tests, forms a and b – Figural tests, forms a and b. Princeton, NJ: Personnel Press.
Torrance, E. P. (1993). Understanding creativity: Where to start? Psychological Inquiry 4(3), 232–234.
Torrance, E. P. (2002). The manifesto: A guide to developing a creative career. Greenwood.
Cohen, J. D., Huprich, J., Jones, W. M., & Smith, S. (2017). Educators’ perceptions of a maker-based learning experience. The International Journal of Information and Learning Technology, 34(5), 428-438. https://doi.org/10.1108/IJILT-06-2017-0050
As the maker movement continues to gain popularity in schools, Cohen, Huprich, Jones, and Smith wanted to analyze the impact of a maker-focused learning experience on educators’ perceptions of learning. They provide a definition of the maker movement and a brief history of its evolution using a variety of sources including journal articles, speeches, and books before reviewing research on how the maker movement impacts learning. Through their literature review, the researchers found that maker education may align with learning theories like experiential education, critical pedagogy, and problem-based learning; may attract students to STEM subjects; may improve spatial reasoning; and may promote agency, empowerment, and growth mindsets.
The researchers then look at research on educator perceptions of maker learning, finding that there is a large body of research focusing on strategies for implementing maker tools in different student populations. They were able to locate three studies examining training educators in maker learning that focused on one-time workshops. The researchers identified a lack of scholarly literature on educators participating in a semester-long maker-based learning experience, leaving an important need for this study.
To examine educators’ perceptions of learning and how they are impacted by a semester-long, maker-focused learning experience, the researchers implemented a qualitative exploratory design (Creswell, 2012). Huprich designed a course focusing on the role maker principles and technologies play in a variety of educational contexts. Cohen served as the instructor of the course, and Huprich served as a teaching assistant. The course occurred at a large university in the Southeastern United States with thirteen students enrolled, twelve of whom participated in the study. Only one of the participants was a practicing teacher, and one participant was a curriculum designer. The remaining ten participants were in some type of pre-service or interdisciplinary studies program. None of the participants had any previous experience with makerspaces or maker communities. As the authors admit, the sample of participants was one of convenience with students who elected to enroll in a course on maker principles.
Data was collected from periodic journal-style reflections, final essays, course reflections, and researcher observations. The participants’ responses showed researchers how students’ thinking evolved over the course of the semester, and the researcher observations provided context to the students’ reflections. The data was analyzed and coded in a two-cycle approach (Miles et al., 2014): descriptive coding and pattern coding. The descriptive coding cycle produced sixteen codes representing topics related to making and learning. The pattern coding cycled revealed two major themes: collaboration and community.
At the beginning of the course, students described their ideal learning environment, establishing a conceptualization of education. As the course progressed, the researchers found that the participants developed an awareness of collaboration as part of an ideal learning environment. Specifically, collaboration skills like peer teaching and learning, help-seeking behaviors, and empathy working with others, were realized by the participants at the conclusion of the study. A similar realization occurred with participants perceptions of community being developed by a maker-based experience, including a diversity of opinions, shared goals and struggles, and a safe environment to experiment and fail.
The researchers believe this study provides a foundation for teacher educators to develop maker-based learning activities. While the participants did engage in maker activities supported by readings, reflections, lectures, presentations, and discussions to analyze the impact on their perceptions of learning, the study is limited by the number and types of participants. Only two of the twelve participants were actively engaged in an educator role outside of the course. The study does provide data about a longer maker-based experience than one-time workshops, a semester-long course is not indicative of maker-based experiences most K-12 students would receive. This study would be more compelling if it had incorporated more practicing educators providing maker-based experiences in a K-12 environment.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research(4th ed.). Boston, MA: Pearson.
Miles, M. B., Huberman, A. M., & Saldaña, J. M. (2014). Qualitative data analysis: A methods sourcebook(3rd ed.), Thousand Oaks, CA: Sage Publications.
Lille, M. (2016). Evaluating the success of makerspace in a public library: The case of Narva City Library MakerLab in Estonia. New Library World, 117(9/10), 587-595. https://doi.org/10.1108/NLW-04-2016-0030
The Narva City Library has implemented a makerspace to motivate its citizens to develop new skills and improve employment prospects. Lille sought to measure the impact of the makerspace on users’ skills, knowledge, and attitudes by evaluating attendance, learned skills, and users’ feelings about the impact after a year of implementation. She provided demographic information about Narva City, specifically in regard to unemployment. 8.4 percent of native Estonian speakers are unemployed compared to 18.8 percent for non-speakers. The Narva City Government has implemented a variety of initiatives and policies to combat the high unemployment rates, one of which includes creating a library makerspace in conjunction with the United States Embassy with the goal of enhancing citizens’ social and entrepreneurial abilities. Lille also detailed a brief history of library makerspaces and their benefits, including diverse user groups, a variety of environments, and opportunities for collaboration.
To measure the impact of the makerspace on its users’ skills, knowledge, and attitudes, Lille developed a basic logic model using quantitative and qualitative measures for evaluating attendance, learned skills, and participants’ feelings. The logic model identified a long-term goal of “better work perspectives” and a short-term goal of “improved competences, social inclusion, new skills, participating in new projects” (Lille, 2016, p. 589). For success indicators, Lille looked for an improvement in the ability to use and access new technologies and an increase in the number of new projects, self-confidence, and goal-oriented behavior. The activities used in the logic model were workshops, networking, and individual projects with inputs from the library that included mentors, workshops, individual training, physical space, and equipment.
Data was collected by surveying 82 participants to measure the number of workshops, training programs, participants, and private projects. Interviews were also conducted with two participants, two librarians, one member of library management, and one representative from the unemployment office. The interviews were designed to measure knowledge, beliefs, values, feelings, skills, creativity, inspiration, and behavior.
Lille found that 675 people participated in MakerLab activities, workshops, and training programs from November 1, 2014 to December 3, 2015. There were a total of 41 workshops conducted, 17 of which were done externally due to room size or the nature of the workshop. For reasons given for participation, 63% of participants took part because they could gain additional knowledge, 9% found workshops linked to their jobs, and 28% attended for other reasons. Overall, Lille found feedback to be positive with respondents likely to participate in similar events in the future.
Lille concluded that the MakerLab project was a success from the perspective of all those interviewed, providing the expected outcomes for users: learning to use new technologies, attempting different methods and materials, and starting new projects. Her suggestions for improvements included integrated mentors, links to the unemployment office, and monitoring participant needs to adjust topics and hours accordingly.
While the data collected in the study does point to the MakerLab project having a positive impact on users’ skills, knowledge, and attitudes, the basic logic model and its measurements that Lille developed is not based in any previous research. Lille also does not share much of the raw data collected from the surveys, instead sharing some demographic and attendance data highlighting successful indicators. Had the study been grounded in previous research and proven methodologies, the results could have been used as an example for how to evaluate the effectiveness of makerspace implementations.