I hesitated to include the A-word (assessment) in the title of this piece. However, it is the process of thinking about what we have taught, how well we have taught it, and what the students have actually learned that I want to address—and this is, after all, what assessment is about. I have experimented with word clouds (explained below) in class before, but this semester in my course on Western Religious Traditions I applied them as an informal but systematic way to assess learning. I found that not only did they help show me what my students knew at the beginning of each module, but also what they learned in the process, which helped me assess my own teaching. Additionally it became a good point of discussion when the students compared the before-and-after word clouds in class, which provided a self-analysis of their own learning.
Word Clouds
Word clouds are a visual representation of a collection of words, with the most frequently occurring words appearing largest, the least frequent appearing smallest, and a wide variety of size-gradations in between. While there are numerous websites for creating word clouds, I have been using wordle.net to create mine. You will need to make sure Java is up-to-date in your browser to use wordle.net. Copy and paste your word bank into the box provided, and then click the GO button. After a brief interlude, your word cloud should appear on the screen. You can adjust the settings to change the word orientation, font, background colors, and basic size. After you have it as you want it, you can download it (wordle.net only allows you to download as a .png, but you can use your Save As or Export As features to change this to a .jpg or .pdf if you so desire). A brief warning: Word clouds can be terribly addicting once you learn how to create them. Try copying your latest article, sans bibliography, to see what comes up. Or better yet, copy and paste your entire book or dissertation. It can be satisfying to see your own writing represented visually.
Initial Assessment
Western Religious Traditions is a survey course covering the basic tenets, praxis, and history of Judaism, Christianity, and Islam, along with select comparative material. To create the “before” word clouds, on the first day that I cover a particular tradition, for example Judaism, I hand out index cards to my students and ask them to write seven key words or phrases about Judaism—what do they know, what is important, etc. Depending on the background of your student population, the response will vary wildly, of course. At a small liberal arts school in rural Kentucky with a very small Jewish student population, the majority of my students have very limited exposure to the faith tradition. It was a challenge for most of them to come up with even seven words about Judaism. In doing this activity, it is important to give the students time in class to create their list while you are not asking them to concentrate on anything else. I found that it takes students no more than 5 minutes to come up with their list. I then collect the cards and proceed with that day’s lesson plan.
At this point you can create the initial word cloud and either show it to them, which I did the next day for Judaism, or you can wait until you have both the “before” and “after” word clouds, which is what I did when we covered Christianity. Either way, plan to show the students at least the before and after.
Final Assessment
At the beginning of the final class on each tradition, I repeat the exercise, asking students to write seven key words or phrases on the tradition. The second iteration of the exercise took less time, and students hand in their index cards quickly. When I teach this class again I will likely include a time immediately before the quiz to have them fill out seven key terms or phrases, as I think doing it on the last day of class created an emphasis in the words/phrases on the materials I was teaching for that particular class.
Results for the Judaism segment:
The white background shows the initial list, while the black background shows the words/phrases submitted after our Judaism module. Created with wordle.net, image by Shayna Sheinfeld, September–October 2015.
Results from the Christianity segment:
The white background shows the initial list, while the black background shows the words/phrases submitted after our Christianity module. Created with wordle.net, image by Shayna Sheinfeld, October 2015.
A Few Notes about Process
It seems important to note some of the problems I experienced in the implementation of the word cloud project; I will specifically refer to the Judaism word clouds (above) as this was my first attempt for this class so I was still experimenting with it. First, why seven words or phrases? There is no particular magic number. I wanted to have a large enough word/phrase base to create an interesting word cloud, and I also wanted to challenge my students: seven words/phrases for the Christianity module was less challenging; it was nearly impossible for them at the beginning of the Islam section. You could easily expand or reduce the number required. I should also note that I had two sections of 25 students each, and I combined their words to create my word cloud; you can still use the exercise successfully with a smaller word base, however.
Second, I kept phrases (such as “bar mitzvah” and “bat mitzvah”) separate in the initial word cloud, which created a rather large showing of the word “mitzvah.” However, most of my students only knew that word in conjunction with the coming-of-age celebrations for Jewish boys (bar mitzvah) and girls (bat mitzvah). In my final word cloud, I kept multi-word phrases together, deleting the space in between so that we could see the actual representation of words like “mitzvah” versus phrases like “bar mitzvah.”
Third, I struggled with how often to rephrase or rewrite my students’ words or phrases. When confronted with obvious spelling variations or misspellings (e.g. “mitzva,” “mitzvah,” “mitsva,” etc.) I chose to standardize their variations so that we could visually see how often a word/phrase came up. However, with other variations that presented more complex differences (e.g. “trinity” versus “holy trinity” in the Christianity segment), I chose to keep the phrasing that the students used. While this made those words/phrases smaller than they would have been combined, it also more accurately represented the complex (and often theological) variations among my students.
What Does the Word Cloud Assess?
In the title of this pedagogical piece I called these word clouds “informal assessment.” This assessment is informal because it is not graded and does not provide a detailed analysis of what each student learned. However, it does show both my students and myself the changes in their knowledge about the faith traditions. By putting the two word clouds side-by-side, we were able to draw some conclusions. I presented the word cloud to my students after each segment. They noted, for instance, that the initial Judaism word cloud contained numerous words that focused on the physical manifestations of Judaism, such as symbols (menorah, star of David), buildings such as synagogues, and an emphasis on the holiday of Hanukkah. Israel and Hebrew also came up frequently. After our six-class session on Judaism, the results became more varied. Texts were better represented, including rabbinic texts, and holidays varied more widely (although Hanukkah is still larger than it should be in my mind). The students noted the representation of different denominations of Jews in antiquity and modern times.
Presenting these word clouds in class created an opportunity for the students to observe and assess visually what they as a group have learned, and to discuss the changes between the initial word cloud and the final one. Student discussion was fruitful on what the changes meant in terms of their understanding of Judaism, and it was decided amongst them that they now had more knowledge of Judaism. As their professor, it was a wonderful opportunity to see students reflecting not just on what they needed to know to pass the test, but to think about what it is they have learned and how it changed from the beginning of the class.
The word clouds proved to be a fun visual activity, as I had hoped for the class, and they also provided an unexpected source of self-assessment. Through them, I was able to visually see what I had been successful at teaching (i.e. texts and history) and less successful at conveying (Israel shrinks considerably, and Zionism is not visible at all). Most of this came as no surprise—I did, after all, have only six classes to teach Judaism. But it also gave me an assessment tool for my own abilities at teaching the material, providing me with an informal guide of what I did well and what needs more attention in the future.
Conclusion
If used at both the beginning and the end of a module, word clouds can provide an informal assessment technique, gauging the progress of what students learn throughout the course of the module. They can be used to spur in-class discussion and have students think about their own learning process through self-reflection. The differences between the initial and final word clouds can also provide feedback for the instructor on their success in conveying information. This assessment can assist with the planning of subsequent classes, as instructors are able to see gaps in the knowledge base of the students. It should also be used to confirm the successes of the instructor, visually noting the themes where students gained knowledge, and thus where the instructor was successful in the classroom.