Exploring GenAI with The Carpentries

As with most Carpentries workshops, the GenAI in Workshops bonus module to Instructor Training began with a series of icebreaker questions for participants.

"How helpful or unhelpful do you think genAI tools are for novices learning to code?"

We found that our answer remained the same before and after the workshop: “It depends”.

What we did

Led by Toby Hodges and Greg Wilson, we used a communal etherpad, practice exercises, and open discussion to explore GenAI-assisted coding. Exercises primarily focused on tool exploration and prompting with the use of a learner profile to generate more useful output. This was a strong inroad for a broader discussion about GenAI, as it allowed all participants, with our varying experiences and pre-existing knowledge of GenAI tools, to begin in the same place.

Throughout the exercises, we continually compared GenAI outputs to our personal expertise. “Is this the same way you would have taught this topic? Is this the same content you would’ve used as an example?”. If someone responded, “No, I would’ve done something different”, then questions were raised about who, in a live coding demonstration, becomes the voice of authority - the human or the AI? We also considered how learners may view an AI tool differently than Instructors when they may feel anxious or embarrassed seeking help, such as asking for things to be re-explained or troubleshooting set-up.

For a group activity, we were asked to choose a challenge question within any curriculum from The Carpentries, and a GenAI chatbot of our choice. We prompted the tool to solve the exercise and then compared and contrasted the outputs and their varying levels of citation, detail, and accuracy. Our discussion revealed that different institutional tool subscriptions, previous user history, and prompting differences resulted in entirely different outputs, even if the question and the tool were the same. This highlights one of many potential challenges instructors will face when using GenAI in teaching.

Why were we motivated to attend

More often than not, we individually engage with GenAI tools and rarely take the time to experiment with others. There are also no community-wide accepted guidelines for good practices when teaching with GenAI or using GenAI in code. “Should we embrace it? How do we incorporate it into our lessons? What pitfalls should we avoid?” These were all questions participants came prepared to explore with others. Many folks also noted that their primary driver for attending the bonus module was a sense of responsibility to address these gaps so that they could better support their learner communities and ensure their lessons remained relevant and impactful for learners.

What we hope is next

This workshop created a great environment for engaging community discussion to occur, so we hope The Carpentries continues to build on this momentum. Experimenting with GenAI is the first step to reasoning whether or not their application is appropriate for full integration with The Carpentries curriculum or standard Instructor Training. Defining the fundamental programming skills that are essential to The Carpentries way of learning will illuminate areas where AI can enhance existing lessons and support Instructors, but it will also flush out areas where AI use may be detrimental to a learner’s skill development. Throughout this process, we encourage The Carpentries to continue to call on the community for input. Additionally, as greater understanding of these tools becomes commonplace, we hope The Carpentries addresses AI ethics, appropriate use, and literacy skills in future iterations of the module.

One workshop does not solve every question about GenAI, but it does foster exactly the kinds of discussions needed for future work to be possible. We strongly encourage other Instructors to register and take part in this bonus module, share your observations and experiences with teaching with GenAI, so that the community can continue to support novice coders everywhere.