This following contains the prompt structure used for few-shot classification of peer feedback comments using a Large Language Model (LLM). The goal was to categorize each comment according to the Code Review Taxonomy described in Appendix B. The prompt was formatted as a sequence of labeled examples that demonstrated how the taxonomy should be applied. These examples were selected and refined iteratively based on manual review to improve alignment with the intended classification scheme.
Prompt
You are an expert software engineering instructor helping to categorize peer code review comments using the following taxonomy:
(Code Review Taxonomy from Appendix B)
Based on the provided definitions, classify each student comment into one of the following categories: S+, S−, S0, SA, G+, G−, G0, GA, or OT. Respond only with the category label, with each label on a new line to make is simple to copy into Microsoft Excel. Use the most specific and relevant category. Do not include explanations or extra text.
User: “I liked how you used a while loop here, it made the logic very clear.”
Assistant: S+
User: “Your function doesn’t return the correct value when the input is negative.”
Assistant: S−
User: “You implemented a loop and an if-statement, which are required for this task.”
Assistant: S0
User: “Try breaking this into two functions to make it easier to follow. This makes the code more readable and easier to debug later.”
Assistant: SA
User: “Great job overall!”
Assistant: G+
User: “This wasn’t very good.”
Assistant: G−
User: “Here’s my feedback.”
Assistant: G0
User: “Next time, think about readability and keep your code DRY.”
Assistant: GA
User: “The weather is nice outside.”
Assistant: OT