STACK Documentation

Documentation home | Category index | Site map

Potential response trees

The potential response tree is the algorithm which establishes the mathematical properties of the student's answer and assigns outcomes. For examples of how to use this, see the entry on improving feedback in the quick start guide.

When is the tree used?

Each potential response tree relies on one or more of the inputs. STACK automatically detects which elements are needed in the answer tests or feedback variables. The first time a student submits an input it is validated. The second time it is submitted it is available for assessment by a potential response tree. Only when all inputs upon which a tree relies are valid and submitted will the tree be traversed.

Before the tree is traversed

Each potential response tree can set Maxima's level of simplification. Before the tree is traversed the feedback variables are evaluated. The feedback variables may depend on the values of the question variables and the inputs. The values of these variables are available to the answer tests and all CASText fields within the tree, for example the feedback could be built using these variables.

Note, you cannot define a feedback variable with the same name as an input. For example, if your input is ans1 then it is tempting to define a feedback variable ans1:exdowncase(ans1) to ensure it is in lower case. Do not do this! Please use a different variable name. This is because in some situations the answer test will choose to take the raw value of ans1 exactly as the student typed it. Any redefinition will interfere with this process.

Traversing the tree

A potential response tree (technically an acyclic directed graph) consists of an arbitrary number of linked nodes we call potential responses.

In each node two expressions are compared using a specified answer tests, and the result is either true or false. A corresponding branch of the tree has the opportunity to each of the following.

  1. Adjust the score, (e.g. assign a value, add or subtract a value)
  2. Add written feedback specifically for the student
  3. Generate an "answer note", used by the teacher for evaluative assessment
  4. Nominate the next node, or end the process.


The outcomes are

  1. The raw score
  2. The penalty for this attempt
  3. Feedback to the student
  4. An answer note

Question Value

The potential response tree itself is expected to return a numerical raw score between \(0\) and \(1\). This number is multiplied by the question value before being returned to the student as feedback or recorded in the database.

Answer note

The answer note is a tag which is key for reporting purposes. It is designed to record the outcome of each answer test and the unique path through the tree. This is automatically generated, but can be changed to something meaningful. When looking for identical paths through the tree we have to do so, regardless of which random numbers were selected in this variant of the question given to a particular student. Hence, this string may not depend on any of the variables.

The answer note is the concatenation of each answer note from the answer tests and then the corresponding true/false branch. This note provides a record of the result of applying each test and the route taken through the tree.

This field is given a default value automatically and is used for reporting students' work.

This field may not be empty and for each node in a tree the string must be unique.

Scores and penalties

A score is generated by each potential response tree. Because the tree is only traversed when all inputs are valid, the score is only generated for a valid attempt.

If a score is generated it is based only on the current values of the inputs. This means that it is not based on either (1) previous values of the inputs, or (2) the number of previous attempts. (Requests have been made to enable attempt number to be available, but this has not been implemented yet.)

If a score is generated then a penalty is also generated. The penalty system is designed to encourage students to make repeated attempts in a formative setting. For example, a student is asked to find \( \int x^2, \mathrm{d}x\).

Attempt 1: \( x^3/3\). Score \(=0\), Penalty \(=0.1\), Feedback: "You have missed a constant of integration."

Attempt 2: \( x^3/3+c\). Score \(=1\), Penalty \(=0\), Feedback: "Well done."

Overall, the potential response tree returns the current score minus total penalties so far, in this example \(0.9\). This is multiplied by the "Question value" set in the potential response tree. These are summed across all potential response trees.

In this example, some colleagues would prefer to give partial credit for missing a constant of integration rather than zero marks and a penalty. In a formative setting, where students have an opportunity to have another attempt, the penalty system has been found to be an effective way to encourage students to have another attempt and to read the feedback. In an examination, where no feedback is available and so further attempts are not made, different choices need to be made and partial credit would be more appropriate than a zero mark.

  • The penalty is given a default value in the question. This is a mandatory field; the default for STACK is 0.1.
  • Penalties are cumulative, but the student will be given the maximum possible mark. I.e. while they accumulate penalties they are never worse off by repeatedly attempting the question. In particular, if the student in the above example makes another attempt and scores \(0\) they will retain their mark of \(0.9\). This is to encourage students to have another go in a formative setting. STACK generates a list of penalty adjusted scores for each attempt, and takes the maximum.
  • The penalty can be assigned a different value in the nodes of the potential response tree. This means, e.g., the teacher can assign a cumulative penalty for a particular answer.
  • The penalties are also controlled at a quiz level by the "question behaviours" mechanism for the quiz. Hence, if you set the behaviour as "Adaptive mode (no penalties)" the penalty assigned will be ignored when the question is used by students in the quiz.

Documentation home | Category index | Site map