FAQ

From Panamath

(Difference between revisions)
Jump to: navigation, search
Line 25: Line 25:
==What is the relationship between performance on the panamath test and that on number line placement?==
==What is the relationship between performance on the panamath test and that on number line placement?==
This is still being studied. Our expectation is that line performance will correlate much better with CV (shout out a number) than with w (who has more), because shout out involves both approximation and mapping to discrete symbols as does the line task.
This is still being studied. Our expectation is that line performance will correlate much better with CV (shout out a number) than with w (who has more), because shout out involves both approximation and mapping to discrete symbols as does the line task.
 +
 +
==Does a participant's reaction time (RT) to respond on a trial affect their performance (e.g., will some individuals respond quickly and with low accuracy while other respond slowly with greater accuracy?==
 +
Both RT and w can carry information about a participant's performance.  For some participants there is a "speed-accuracy tradeoff" where they may be slow and accurate or fast and inaccurate.  For other participants, both RT and w may indicate difficulties (e.g., participants who are particularly challenged have shown longer RTs and lower accuracy compared to their peers).  For this reason, Panamath will report both w and mean RT to respond for each participant.  We recommend  using mean RT to respond as a second variable along with w in all regressions. 
 +
The experimenter can suggest to a participant that they should "slow down" or "be more careful" if they feel that the participant is moving too quickly through the trials.  The experimenter might also remind a participant that they should "give your best guess as soon as you can" if they feel the participant is moving too slowly through the trials (e.g., especially if the experimenter believes the participant is attempting to verbally count the number of items in their head based on a remembered image of the screen. This type of counting is not what Panamath is trying to measure.  Note, however, that the limits of Visual Working Memory, and the presence of a mask, which is standard in Panamath, makes this type of mental counting from memory a difficult strategy to employ and it should arise in very few participants).  The experimenter need not mention any of this feedback and Panamath should be robust in the face of different strategies so long as both RT and w are included in subsequent analyses. In most cases, subjects hit upon similar strategies, or strategies that fall on a speed-accuracy tradeoff continuum, and Panamath can estimate internal precision even if different strategies are used.  One exception may be young children where it is very important that the experimenter ensures that the child understands the task and is engaging it appropriately.  For children, at any point during testing, the experimenter can give "slow down" or "best guess" verbal feedback like the above comments and can also remind the child to make their best guess about "which color had more dots in number; the sizes of the dots doesn't matter, just which has more in number".  We find that such reminders can help children stay on task."

Revision as of 21:31, 10 October 2011

Contents

Can I skip the first instruction screen?

Yes. The instruction screen can be skipped by going to "Set Advanced Settings" on the main screen, under "General," check the "Skip instructions" box.

How should the feedback sounds be explained to children?

With kids 6-10 you can tell them what the beeps mean (high beep means correct, low beep for incorrect response.) For children 3-5 we find that having beeps on but not mentioning them is better. With kids 6-10, even if the beeps are not mentioned, they don't ask about them. It is set up to sound like the computer saying "I got your answer." Either way (mentioning/ not mentioning) is fine.

Are the kids supposed to learn from their mistakes based on the sound feedback?

Feedback is a complicated issue with both children and adults. We have seen that the performance of children can be strongly negatively affected by hearing negative feedback during the task. This effect is most pronounced for young children (e.g., 3-4 year olds) and for explicit verbal feedback (e.g., "oh, that's not right"), but the negative effects of feedback can also be seen at older ages (e.g., ages 6-10). All of these effects are compounded if a participant gets multiple trials in a row wrong (as can sometimes happen with younger children). At the same time, both children and adults benefit from being given feedback. Their performance improves when they hear wether they have gotten the previous trial right or wrong. Evidence suggests that the feedback helps them gauge their perceptual biases and correct for these biases leading to improved performance. It is not simply that it is more engaging to hear the feedback. However, it is true that the task is more engaging with the feedback and this helps keep children and adults on task (e.g., the timing of the feedback was chosen to give a natural feel that the program is responding to the subject, this is engaging even if the right/wrong nature of the feedback is not stressed). For all these reasons, there is not a simple answer as to which approach to feedback is best. If a solid estimate of a participant's abilities is what is desired, having the sounds present while not explicitly mentioning the right/wrong nature of the tones is a good approach. Calling young children's attention to the right/wrong nature of the feedback runs a risk of leading children to be stressed and perform more poorly (e.g., the deleterious effects of explicit negative feedback could become a manipulation of interest for a researcher investigating math anxiety). With older children and adults, this is less of a concern and it may be possible to tell them explicitly about the right/wrong nature of the feedback, but for adults and teenagers explicitly mentioning the right/wrong nature of the feedback is not necessary as they will rapidly figure this out for themselves. In an attempt to get the benefits of a more engaging task while minimizing the potential negative effects of anxiety, we suggest having the feedback sounds present but not stressing the right/wrong nature to subjects, especially young children. A task with no beeps is very boring for the children. A task that clearly tells them they are wrong on certain trials is unpleasant for them. But there is no hard and fast rule in this area. The effects of feedback across ages is a current research question we are exploring.

What if a child gets all of the practice items wrong? Do I still proceed with the test trials?

Usually, we don't find terrible performance early on. You should make sure the child knows exactly what they are supposed to do. You can tell them everything about the task-including that sometimes the sizes of dots are different, etc. and that they should always answer based on number. The kids generally find this to be very intuitive. Just try to have your instruction finished before trial number 6 or so, and from that point on you can just pepper in, "You are doing great!" and positive comments like that whenever you feel it is necessary. Because it is psychophysics, we want the kids to know exactly what the game is and be comfortable with it. Then, the measure is focused on their internal ability and not on their understanding of what is expected of them. You can give lots of feedback up through the first 1-6 trials (6 isn't a magic number, it is just that by trial 6 any confusions have likely been cleared up.)

Can children manage responding reliably on a laptop keyboard?

Yes. We use a laptop for all of our in person testing.

For the results folders, apart from having a unique, recognizable identifier for each subject, what would be a good organizing principle (age, sex, school, etc.) for the results?

It would be good to retain any info that you think might be a quesiton of interest. e.g., school, sex, etc. Age should be in the output file, but you might like to keep DOB for more precise info. Making long file names is fine and is a good way to keep the info.

Will gender differences affect the results of Panamath?

While we have seen some differences in precision and especially precision between boys and girls at older ages, at the moment, any such difference appears to be attributable to factors other than sex (eg. interest in mathematics, self selection for engaging in studies of mathematics, etc).

What is the relationship between performance on the panamath test and that on number line placement?

This is still being studied. Our expectation is that line performance will correlate much better with CV (shout out a number) than with w (who has more), because shout out involves both approximation and mapping to discrete symbols as does the line task.

Does a participant's reaction time (RT) to respond on a trial affect their performance (e.g., will some individuals respond quickly and with low accuracy while other respond slowly with greater accuracy?

Both RT and w can carry information about a participant's performance. For some participants there is a "speed-accuracy tradeoff" where they may be slow and accurate or fast and inaccurate. For other participants, both RT and w may indicate difficulties (e.g., participants who are particularly challenged have shown longer RTs and lower accuracy compared to their peers). For this reason, Panamath will report both w and mean RT to respond for each participant. We recommend using mean RT to respond as a second variable along with w in all regressions. The experimenter can suggest to a participant that they should "slow down" or "be more careful" if they feel that the participant is moving too quickly through the trials. The experimenter might also remind a participant that they should "give your best guess as soon as you can" if they feel the participant is moving too slowly through the trials (e.g., especially if the experimenter believes the participant is attempting to verbally count the number of items in their head based on a remembered image of the screen. This type of counting is not what Panamath is trying to measure. Note, however, that the limits of Visual Working Memory, and the presence of a mask, which is standard in Panamath, makes this type of mental counting from memory a difficult strategy to employ and it should arise in very few participants). The experimenter need not mention any of this feedback and Panamath should be robust in the face of different strategies so long as both RT and w are included in subsequent analyses. In most cases, subjects hit upon similar strategies, or strategies that fall on a speed-accuracy tradeoff continuum, and Panamath can estimate internal precision even if different strategies are used. One exception may be young children where it is very important that the experimenter ensures that the child understands the task and is engaging it appropriately. For children, at any point during testing, the experimenter can give "slow down" or "best guess" verbal feedback like the above comments and can also remind the child to make their best guess about "which color had more dots in number; the sizes of the dots doesn't matter, just which has more in number". We find that such reminders can help children stay on task."

Personal tools