Tutorial 3: How to Determine w Using Our Automated Scripts
This is a tutorial to help you compute w for your Panamath data using our automated scripts.
This tutorial is a work in progress!
Currently, the tutorial only works for Windows machines with Microsoft Excel.
Step 1: Download software and scripts
Download the latest version of the statistical analysis software, R: http://cran.r-project.org/bin/windows/base/
Download our analysis scripts: http://www.panamath.org/scripts/calculate_w_panamath_v2.0.zip. Unzip the .zip file.
Step 2: Collect the results files
Go to the unzipped directory (folder) in your Windows Explorer. There should be a directory called "analysis". Place all of the Excel results files generated by Panamath into the unzipped directory, at the same lever as the "analysis" directory. You should be able to find all of the Panamath results files in the "results" directory of your Panamath program root directory.
Step 3: Change the path to R for your computer
Double-click the "analysis" directory to enter it. There are four files:
- "DotsData.xls" - this file contains an Excel macro that will copy and combine data from your Excel results files into a single "DotsData.csv" file that it will generate.
- "RunWeberAnalysis.bat" - this batch file tells Windows to run the R script on the "DotsData.csv" data file.
- Your job: Right-click "RunWeberAnalysis.bat" and select "Edit" from the options menu. The file should open in Notepad. Change "D:\Documents\Program Files\R\R-2.15.2\" to be whatever the path to R is on your computer. This R directory should have a directory called "bin" within. Save the file and close Notepad.
- "calc_w_simple.r" and "calc_w_simple_fcns.r" - these two files contain the R script that reads the "DotsData.csv" file generated by "dotsSummary.xls" and computes Weber fractions and other statistics for each subject.
Step 4: Run the scripts
Double-click "dotsSummary.xls" to open it. You should be asked whether to enable macros. Click "Enable Content". There should be four buttons in the Excel worksheet you open it. The file explains what each button does. To generate Weber fractions for each subject in your set of results files, simply click the "Do all three!" button.
Step 5: Review the Weber fractions
A file "WeberResults.csv" will be generated by the previous step. Close the "dotsSummary.xls" file and double-click the "WeberResults.csv" to open it.
Each row in the file (after the header row) represents the statistics calculated for each subject, identified by the subject id in the first column.
Under default Panamath settings, there are four bins for the numerosity ratios (# dots in the larger set / # dots in the smaller set) presented to the subject. The lower bound of the four ratio bins are labeled in the first row, second through fifth columns [at some point, this may change to be the center of the ratio bins, as is shown in the summary worksheet of each Panamath results file]. Subject accuracy, in percent correct, on each for trials in each of the ratio bins is shown below those labels.
In the sixth column, labeled "w.ls", are the Weber fractions computed for each subject using the nonlinear Gauss-Newton least-squares fitting method. If values in this column are "NA", that means that either the corresponding subjects performed awesomely at 100% or very close to 100% and so a Weber fraction could not be fit, or they performed at or below chance level that a Weber fraction could not be fit. Note that the Weber curve is an ogive and cannot drop below 50% (chance performance). In these cases, the subject's Weber fraction is effectively Infinity, and the subject should retake the test either under the same conditions or with easier number ratios in order to get a better estimate of the subject's Weber fraction.
The sixth and seventh columns contain the percent correct and average response time (RT). This includes display time since under default Panamath settings, the subject can respond before the dots display disappears. Both Weber fraction and RT (or accuracy and RT) are measures of the precision of a subject's Approximate Number System.
The eighth and ninth columns contain the number of trials and number of RT outliers for each subject.
By default, trials with outlier RTs are removed from each subject's data before computing Weber fraction, percent correct, or average RT. An RT value is deemed an outlier according to the non-recursive Jolicouer method for determining outliers (see Van Selts, M. & Jolicoeur, P. (1994). A solution to the effect of sample size on outlier elimination. Quarterly Journal of Experimental Psychology, 47A, 631-650.) This method deems a value an outlier if it is more than X standard deviations away from the mean, where X is dependent on the number of trials, increasing with more trials. If there are only 20 trials, this method equates to marking all RTs more than 2.326 standard deviations away from the mean (in either direction) as outliers. If there are more than 100 trials, this method simplifies to marking all RTs more than 2.5 standard deviations away from the mean as outliers.