expyriment.misc.data_preprocessing.Aggregator

class expyriment.misc.data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd', read_variables=None)

A class implementing a tool to aggregate Expyriment data.

This class is used to handle the multiple data files of a Experiment and process (i.e, aggregate) the data for further analysis

Examples

This tool helps, for instance, to aggregate your data for certain combinations of independent variables. E.g., data of a numerical magnitude judgement experiment. The code below makes a file with mean and median RTs and a second file with the errors and the number of trials:

from expyriment.misc import data_preprocessing
agg = data_preprocessing.Aggregator(data_folder= "./mydata/",
                        file_name = "MagnitudeJudgements")
agg.set_computed_variables(["parity = target_number % 2",
               "size = target_number > 65"])
agg.set_independent_variables(["hand", "size" , "parity"])

agg.set_exclusions(["trial_counter < 0",
                    "error != 0",
                    "RT < 2*std",
                    "RT > 2*std" # remove depending std in iv factor
                                 # combination for each subject
                    ])
agg.set_dependent_variables(["mean(RT)", "median(RT)"])
agg.aggregate(output_file="rts.csv")

agg.set_exclusions(["trial_counter < 0"])
agg.set_dependent_variables(["sum(error)", "n_trials"])
agg.aggregate(output_file="errors.csv")

Methods

__init__(data_folder, file_name, suffix='.xpd', read_variables=None)

Create an aggregator.

Parameters:

data_folder :str :

folder which contains of data of the subjects

file_name : str

name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files)

suffix : str, optional

if specified only files that end with this particular suffix will be considered (default=.xpd)

read_variables : array of str, optional

array of variable names, read only the specified variables

add_variables(variable_names, data_columns)

Adds a new variable to the data.

Parameters:

variable_names : str

name of the new variable(s)

data_columns : numpy.array

the new data columns as numpy array

Notes

The amount of variables and added columns must match. The added data must also match the number of rows. Note, manually added variables will be lost if cases will be excluded afterwards via a call of the method set_exclusions.

added_variables

Getter for added variables.

aggregate(output_file=None, column_subject_id=0)

Aggregate the data as defined by the design.

The design will be printed and the resulting data will be return as numpy.array together with the variable names.

Parameters:

output_file : str, optional

name of data output file. If this output_file is defined the function write the results as csv data file

column_subject_id : int, optional

data column containing the subject id (default=0)

Returns:

result : numpy.array

new_variable_names : list of strings

computed_variables

Getter for computed variables.

concatenated_data

Getter for concatenated_data.

Returns:

data : numpy.array

variables : list of str

Notes

Returns all data of all subjects as numpy.array and all variables names (including added variables). According to the defined design, the result contains the new computed variables and the subject variables from the headers of the Expyriment data files.

If data have been loaded and no new variable or exclusion has been defined the concatenated_data will merely return the previous data without re-processing.

data_files

Getter for data_files.

The list of the data files considered for the analysis.

data_folder

Getter for data_folder.

dependent_variables

Getter for dependent variables.

exclusions

Getter for exclusions.

file_name

Getter for file_name.

get_data(filename, recode_variables=True, compute_new_variables=True, exclude_trials=True)

Read data from from a single Expyriment data file.

Parameters:

filename : str

name of the Expyriment data file

recode_variables : bool, optional

set to False if defined variable recodings should not be applied (default=True)

compute_new_variables : bool, optional

set to False if new defined variables should not be computed (default=True)

exclude_trials : bool, optional

set to False if exclusion rules should not be applied (default=True)

Returns:

data : numpy.array

var_names : list

list of variable names

info : str

subject info

comment : str

comments in data

Notes

The function can be only applied on data of aggregator.data_files, that is, on the files in the defined data folder that start with the experiment name. According to the defined design, the result contains recoded data together with the new computed variables, and the subject variables from the headers of the Expyriment data files.

get_variable_data(variables)

Returns the column of data as numpy array.

Parameters:

variables : list of str

names of the variables to be extracted

Returns:

data : numpy.array

independent_variables

Getter for independent_variables.

print_n_trials(variables)

Print the number of trials in the combinations of the independent variables.

Parameters:

variables : str or list

A string or a list of strings that represent the names of one or more data variables (aggregator.variables)

Notes

The functions is for instance useful to quickly check the experimental design.

reset(data_folder, file_name, suffix='.xpd', variables=None)

Reset the aggregator class and clear design.

Parameters:

data_folder : str

folder which contains of data of the subjects

file_name : str

name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files)

suffix : str, optional

if specified only files that end with this particular suffix will be considered (default=.xpd)

variables : array of str, optional

array of variable names, process only the specified variables

set_computed_variables(compute_syntax)

Set syntax to compute new variables.

The method defines the variables, which will be computed. It can not be applied on variables that have been added manually via add_variables. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters:

compute_syntax : str or list

A string or a list of strings that represent the syntax to compute the new variables

Notes

Compute Syntax:

{new-variable} = {variable} {relation/operation} {variable/value}
    {new-variable} -- a new not yet defined variable name
    {variable}     -- a defined data variable
    {relation}     --  ==, !=, >, <, >=, <=, => or <=
    {operation}    -- +, -, *, / or %
    {value}        -- string or numeric

set_dependent_variables(dv_syntax)

Set dependent variables.

Parameters:

dv_syntax : str or list

syntax describing the dependent variable by a function and variable, e.g. mean(RT)

Notes

Syntax:

{function}({variable})
    {function} -- mean, median, sum, std or n_trials
                  Note: n_trials counts the number of trials
                  and does not require a variable as argument
    {variable} -- a defined data variable

set_exclusions(rule_syntax)

Set rules to exclude trials from the analysis.

The method indicates the rows, which are ignored while reading the data files. It can therefore not be applied on variables that have been added later via add_variables and results in a loss of all manually added variables. Setting exclusions requires re-reading of the data files and might be therefore time consuming. Thus, call this method always at the beginning of your analysis script.

Parameters:

rule_syntax : str or list

A string or a list of strings that represent the rules to exclude trials

Notes

Rule syntax:

{variable} {relation} {variable/value}
    {variable}  -- a defined data variable
    {relation}  --  ==, !=, >, <, >=, <=, => or <=
    {value}     -- string or numeric

    If value is "{numeric} * std", trails are excluded in which
    the variable is below or above {numeric} standard deviations
    from the mean. The relations "==" and "!=" are not allow in
    this case. The exclusion criterion is apply for each subject
    and factor combination separately.

set_independent_variables(variables)

Set the independent variables.

Parameters:

variables : str or list

the name(s) of one or more data variables (aggregator.variables)

set_subject_variables(variables)

Set subject variables to be considered for the analysis.

The method sets the subject variables. Subject variables are between subject factors or other variables defines in the subject information section (#s) of the Expyriment data file. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters:

variables : str or list

A string or a list of strings that represent the subject variables

set_variable_recoding(recoding_syntax)

Set syntax to recode variables.

The method defines the variables, which will recoded. It can not be applied on variables that have been added later via add_variables. Recoding variables requires re-reading of the data files and might be therefore time consuming.

Parameters:

rule_syntax : str or list

A string or a list of strings that represent the variable recoding syntax

Notes

Recoding syntax:

{variable}: {old_value1} = {new_value1}, {old_value2} = {new_value2},...

subject_variables

Getter for subject variable.

variable_recodings

Getter for variable recodings.

variables

Getter for variables.

The specified variables including the new computer variables and between subject variables and added variables.

write_concatenated_data(output_file=None, delimiter=', ')

Concatenates data and writes it to a csv file.

Parameters:

output_file : str, optional

name of data output file If not specified data will the save to {file_name}.csv

delimiter : str

delimiter character (default=”,”)

write_concatenated_data_to_R_data_frame(output_file)

Creates a R data frame of the concatenated data and stores it in a RDS file.

Parameters:

output_file : str, optional

name of RDS output file If not specified data will the save to {file_name}.csv

Notes

This method requires R and the Python package ‘Rpy2’.