expyriment.misc.data_preprocessing.Aggregator¶

class expyriment.misc.data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd', read_variables=None)¶

A class implementing a tool to aggregate Expyriment data.

This class is used to handle the multiple data files of a Experiment and process (i.e, aggregate) the data for further analysis

Examples

This tool helps, for instance, to aggregate your data for certain combinations of independent variables. E.g., data of a numerical magnitude judgement experiment. The code below makes a file with mean and median RTs and a second file with the errors and the number of trials:

from expyriment.misc import data_preprocessing
agg = data_preprocessing.Aggregator(data_folder= "./mydata/",
                        file_name = "MagnitudeJudgements")
agg.set_computed_variables(["parity = target_number % 2",
               "size = target_number > 65"])
agg.set_independent_variables(["hand", "size" , "parity"])

agg.set_exclusions(["trial_counter < 0",
                    "error != 0",
                    "RT < 2*std",
                    "RT > 2*std" # remove depending std in iv factor
                                 # combination for each subject
                    ])
agg.set_dependent_variables(["mean(RT)", "median(RT)"])
agg.aggregate(output_file="rts.csv")

agg.set_exclusions(["trial_counter < 0"])
agg.set_dependent_variables(["sum(error)", "n_trials"])
agg.aggregate(output_file="errors.csv")

__init__(data_folder, file_name, suffix='.xpd', read_variables=None)¶

Create an aggregator.

Parameters:

data_folder :str folder which contains of data of the subjects file_name : str name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files) suffix : str, optional if specified only files that end with this particular suffix will be considered (default=.xpd) read_variables : array of str, optional array of variable names, read only the specified variables

add_variables(variable_names, data_columns)¶

Adds a new variable to the data.

Parameters:	variable_names : str name of the new variable(s) data_columns : numpy.array the new data columns as numpy array

Notes

The amount of variables and added columns must match. The added data must also match the number of rows. Note, manually added variables will be lost if cases will be excluded afterwards via a call of the method set_exclusions.

added_variables¶

Getter for added variables.

aggregate(output_file=None, column_subject_id=0)¶

Aggregate the data as defined by the design.

The design will be printed and the resulting data will be return as numpy.array together with the variable names.

Parameters:	output_file : str, optional name of data output file. If this output_file is defined the function write the results as csv data file column_subject_id : int, optional data column containing the subject id (default=0)
Returns:	result : numpy.array new_variable_names : list of strings

computed_variables¶

Getter for computed variables.

concatenated_data¶

Getter for concatenated_data.

Returns:	data : numpy.array variables : list of str

Notes

Returns all data of all subjects as numpy.array and all variables names (including added variables). According to the defined design, the result contains the new computed variables and the subject variables from the headers of the Expyriment data files.

If data have been loaded and no new variable or exclusion has been defined the concatenated_data will merely return the previous data without re-processing.

data_files¶

Getter for data_files.

The list of the data files considered for the analysis.

data_folder¶

Getter for data_folder.

dependent_variables¶

Getter for dependent variables.

exclusions¶

Getter for exclusions.

file_name¶

Getter for file_name.

get_data(filename, recode_variables=True, compute_new_variables=True, exclude_trials=True)¶

Read data from from a single Expyriment data file.

Parameters:	filename : str name of the Expyriment data file recode_variables : bool, optional set to False if defined variable recodings should not be applied (default=True) compute_new_variables : bool, optional set to False if new defined variables should not be computed (default=True) exclude_trials : bool, optional set to False if exclusion rules should not be applied (default=True)
Returns:	data : numpy.array var_names : list list of variable names info : str subject info comment : str comments in data

Notes

The function can be only applied on data of aggregator.data_files, that is, on the files in the defined data folder that start with the experiment name. According to the defined design, the result contains recoded data together with the new computed variables, and the subject variables from the headers of the Expyriment data files.

get_variable_data(variables)¶

Returns the column of data as numpy array.

Parameters:	variables : list of str names of the variables to be extracted
Returns:	data : numpy.array

independent_variables¶

Getter for independent_variables.

print_n_trials(variables)¶

Print the number of trials in the combinations of the independent variables.

Parameters:	variables : str or list A string or a list of strings that represent the names of one or more data variables (aggregator.variables)

Notes

The functions is for instance useful to quickly check the experimental design.

reset(data_folder, file_name, suffix='.xpd', variables=None)¶

Reset the aggregator class and clear design.

Parameters:

data_folder : str folder which contains of data of the subjects file_name : str name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files) suffix : str, optional if specified only files that end with this particular suffix will be considered (default=.xpd) variables : array of str, optional array of variable names, process only the specified variables

set_computed_variables(compute_syntax)¶

Set syntax to compute new variables.

The method defines the variables, which will be computed. It can not be applied on variables that have been added manually via add_variables. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters:	compute_syntax : str or list A string or a list of strings that represent the syntax to compute the new variables

Notes

Compute Syntax:

{new-variable} = {variable} {relation/operation} {variable/value}
    {new-variable} -- a new not yet defined variable name
    {variable}     -- a defined data variable
    {relation}     --  ==, !=, >, <, >=, <=, => or <=
    {operation}    -- +, -, *, / or %
    {value}        -- string or numeric

set_dependent_variables(dv_syntax)¶

Set dependent variables.

Parameters:	dv_syntax : str or list syntax describing the dependent variable by a function and variable, e.g. mean(RT)

Notes

Syntax:

{function}({variable})
    {function} -- mean, median, sum, std or n_trials
                  Note: n_trials counts the number of trials
                  and does not require a variable as argument
    {variable} -- a defined data variable

set_exclusions(rule_syntax)¶

Set rules to exclude trials from the analysis.

The method indicates the rows, which are ignored while reading the data files. It can therefore not be applied on variables that have been added later via add_variables and results in a loss of all manually added variables. Setting exclusions requires re-reading of the data files and might be therefore time consuming. Thus, call this method always at the beginning of your analysis script.

Parameters:	rule_syntax : str or list A string or a list of strings that represent the rules to exclude trials

Notes

Rule syntax:

{variable} {relation} {variable/value}
    {variable}  -- a defined data variable
    {relation}  --  ==, !=, >, <, >=, <=, => or <=
    {value}     -- string or numeric

    If value is "{numeric} * std", trails are excluded in which
    the variable is below or above {numeric} standard deviations
    from the mean. The relations "==" and "!=" are not allow in
    this case. The exclusion criterion is apply for each subject
    and factor combination separately.

set_independent_variables(variables)¶

Set the independent variables.

Parameters:	variables : str or list the name(s) of one or more data variables (aggregator.variables)

set_subject_variables(variables)¶

Set subject variables to be considered for the analysis.

The method sets the subject variables. Subject variables are between subject factors or other variables defines in the subject information section (#s) of the Expyriment data file. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters:	variables : str or list A string or a list of strings that represent the subject variables

set_variable_recoding(recoding_syntax)¶

Set syntax to recode variables.

The method defines the variables, which will recoded. It can not be applied on variables that have been added later via add_variables. Recoding variables requires re-reading of the data files and might be therefore time consuming.

Parameters:	rule_syntax : str or list A string or a list of strings that represent the variable recoding syntax

Notes

Recoding syntax:

{variable}: {old_value1} = {new_value1}, {old_value2} = {new_value2},...

subject_variables¶

Getter for subject variable.

variable_recodings¶

Getter for variable recodings.

variables¶

Getter for variables.

The specified variables including the new computer variables and between subject variables and added variables.

write_concatenated_data(output_file=None, delimiter=', ')¶

Concatenates data and writes it to a csv file.

Parameters:	output_file : str, optional name of data output file If not specified data will the save to {file_name}.csv delimiter : str delimiter character (default=”,”)

write_concatenated_data_to_R_data_frame(output_file)¶

Creates a R data frame of the concatenated data and stores it in a RDS file.

Parameters:	output_file : str, optional name of RDS output file If not specified data will the save to {file_name}.csv

Notes

This method requires R and the Python package ‘Rpy2’.