expyriment.misc.data_preprocessing.Aggregator

class expyriment.misc.data_preprocessing.Aggregator(data_folder, file_name, suffix='.xpd')[source]

A class implementing a tool to aggregate expyriment data.

This class is used to handle the multiple data files of a experiment and preprocess (i.e, aggregate) the data for further analysis

Examples

This tool helps, for instance, to aggregate your data for certain combinations of independent variables. E.g., data of a numerical magnitude judgement experiment. The code below makes a file with mean and median RTs and a second file with the errors and the number of trials:

from expyriment.misc import data_preprocessing
agg = data_preprocessing.Aggregator(data_folder= "./mydata/",
                        file_name = "MagnitudeJudgements")
agg.set_computed_variables(["parity = target_number % 2",
               "size = target_number > 65"])
agg.set_independent_variables(["hand", "size" , "parity"])

agg.set_exclusions(["trial_counter < 0", "error != 0"])
agg.set_dependent_variables(["mean(RT)", "median(RT)"])
agg.aggregate(output_file="rts.csv")

agg.set_exclusions(["trial_counter < 0"])
agg.set_dependent_variables(["sum(error)", "n_trials"])
agg.aggregate(output_file="errors.csv")

Methods

__init__(data_folder, file_name, suffix='.xpd')[source]

Create an aggregator.

Parameters :

data_folder :str :

folder which contains of data of the subjects

file_name : str

name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files)

suffix : str, optional

if specified only files that end with this particular suffix will be considered (default=.xpd)

add_variables(variable_names, data_columns)[source]

Adds a new variable to the data.

Parameters :

variable_names : str

name of the new variable(s)

data_columns : numpy.array

the new data columns as numpy array

Notes

The amount of variables and added columns must match. The added data must also match the number of rows. Note, manually added variables will be lost if cases will be excluded afterwards via a call of the method set_exclusions.

added_variables[source]

Getter for added variables.

aggregate(output_file=None, column_subject_id=0)[source]

Aggregate the data as defined by the design.

The design will be printed and the resulting data will be return as numpy.array together with the variable names.

Parameters :

output_file : str, optional

name of data output file. If this output_file is defined the function write the results as csv data file

subject variable : int, optional

data column containing the subject id (odefault=0)

Returns :

result : numpy.array

new_variable_names : list of strings

computed_variables[source]

Getter for computed variables.

concatenated_data[source]

Getter for concatenated_data.

Returns :

data : numpy.array

variables : list of str

Notes

Returns all data of all subjects as numpy.array and all variables names (including added variables). According to the defined design, the result contains the new computed variables and the subject variables from the headers of the Expyriment data files.

If data have been loaded and no new variable or exclusion has been defined the concatenated_data will merely return the previous data without re-processing.

data_files[source]

Getter for data_files.

The list of the data files considered for the analysis.

data_folder[source]

Getter for data_folder.

dependent_variables[source]

Getter for dependent variables.

exclusions[source]

Getter for exclusions.

file_name[source]

Getter for file_name.

get_data(filename, recode_variables=True, compute_new_variables=True, exclude_trials=True)[source]

Read data from from Expyriment data file.

Parameters :

filename : str

name of the Expyriment data file

recode_variables : bool, optional

set to False if defined variable recodings should not be applied (default=False)

compute_new_variables : bool, optional

set to False if new defined variables should not be computed (default=False)

exclude_trials : bool, optional

set to False if exclusion rules should not be applied (default=False)

Returns :

data : numpy. arry

var_names : list

list of variable names

info : str

subject info

comment : str

comments in data

Notes

The function can be only applied on data that in aggregator.data_files, that is, on the files that in defined data folder and the start with the experiment name. According to the defined design, the result contains recoded data together with the new computed variables, and the subject variables from the headers of the Expyriment data files.

get_variable_data(variables)[source]

Returns the column of data as numpy array.

Parameters :

variables : list of str

names of the variables to be extracted

Returns :

data : numpy.array

independent_variables[source]

Getter for independent_variables.

print_n_trials(variables)[source]

Print the number of trials in the combinations of the independent variables.

Parameters :

variables : str or list

A string or a list of strings that represent the names of one or more data variables (aggregator.variables)

Notes

The functions is for instance useful to quickly check the experimental design.

reset(data_folder, file_name, suffix='.xpd')[source]

Reset the aggregator class and clear design.

Parameters :

data_folder : str

folder which contains of data of the subjects

file_name : str

name of the files. All files that start with this name will be considered for the analysis (cf. aggregator.data_files)

suffix : str, optional

if specified only files that end with this particular suffix will be considered (default=.xpd)

set_computed_variables(compute_syntax)[source]

Set syntax to compute new variables.

The method defines the variables, which will be computed. It can not be applied on variables that have been added manually via add_variables. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters :

compute_syntax : str or list

A string or a list of strings that represent the syntax to compute the new variables

Notes

Compute Syntax:

{new-variable} = {variable} {relation/operation} {variable/value}
    {new-variable} -- a new not yet defined variable name
    {variable}     -- a defined data variable
    {relation}     --  ==, !=, >, <, >=, <=, => or <=
    {operation}    -- +, -, *, / or %
    {value}        -- string or numeric
set_dependent_variables(dv_syntax)[source]

Set dependent variables.

Parameters :

dv_syntax : str or list

syntax describing the dependent variable by a function and variable, e.g. mean(RT)

Notes

Syntax:

{function}({variable})
    {function} -- mean, median, sum, std or n_trials
                  Note: n_trials counts the number of trials
                  and does not require a variable as argument
    {variable} -- a defined data variable
set_exclusions(rule_syntax)[source]

Set rules to exclude trials from the analysis.

The method indicates the rows, which are ignored while reading the data files. It can therefore not be applied on variables that have been added later via add_variables and results in a loss of all manually added variables. Setting exclusions requires re-reading of the data files and might be therefore time consuming. Thus, call this mathod always at the beginning of your analysis script.

Parameters :

rule_syntax : str or list

A string or a list of strings that represent the rules to exclude trials

Notes

Rule syntax:

{variable} {relation} {variable/value}
    {variable}  -- a defined data variable
    {relation}  --  ==, !=, >, <, >=, <=, => or <=
    {value}     -- string or numeric
set_independent_variables(variables)[source]

Set the independent variables.

Parameters :

variables : str or list

the name(s) of one or more data variables (aggregator.variables)

set_subject_variables(variables)[source]

Set subject variables to be considered for the analysis.

The method sets the subject variables. Subject variables are between subject factors or other variables defines in the subject information section (#s) of the Expyriment data file. The method requires a re-reading of the data files and might be therefore time consuming.

Parameters :

variables : str or list

A string or a list of strings that represent the subject variables

set_variable_recoding(recoding_syntax)[source]

Set syntax to recode variables.

The method defines the variables, which will recoded. It can not be applied on variables that have been added later via add_variables. Recoding variables requires re-reading of the data files and might be therefore time consuming.

Parameters :

rule_syntax : str or list

A string or a list of strings that represent the variable recoding syntax

Notes

Recoding syntax:

{variable}: {old_value1} = {new_value1}, {old_value2} = {new_value2},...
subject_variables[source]

Getter for subject variable.

variable_recodings[source]

Getter for variable recodings.

variables[source]

Getter for variables.

The specified variables including the new computer variables and between subject variables and added variables.

write_concatenated_data(output_file=None, delimiter=', ')[source]

Concatenate data and write it to a csv file.

Parameters :

output_file : str, optional

name of data output file If no specified data will the save to {file_name}.csv

delimiter : str

delimiter character (default=”,”)

Previous topic

expyriment.misc.data_preprocessing

Next topic

expyriment.misc.defaults

This Page