deepSTRF.models.video package

Submodules

deepSTRF.models.video.video_model module

class deepSTRF.models.video.video_model.VideoEncodingModel(spatial_resol, temporal_window_size: int, out_neurons: int = 1, output_activation: Module = None, *args, **kwargs)

Bases: NeuralModel

General mother class for ENCODING models of VIDEO sensory neural responses.

The forward() method takes as input a single-channel spectrogram of shape (B, C, H, W, T)

TODO: prefiltering_dict example

Initialize internal Module state, shared by both nn.Module and ScriptModule.

STRF_gradmap(T=None)

Get the SPATIO-Temporal Receptive Field (STRF) of the OUTPUT neurons, with a history of T timesteps, as

the changes in the stimulus that elicit an increase in output activity.

cf. Rançon et al. (2025), “Temporal recurrence as a general mechanism to explain neural responses in

the auditory system”, BioRxiv

Returns a (N, C, H, W, T) tensor

TODO:

• handle multiple input channels (on & off) because of adaptrans ?

• allow custom losses ? (e.g. sustained activity rather than last spike ?)

validate()

Check that the instance is deepSTRF-compatible.

Subclasses should call super().validate() and then add their own checks (e.g. AudioEncodingModel checks F, T > 0).

Raises:

AssertionError – If self.O is not a positive int, if readout is unset or not an torch.nn.Module, or if any of the wav2spec / prefiltering / core slots is not an torch.nn.Module.

Module contents

class deepSTRF.models.video.VideoEncodingModel(spatial_resol, temporal_window_size: int, out_neurons: int = 1, output_activation: Module = None, *args, **kwargs)

Bases: NeuralModel

General mother class for ENCODING models of VIDEO sensory neural responses.

The forward() method takes as input a single-channel spectrogram of shape (B, C, H, W, T)

TODO: prefiltering_dict example

Initialize internal Module state, shared by both nn.Module and ScriptModule.

STRF_gradmap(T=None)

Get the SPATIO-Temporal Receptive Field (STRF) of the OUTPUT neurons, with a history of T timesteps, as

the changes in the stimulus that elicit an increase in output activity.

cf. Rançon et al. (2025), “Temporal recurrence as a general mechanism to explain neural responses in

the auditory system”, BioRxiv

Returns a (N, C, H, W, T) tensor

TODO:

• handle multiple input channels (on & off) because of adaptrans ?

• allow custom losses ? (e.g. sustained activity rather than last spike ?)

validate()

Check that the instance is deepSTRF-compatible.

Subclasses should call super().validate() and then add their own checks (e.g. AudioEncodingModel checks F, T > 0).

Raises:

AssertionError – If self.O is not a positive int, if readout is unset or not an torch.nn.Module, or if any of the wav2spec / prefiltering / core slots is not an torch.nn.Module.