deepSTRF.datasets.video package

Submodules

deepSTRF.datasets.video.video_dataset module

class deepSTRF.datasets.video.video_dataset.VideoNeuralDataset(path: str, dt_ms: float)[source]

Bases: NeuralDataset

Neural dataset class for video stimuli

Stimuli are in the form of (C, H, W, T) tensors TODO: description

add_noise_to_videos(video_tensor, noise_level=0.1)[source]: Add random Gaussian noise to the video stimuli.

change_spatial_resolution(video_tensor, spatial_resol)[source]

Change the spatial resolution of the video stimuli.

Handles video_tensor of shape (S, C, H, W, T) or (N, S, C, H, W, T).

change_temporal_resolution(native_temporal_resolution_video, video_tensor, native_temporal_resolution_responses, responses_tensor, new_dt)[source]

Change the temporal resolution of both video stimuli and responses.

Handles video tensors of shape (S, C, H, W, T) or (N, S, C, H, W, T).

Args:: native_temporal_resolution_video (float): Native video time bin size (ms). video_tensor (torch.Tensor): Video tensor. native_temporal_resolution_responses (float): Native response time bin size (ms). responses_tensor (torch.Tensor): Responses tensor of shape (N, S, R, T). new_dt (float): New desired time bin size (ms).
Returns:: responses_out (torch.Tensor): Tensor with updated time resolution for responses. videos_out (torch.Tensor): Tensor with updated time resolution for videos. new_T (int): New time dimension length.

change_video_temporal_resolution(video, native_dt: int, target_dt: int)[source]

Changes the temporal resolution of a video by simple subsampling.

Args:: video (torch.Tensor): Video tensor of shape (C, H, W, T). native_dt (int): Native temporal resolution in milliseconds (e.g., 12). target_dt (int): Target temporal resolution in milliseconds.
Returns:: torch.Tensor: Resampled video tensor. int: New time dimension T after resampling.

from_rgb_to_grayscale(video_tensor)[source]: Convert RGB video stimuli to grayscale.

get_dt()[source]: Returns the time step

get_responses_signal_power(responses_tensor)[source]: Compute the signal power for the neural responses.

normalize_responses(responses_tensor)[source]: Normalize the neural responses by dividing it by the time dimension.

normalize_videos(video_tensor)[source]: Normalize the video stimuli by dividing it by the time dimension.

smooth_responses(responses_tensor, sigma_time_step=2, kernel_size=11, padding=5)[source]: Smooth the neural responses using a Gaussian kernel.

split_into_clips(clip_length_frames: int)[source]

Splits the responses and videos along the time dimension into fixed-length clips. If the total number of time bins is not divisible by the clip length, the last clip is zero-padded.

Args:: clip_length_frames (int): Desired length of each clip, in number of frames/time bins.
Returns:: responses_out (torch.Tensor): Shape (N, S*n_clips, R, T_clip) videos_out (torch.Tensor): Shape (S*n_clips, C, H, W, T_clip)

Module contents

class deepSTRF.datasets.video.VideoNeuralDataset(path: str, dt_ms: float)[source]