list of starting_times for external_files in ImageSeries

[Saksham20]

Currently, for an ImageSeries if external_files is specified as a list of videos, there is no way to specify a list of starting_times corresponding to each of the videos.
@CodyCBakerPhD had brought this issue up #1318

Describe the solution you'd like
I think, with format="external" we should allow a list of either starting_times or timestamps (list of arrays).

Code:

# currently what works:
img_srs_external = ImageSeries(name='external igs', format='external',rate=2.0,starting_time=0.0,
                               external_file=["temp1.avi","temp2.avi"]) # this works

# what is requested: 
img_srs_external = ImageSeries(name='externaligs', format='external',rate=2.0, starting_time=[0.0, 100.0],
                               external_file=["temp1.avi","temp2.avi"])
# OR:
img_srs_external = ImageSeries(name='externaligs', format='external', timestamps=[np.arange(100), np.arange(101,200)],
                               external_file=["temp1.avi","temp2.avi"])

Checklist

• Have you ensured the feature or change was not already reported?
• Have you included a brief and descriptive title?
• Have you included a clear description of the problem you are trying to solve?
• Have you included a minimal code snippet that reproduces the issue you are encountering?
• Have you checked our Contributing document?

[oruebel]

@Saksham20 please correct me if I missed anything. Based on our discussions I believe we decided that:

1. For the case where users have multiple non-consecutive movies that are distinct recordings, creating separate ImageSeries for the movies is approbriate
2. For the case where a single recording is broken down into consecutive movies (i.e., where movie 2 starts right when movie 1 ends), having a single starting_time and rate is sufficient
3. For the case where a single recording is broken down into consecutive movies that have gaps in time (i.e., movie 2 starts some time after movie 1) the user should provide a single array with the explicit timestamps for all frames. In this case if movie 1 has 100 frames and movie 2 has 50 then the first 100 timestamps would refer to movie 1 and the next 50 timestamps to movie 2 and so on.

This approach avoids the need for breaking changes to the schema. While the approach proposed here is potentially useful, we do not have a specific use-case right now that requires this features and where the above alternate solutions are not working. We will revisit this issue when a specific use case for this comes up.

We also discussed the need for helper functions in PyNWB for ImageSeries to make the above approaches easier to use for users, e.g,.,:

• Map timestamps to particular frame in a particular video to make it easy to find the frame for a given timestamp

[Saksham20]

Yes, that is correct. For trialized videos, the user can specify the timestamps reflecting the times of each frame in every video. The length of the timestamps should be the sum of frame count of all videos specified in external_files.

Currently, we are working on a helper function (to visualise using widgets in a Jupiter notebook) using NWBWidgets.