-
Notifications
You must be signed in to change notification settings - Fork 1
patchmemory/cvtracer
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
############################################################################### # # # This is the cv-tracer software package, designed for tracking and # # analyzing dynamics of fish swimming in groups. It can be downloaded at # # # # https://github.com/patchmemory/cvtracer # # # # Software designed and developed by Adam Patch and Yaouen Fily at Florida # # Atlantic University. Began with github.com/vivekhsridhar/tracktor. # # # ############################################################################### In the cv-tracer package, one will find the modules TrAQ, Analysis, and GUI These three modules provide necessary tools for different stages of tracking birds-eye video of fish swimming in a tank, reviewing the results alongside video to check for errors, and analyzing the results across trials. Scripts for executing all tasks for tracking can be found in scripts/track subdirectory of each base module (e.g. TrAQ/scripts, Analysis/scripts). This document provides a basic overview of how to use this software and is broken into three sections: 1. Tracing and processing tracks 2. Analyzing results across trials 3. Reviewing kinematics alongside video This software is designed in python with free libraries so that it may be straightforwardly modified to incorporate more advanced methods of tracking and analysis. ############################# Tracing and processing tracks ############################# One can completely track and process a video trial, by executing the following commands and providing relevant input parameters. > python prepare.py [input_video] -opts > python trace.py [trial] -opts > python process.py [trial] -opts These each perform the following tasks: prepare.py: Loads video for the first time and takes down information about the run like the raw video file, the number of individuals, the type of fish, the total length of the run, and any other relevant input for characterizing the trial. The script then generates a Trial object to store all of this information. Once the trial object has been created, the script will prompt the user to utilize a GUI to locate the tank edges and store the result as a Tank object inside the trial object. A Group object (which contains Individual objects) within the Trial object will also be generated to store and analyze data at the time of tracing and processing. Finally, the script will generate a directory with a concise name reflecting that of the original video file, moving that file to this new directory as raw.mp4. The Trial object will also be stored in this directory with the name trial.pik trace.py: Loads Trial object and utilizes Tracer and VideoCV objects to track coordinates of fish throughout that trial. At each frame, Tracer will gather and sort the findings of VideoCV in order to best trace the location of fish throughout the run, ignoring fish misidentified outside of the Tank, as identified in prepare.py. The resulting coordinates are stored as Individuals in the Trial's Group. Trial object, now containing Tank, Group, and all relevant trial-specific information, is again saved in the trial-specific location, as found at the beginning of tracing. process.py: After the initial tracking has occurred, it is useful to now better calculate kinematic quantities like velocity and acceleration in both translational and rotational coordinates (latter w.r.t. director angle). Using these quantities, it is then useful to provide some initial statistical analysis without any cuts, for the sake of quickly summarizing the behavior of fish in each run. With further development, this stage would also be the place to fix errors in the initial tracks which may have issues with occlusions and jumping points. At this point in time, it only really makes sense to perhaps identify frames and points of interest for the sake of troubleshooting in the future. For our first paper, these issues are handled by simple ad-hoc cuts which will be made to obviously erraneous data during the next stage, Analysis, so that all Trials will be treated uniformly. ############################### Analyzing results across trials ############################### Once the tracking of a set of trials is complete, one can run the following command and relevant input parameters in order to analyze data given a set of cuts. > python analyze.py [trial_list] -opts While the analysis itself may be modified, this will perform a few basic tasks regardless of the specific analysis. analysis.py: First, all Trial objects will be loaded from their save locations and stored in an Archive object. This object has the ability to call functions on each trial (e.g. make a consistent set of cuts) and to combine statistical results across trials, storing results according to analysis-specific cuts applied. Once results have been computed and collected, they can be combined and stored in the Archive object's result dictionary. This allows easy access via the Archive object later. Utilizing the Analysis.Plot module, one can efficiently plot distributions and results for each trial and across trials. Trial-specific results will be stored in the Trial's home directory while the results across trials will be stored in results/. #################################### Reviewing kinematics alongside video #################################### While one can, in principle, follow all of the above steps, it is always useful to verify kinematic quantities being calculated correctly via some kind of visual sanity check. The easiest way to do this is to utilize the cvDataViewer, found in the GUI module. One can execute this GUI by simply > python GUI/cvDataViewer.py [trial_directory] -v cv.mp4 This will open a window containing a video of the tracked fish as well as a side panel displaying time-dependent results of three quantities: distance to the wall, speed, and angular speed. This tool facilitates checks on tracking throughout a run by allowing users to see directly if quantities are being properly calculated across frames of the video. A couple examples of how one might use this information... One may find that the tracer is having trouble locating fish at every frame, resulting in jumps from track-to-track. By viewing this in the cvDataViewer, one may step back to the tracking stage and try using a different set of contour-tracing parameters to better identify fish and then go back to verify after a second round of tracking has completed. Another may find that while the tracing has been done optimally, there are still undesirable errors where fish appear to be jumping across the tank or turning in an unphysical way. Despite the shortcomings of the tracking software, this provides a basis for eliminating erroneous frames by making cuts at the stage of Analysis, in order to consistently treat systematic errors across runs. ############ Dependencies ############ The following python libraries are required for full functionality of cv-tracer: - argparse - copy - matplotlib - numpy - opencv (cv2, which version?) - pickle - PyQt5 - pyqtgraph - scikit-learn - scipy
About
software for tracking and analyzing behavior of fish in groups
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published