vi3o - VIdeo and Image IO

Utility for loading/saving/displaying video and images. It gives random access to the video frames. For recordings origination from Axis cameras in mjpg (in http multipart format) or H264 (in .mkv format) the camera system time at the time of capture is provided as a system timestamp for each frame.

To get system timestamps in H.264 recordings “User data” has to be enabled. It is found by clicking “Setup”, “System Options”, “Advanced”, “Plain Config” and choosing “Image” followed by “Select group”.

Status

Work in progress…

Installation

First, install some dependencies:

sudo apt-get install libjpeg62-turbo-dev libavcodec-dev libswscale-dev libffi-dev

Then there are a few different ways to install vi3o:

  • Use pip to install vi3o with the minimal set of dependencies:

    pip install vi3o
    
  • or install vi3o with all necessary dependencies:

    pip install "vi3o[full]"
    

    The necessary additional dependencies for full functionality are:

    • pillow < 7.0
    • pyglet < 1.5
  • or get the source code via the Python Package Index.

  • or get it from Github:

    git clone https://github.com/hakanardo/vi3o.git
    cd vi3o
    python setup.py install
    

Whats new

v0.10.0

  • Allow the convert option of vi3o.image.imread() to work with truncated images
  • Expose Axis camera MAC in vi3o.mkv.Mkv video
  • Safer handling of cache files when multiple processes simultaniously opens the same vi3o.mkv.Mkv video
  • Add vi3o.debugview.DebugView.window_to_image_coord() on vi3o.debugview.DebugView which can be usefull when overriding vi3o.debugview.DebugView.on_mouse_motion() in subclasses.

v0.9.3

v0.9.2

  • Added vi3o.utils.VideoFilter for transforming frames on load
  • Added convert options to vi3o.image.imread()

v0.9.1

  • Make the imageio backend fps default to 25 if not specified
  • Fix pillow > 6.1.1 compatibility bug with bad img
  • Repair tests for python 3-3.5
  • Repair AxisCam (imwrite fp interface)
  • Fixes race in index_file creation

v0.9.0

  • Added vi3o.recording.Recording for reading AXIS NAS/SD-card recordings.
  • Fixed segfault when requesting grey encoding.
  • Fixed library compatibility with pathlib.
  • Fixed compatibility for Pyglet 1.4.
  • Fixed pinning of dependencies to Python 2.7 compatible versions.
  • Fixed segfault when requesting gray video.
  • Fixed timestamps property of frames from vi3o.cat.VideoCat to be relative to first frame in first video.

v0.8.0

  • Silence some compiler warnings
  • Added reindex parameter to vi3o.mkv.Mkv
  • Support for sliced views of vi3o.opencv.CvVideo
  • Support for mkv files with both keyframe and user data frame in the same matrosca frame
  • Support for negative timecodes in mkv streams
  • Use imageio as default fallback video decoder instead as opencv is missbehaving
  • Silence deprication warning
  • Removed deprecated pixel format warning on Ubuntu Xenial
  • Bumped index version number to eusure indexes are recreated with above bugfixes

v0.7.4

  • Remove numpy as dependency to setup.py
  • Fixed dependencies for setuptools test command
  • Removed deprecated pixel format warning
  • Added script to run unit tests localy in docker

v0.7.3

  • Support for random access in mjpeg coded .mkv files.

v0.7.2

  • Support for .mkv files with mjpg codec. Timestamps are in this case red from jpg-headers so there is no need for user-data packages in the mkv stream.
  • Passing grey=True to vi3o.mjpg.Mjpg no longer results in segfault.

v0.7.0

v0.6.1

  • Setup now depends on numpy
  • No longer depends on cv2.cv.FOURCC, which has been droppen in recent opencv version
  • Recognize upper case filename extntions
  • Select betwen av_frame_alloc and avcodec_alloc_frame based on libav version

v0.6.0

v0.5.2

  • Slightly lighter background color behind images in DebugView to distinguish black backgrounds from outside image.
  • Support for reading system timestamps from more recnt Axis cameras.
  • Added a OpenCV fallback to allow unknown video formats to handled as Video object even if there are no system timestamps.
  • Fixed a segfault when parsing broken or truncated mkv files.

v0.5.0

  • Added vi3o.netcam.AxisCam for reading video directly from Axis camera
  • Add systimes property to Mkv and SyncedVideos to get a list of all system timestamps without decoding the frames.
  • Switch to setuptools for proper handing of cffi dependency
  • Remove numpy dependency during setup
  • Dont try to decode truncated mkv frames

v0.4.0

  • Allow negative indexes to wrap around in Video objects.
  • Added vi3o.SyncedVideos for syncornizing videos using systime.
  • Support for showing images of different size side by side in the debug viewer.
  • Support for showing images of different size one after the other in the debug viewer.
  • Move the generated .idx files to the user .cache dir
  • Regenerate the .idx files if the video is modfied
  • Added vi3o.image.imrotate().
  • Added vi3o.image.imshow().
  • Added support for greyscale mjpg files.

Overview

Video recordings are are handled using Video objects, which are sliceable to provide video object representing only part of the entire file,

from vi3o import Video

recoding = Video("myfile.mkv")
monochrome = Video("myfile.mkv", grey=True)

first_part = recoding[:250]
last_part = recoding[-250:]
half_frame_rate = recoding[::2]
backwards = recoding[::-1]

The video object can be used to iterate over the frames in the video,

for frame in recoding:
    ...

It also supoprts random access to any frame,

first_frame = recoding[0]
second_frame = recoding[1]
last_frame = recoding[-1]

The frame objects returned are numpy ndarray subclasses with a few extra properties:

  • frame.index - The index of the frame within in the video (i.e video[frame.index] == frame)
  • frame.timestamp - The timestamp of the frame as a float in seconds
  • frame.systime - The system timestamp specifying when the frame was aquired (a float of seconds elapsed since the Epoch, 1970-01-01 00:00:00 +0000).

The video frames can be displayed using the debug viewer,

from vi3o import Video, view

for img in Video("myfile.mkv"):
    view(img)

This opens a window showing the video which can be controlled using:

  • Space - pauses and unpases
  • Enter - step forward a single frame if paused
  • Mouse wheel - zoomes in/out the video
  • Click and drag - pan around in the video
  • z - zoomes video to fit the window
  • f - toggles fullscreen mode
  • d - starts pdb debugger
  • s - Toggle enforced rescaling of all images into the 0..255 range

To show multiple images side by side in the window, call vi3o.flipp() to start colled images and then once more to show the collected images and restart the collecting:

from vi3o import Video, view, flipp

for img in Video("myfile.mkv"):
    flipp()

    brighter = 2 * img
    darker = 0.5 * img

    view(img)
    view(brighter)
    view(darker)

It is also possible to list all timestamps in a video without decoding the image data using:

from vi3o import Video

Video("myfile.mkv").systimes

Modules

vi3o — VIdeo and Image IO

class vi3o.SyncedVideos(*filenames_or_videos)[source]

Synchronize a set of videos using the systime timestamps. Frames will be dropped to adjust the frame rate to match the video with the lowest frame rate. Initial and trailing parts of the videos where there are not frames from all vidos will be dropped. To for example play 3 videos syncronized side by side, use:

from vi3o import SyncedVideos, view, flipp

for a, b, c in SyncedVideos('a.mkv', 'b.mkv', 'c.mkv'):
    flipp()
    view(a)
    view(b)
    view(c)

It is also possible to access random frames or slices:

from vi3o import SyncedVideos

recoding = SyncedVideos('a.mkv', 'b.mkv', 'c.mkv'):
first_part = recoding[:250]
last_part = recoding[-250:]
half_frame_rate = recoding[::2]
backwards = recoding[::-1]

The input argument filenames_or_videos is a list of either file names or Video objects.

indexes

Retunrs a list of frame indexes of the frames used in the synced stream.

systimes

Retunrs a list of systime timestamps without decoding any pixel data.

vi3o.Video(filename, grey=False)[source]

Creates a Video object representing the video in the file filename. See Overview above.

class vi3o.VideoCat(videos, **kwargs)[source]

Concatenates multiple video files into a single video object. The videos parameter is a list of videos to be concatenated. It can either be a list of filenames or a list of other Video objects. Typical usage:

from vi3o import VideoCat

for img in VideoCat(['part1.mkv', 'part2.mkv']):
    ...
videos

Compatibility method

class vi3o.VideoGlob(pathname)[source]

Subclass of VideoCat that is initiated with a glob.glob() wildcard string instead of a list of videos. The wildcard is expanded into a list of filenames that is the sorted before concatenated.

vi3o.flipp(name='Default', pause=None, aspect_ratio=None)[source]

After vi3o.flipp() is called, subsequent calls to vi3o.view() will no longer display the images directly. Instead they will be collected and concatinated. On the next call to vi3o.flipp() all the collected images will be displayed. If pause is set to True/False, the viewer is paused/unpaused after the image is displayed. If aspect_ratio is set the images will be stacked in such a way that the total aspect ratio is close to aspect_ratio.

vi3o.view(img, name='Default', scale=False, pause=None)[source]

Show the image img (a numpy array) in the debug viewer window named name. If scale is true the image intensities are rescaled to cover the 0..255 range. If pause is set to True/False, the viewer is paused/unpaused after the image is displayed.

vi3o.viewsc(img, name='Default', pause=None)[source]

Calls vi3o.view() with scale=True.

vi3o.mjpg — Motion JPEG video loading

class vi3o.mjpg.Mjpg(filename, grey=False)[source]

If a filename that ends with .mjpg is passed to vi3o.Video() this kind of object is returned. It has a few additional format specific properties:

firmware_version

The firmware version running in the camera when it made this recording.

hwid

The Axis hardware id of the camera that made this recording.

serial_number

The Axis serial number or mac address of the camera that made this recording.

vi3o.mjpg.jpg_info(filename)[source]

Reads a single jpeg image from the file filename and extracts the Axis user data header. The information it contains is returned as a dict with the keys “hwid”, “serial_numer” and “firmware_version”.

Interprets the storage format for recordings in AXIS video storage format, e.g. video stored on a NAS or SD card.

class vi3o.recording.Recording(metadata, **kwargs)[source]

Load video from a folder containing a Axis recording from e.g. a SD card or a NAS

Axis stores videos in a blocked format together with XML metadata. The XML metadata contains time information which may be used to deduce the wall time of the recordings should the video streams themselves lack the proper metadata.

Either use the convenience method vi3o.Video

import vi3o
recording = vi3o.Video("/path/to/recording.xml")

Or load metadata manually using read_recording_xml

import vi3o
metadata = vi3o.read_recording_xml("/path/to/recording.xml")
recording = Recording(metadata)
systimes

Return the systimes of all frames in recording

class vi3o.recording.RecordingBlock(start, stop, filename, status)
filename

Alias for field number 2

start

Alias for field number 0

status

Alias for field number 3

stop

Alias for field number 1

class vi3o.recording.RecordingMetadata(recording_id, channel, start, stop, blocks, width, height, framerate)
blocks

Alias for field number 4

channel

Alias for field number 1

framerate

Alias for field number 7

height

Alias for field number 6

recording_id

Alias for field number 0

start

Alias for field number 2

stop

Alias for field number 3

width

Alias for field number 5

vi3o.recording.read_recording_xml(recording_xml)[source]

Read metadata from an Axis recording from e.g. a SD card or a NAS

import vi3o

metadata = vi3o.read_recording_xml("/path/to/recording.xml")
print("Width: {}, Height: {}".format(metadata.width, metadata.height))
print("Number of blocks: {}".format(len(metadata.blocks)))

vi3o.image — Image handling

class vi3o.image.ImageDirOut(dirname, format='jpg', append=False)[source]

Creates a directory called dirname for storing a sequence of images in the format specified by format. If append is not True (default) the content of the directory will be cleard if it excists.

view(img)[source]

Save the image img as dirname/xxxxxxxx.format where xxxxxxxx is an 8 digit sequence number.

viewsc(img)[source]

Rescales the intensities of the image img to cover the 0..255 range, and call view() to save it.

vi3o.image.imload(filename, repair=False, convert=None)

Load an image from the file filename. If repair is True, attempts will be made to decode broken frames, in which case partially decoded frames might be returned. A warning is printed to standard output unless repair is set to vi3o.image.Silent. To convert the image to a specific colorspace, set convert to the deciered pillow colormode.

vi3o.image.imread(filename, repair=False, convert=None)[source]

Load an image from the file filename. If repair is True, attempts will be made to decode broken frames, in which case partially decoded frames might be returned. A warning is printed to standard output unless repair is set to vi3o.image.Silent. To convert the image to a specific colorspace, set convert to the deciered pillow colormode.

vi3o.image.imrotate_and_scale(img, angle, scale, center=None, size=None, interpolation=0, point=None)[source]

Rotate the image, img, angle radians around the point center which defaults to the center of the image. The output image size is specified in size as (width, height). If point is specifed as (x, y) it will be taken as a coordinate in the original image and be transformed into the corresponing coordinate in te output image and returned instead of the image. Integer coordinates are considered to be at the centers of the pixels.

vi3o.image.imsave(img, filename, format=None)[source]

Save the image img into a file named filename. If the fileformat is not specified in format, the filename extension will be used as format.

vi3o.image.imsavesc(img, filename, format=None)[source]

Rescales the intensities of the image img to cover the 0..255 range and then calls vi3o.image.imsave() to save it.

vi3o.image.imscale(img, size, interpolation=0)[source]

Scales the image img into a new size specified by size. It can either be a number specifying a factor that relates the old size of the new or a 2-tuple with new size as (width, height).

vi3o.image.imshow(img)[source]

Display the image img in the DebugViewer and pause the viewer with the image showing.

vi3o.image.imshowsc(img)[source]

Rescales (and translates) the intensities of the image img to cover the 0..255 range. Then display the image img in the DebugViewer and pause the viewer with the image showing.

vi3o.image.imview(img)

Display the image img in the DebugViewer and pause the viewer with the image showing.

vi3o.image.imviewsc(img)

Rescales (and translates) the intensities of the image img to cover the 0..255 range. Then display the image img in the DebugViewer and pause the viewer with the image showing.

vi3o.image.imwrite(img, filename, format=None)

Save the image img into a file named filename. If the fileformat is not specified in format, the filename extension will be used as format.

vi3o.image.ptpscale(img)[source]

Rescales (and translates) the intensities of the image img to cover the 0..255 range.

vi3o.netcam — Live camera handling

class vi3o.netcam.AxisCam(ip, width=None, height=None, username=None, password=None, no_proxy=False, **kwargs)[source]

Loads an mjpg stream directly from an Axis camera with hostname ip with the resolution width*x*height using the username and password as credntials. If no_proxy is True, the proxy settings from the environment will be ignored and any other keyword parameter will be passed on to the camera as a VAPIX parameter.

Comments and bugs

There is a mailing list for general discussions and an issue tracker for reporting bugs and a continuous integration service that’s running tests.