Friday, September 20, 2019

Creating and consuming tensorflow record files

TFRecord files can be confusing. They are the preferred data containers for training tensorflow models when using the object detection api (github).

It took me a while to converge on code I like for generating TFRecord files (including hard negative examples, with no bounding boxes), and for consuming TFRecord files to display their contents. The latter is especially important, as when you roll your own augmentation pipeline it is really helpful to look at the data that your are using to train your network, just to be sure everything looks reasonable.

I've encapsulated my experience with this in a github repo, tfrecord-view, which has a script for encoding data into a TFRecord file (given images and VOC-Pascal encoded xml annotation files), and a script for consuming a TFRecord file:

https://github.com/EricThomson/tfrecord-view

It includes data and annotation files so you can test it out easily.

2 comments:

STUDY IN RUSSIA said...

thats sounds nice I used your code to create tfrecord to train an object detection.
do you think this could help in reducing false postives ?
i used to make other class for object detection to be unknown but I don't like it since there are many objects to consider in one class, also no pattern in this class. i hope the negtive example method works

Eric Thomson said...

Hi STUDY -- I found it essential to include hard negatives to avoid false negatives in my object detection pipeline (using faster-rcnn).