Tracking API implementation details

クラス
class	cv::detail::tracking::TrackerContribFeature
	の抽象基底クラスTrackerContribFeatureの抽象的な基底クラスで、そのフィーチャーを表します。[【詳解】（英語］
class	cv::detail::tracking::TrackerContribFeatureSet
	特徴量の抽出と選択を管理するクラス[【詳解】（英語］
class	cv::detail::tracking::TrackerContribSamplerAlgorithm
	の抽象基底クラスTrackerContribSamplerAlgorithmは、特定のサンプラーのアルゴリズムを表すものです。[【詳解】（英語］
class	cv::detail::tracking::TrackerContribSampler
	トラッカーのモデルを更新するために，リージョンを選択するためのサンプラーを管理するクラス [AAM] Sampling e Labeling. 表IとセクションIII Bを参照[【詳解】（英語］
class	cv::detail::tracking::TrackerStateEstimatorAdaBoosting
	TrackerStateEstimatorAdaBoostingADA-Boostingに基づく[【詳解】（英語］
class	cv::detail::tracking::TrackerStateEstimatorSVM
	TrackerStateEstimatorSVMに基づく[【詳解】（英語］
class	cv::detail::tracking::TrackerContribSamplerCSC
	TrackerSamplerMILアルゴリズムで使用されるCSC（current state centered）に基づくTrackerMIL [【詳解】（英語］
class	cv::detail::tracking::TrackerSamplerCS
	TrackerContribSamplerCS（現在の状態）に基づいて、アルゴリズムTrackerBoostingで使用される[【詳解】（英語］
class	cv::detail::tracking::TrackerSamplerPF
	このサンプラーはパーティクルフィルタリングに基づいています。[【詳解】（英語］
class	cv::detail::tracking::TrackerFeatureFeature2d
	TrackerContribFeatureに基づくFeature2D [【詳解】（英語］
class	cv::detail::tracking::TrackerFeatureHOG
	TrackerContribFeatureHOGに基づく[【詳解】（英語］
class	cv::detail::tracking::TrackerContribFeatureHAAR
	TrackerContribFeatureHAARの特徴に基づいて、以下のように使用されます。TrackerMIL他にも多くのアルゴリズムで使用されています[【詳解】（英語］
class	cv::detail::tracking::TrackerFeatureLBP
	TrackerContribFeatureLBPに基づく[【詳解】（英語］
class	cv::detail::tracking::TrackerFeature
	の抽象基底クラスTrackerFeatureの抽象的な基底クラスで、そのフィーチャーを表します。[【詳解】（英語］
class	cv::detail::tracking::TrackerFeatureSet
	特徴量の抽出と選択を管理するクラス[【詳解】（英語］
class	cv::detail::tracking::TrackerSamplerAlgorithm
	の抽象基底クラスTrackerSamplerAlgorithmは、特定のサンプラーのアルゴリズムを表すものです。[【詳解】（英語］
class	cv::detail::tracking::TrackerSampler
	トラッカーのモデルを更新するために，リージョンを選択するためのサンプラーを管理するクラス [AAM] Sampling e Labeling. 表IとセクションIII Bを参照[【詳解】（英語］
class	cv::detail::tracking::TrackerTargetState
	の抽象基底クラスTrackerTargetStateターゲットの状態の候補を表す[【詳解】（英語］
class	cv::detail::tracking::TrackerStateEstimator
	の抽象基底クラスTrackerStateEstimator最も可能性の高いターゲットの状態を推定する[【詳解】（英語］
class	cv::detail::tracking::TrackerModel
	ターゲットのモデルを表す抽象クラス。[【詳解】（英語］
class	cv::detail::tracking::TrackerSamplerCSC
	TrackerSamplerMILアルゴリズムで使用されるCSC（current state centered）に基づくTrackerMIL [【詳解】（英語］

型定義
typedef std::vector< std::pair< Ptr< TrackerTargetState >, float > >	cv::detail::ConfidenceMap
	フレームでのターゲットのモデルを表す(すべての状態とスコア)[【詳解】（英語］
typedef std::vector< Ptr< TrackerTargetState > >	cv::detail::Trajectory
	すべてのフレームの推定状態を表す[【詳解】（英語］

詳解

Long-term optical tracking API

Long-term optical tracking is an important issue for many computer vision applications in real world scenario. The development in this area is very fragmented and this API is an unique interface useful for plug several algorithms and compare them. This work is partially based on [AAM] and [AMVOT] .

These algorithms start from a bounding box of the target and with their internal representation they avoid the drift during the tracking. These long-term trackers are able to evaluate online the quality of the location of the target in the new frame, without ground truth.

There are three main components: the TrackerContribSampler, the TrackerContribFeatureSet and the TrackerModel. The first component is the object that computes the patches over the frame based on the last target location. The TrackerContribFeatureSet is the class that manages the Features, is possible plug many kind of these (HAAR, HOG, LBP, Feature2D, etc). The last component is the internal representation of the target, it is the appearance model. It stores all state candidates and compute the trajectory (the most likely target states). The class TrackerTargetState represents a possible state of the target. The TrackerContribSampler and the TrackerContribFeatureSet are the visual representation of the target, instead the TrackerModel is the statistical model.

A recent benchmark between these algorithms can be found in [OOT]

Creating Your Own %Tracker

If you want to create a new tracker, here's what you have to do. First, decide on the name of the class for the tracker (to meet the existing style, we suggest something with prefix "tracker", e.g. trackerMIL, trackerBoosting) – we shall refer to this choice as to "classname" in subsequent.

• Declare your tracker in modules/tracking/include/opencv2/tracking/tracker.hpp. Your tracker should inherit from Tracker (please, see the example below). You should declare the specialized Param structure, where you probably will want to put the data, needed to initialize your tracker. You should get something similar to :

  
  class
  CV_EXPORTS_W
  TrackerMIL
  :
  public
  Tracker
  
  {
  
  public:
  
  struct
  CV_EXPORTS Params
  
  {
  
  Params();
  
  //parameters for sampler
  
  
  float
  samplerInitInRadius;
  // radius for gathering positive instances during init
  
  
  int
  samplerInitMaxNegNum;
  // # negative samples to use during init
  
  
  float
  samplerSearchWinSize;
  // size of search window
  
  
  float
  samplerTrackInRadius;
  // radius for gathering positive instances during tracking
  
  
  int
  samplerTrackMaxPosNum;
  // # positive samples to use during tracking
  
  
  int
  samplerTrackMaxNegNum;
  // # negative samples to use during tracking
  
  
  int
  featureSetNumFeatures;
  // #features
  
  
  
  
  void
  read(
  const
  FileNode& fn );
  
  void
  write(
  FileStorage& fs )
  const;
  
  };

  of course, you can also add any additional methods of your choice. It should be pointed out, however, that it is not expected to have a constructor declared, as creation should be done via the corresponding create() method.
• Finally, you should implement the function with signature :

  
  Ptr<classname>
  classname::create(const
  classname::Params &parameters){
  
  ...
  }

  That function can (and probably will) return a pointer to some derived class of "classname", which will probably have a real constructor.

Every tracker has three component TrackerContribSampler, TrackerContribFeatureSet and TrackerModel. The first two are instantiated from Tracker base class, instead the last component is abstract, so you must implement your TrackerModel.

TrackerContribSampler

TrackerContribSampler is already instantiated, but you should define the sampling algorithm and add the classes (or single class) to TrackerContribSampler. You can choose one of the ready implementation as TrackerContribSamplerCSC or you can implement your sampling method, in this case the class must inherit TrackerContribSamplerAlgorithm. Fill the samplingImpl method that writes the result in "sample" output argument.

Example of creating specialized TrackerContribSamplerAlgorithm TrackerContribSamplerCSC : :

class
CV_EXPORTS_W
TrackerContribSamplerCSC
:
public
TrackerContribSamplerAlgorithm

{

public:

TrackerContribSamplerCSC(
const
TrackerContribSamplerCSC::Params
&parameters =
TrackerContribSamplerCSC::Params() );

~TrackerContribSamplerCSC();

...



protected:

bool
samplingImpl(
const
Mat& image,
Rect
boundingBox, std::vector<Mat>& sample );

...


};

Example of adding TrackerContribSamplerAlgorithm to TrackerContribSampler : :

//sampler is the TrackerContribSampler


Ptr<TrackerContribSamplerAlgorithm>
CSCSampler =
new
TrackerContribSamplerCSC( CSCparameters );

if( !sampler->addTrackerSamplerAlgorithm( CSCSampler ) )

return
false;



//or add CSC sampler with default parameters


//sampler->addTrackerSamplerAlgorithm( "CSC" );

参照

TrackerContribSamplerCSC, TrackerContribSamplerAlgorithm

TrackerContribFeatureSet

TrackerContribFeatureSet is already instantiated (as first) , but you should define what kinds of features you'll use in your tracker. You can use multiple feature types, so you can add a ready implementation as TrackerContribFeatureHAAR in your TrackerContribFeatureSet or develop your own implementation. In this case, in the computeImpl method put the code that extract the features and in the selection method optionally put the code for the refinement and selection of the features.

Example of creating specialized TrackerFeature TrackerContribFeatureHAAR : :

class
CV_EXPORTS_W
TrackerContribFeatureHAAR
:
public
TrackerFeature

{

public:

TrackerContribFeatureHAAR(
const
TrackerContribFeatureHAAR::Params
&parameters =
TrackerContribFeatureHAAR::Params() );

~TrackerContribFeatureHAAR();

void
selection(
Mat& response,
int
npoints );

...



protected:

bool
computeImpl(
const
std::vector<Mat>& images,
Mat& response );

...


};

Example of adding TrackerFeature to TrackerContribFeatureSet : :

//featureSet is the TrackerContribFeatureSet


Ptr<TrackerFeature>
trackerFeature =
new
TrackerContribFeatureHAAR( HAARparameters );
featureSet->addTrackerFeature( trackerFeature );

参照

TrackerContribFeatureHAAR, TrackerContribFeatureSet

TrackerModel

TrackerModel is abstract, so in your implementation you must develop your TrackerModel that inherit from TrackerModel. Fill the method for the estimation of the state "modelEstimationImpl", that estimates the most likely target location, see [AAM] table I (ME) for further information. Fill "modelUpdateImpl" in order to update the model, see [AAM] table I (MU). In this class you can use the :cConfidenceMap and :cTrajectory to storing the model. The first represents the model on the all possible candidate states and the second represents the list of all estimated states.

Example of creating specialized TrackerModel TrackerMILModel : :

class
TrackerMILModel :
public
TrackerModel

{

public:

TrackerMILModel(
const
Rect& boundingBox );

~TrackerMILModel();

...



protected:

void
modelEstimationImpl(
const
std::vector<Mat>& responses );

void
modelUpdateImpl();

...


};

And add it in your Tracker : :

bool
TrackerMIL::initImpl(
const
Mat& image,
const
Rect2d& boundingBox )
{

...

//model is the general TrackerModel field of the general Tracker


model =
new
TrackerMILModel( boundingBox );

...
}

In the last step you should define the TrackerStateEstimator based on your implementation or you can use one of ready class as TrackerStateEstimatorMILBoosting. It represent the statistical part of the model that estimates the most likely target state.

Example of creating specialized TrackerStateEstimator TrackerStateEstimatorMILBoosting : :

class
CV_EXPORTS_W TrackerStateEstimatorMILBoosting :
public
TrackerStateEstimator

{

class
TrackerMILTargetState :
public
TrackerTargetState


{

...

};



public:

TrackerStateEstimatorMILBoosting(
int
nFeatures = 250 );

~TrackerStateEstimatorMILBoosting();

...



protected:

Ptr<TrackerTargetState>
estimateImpl(
const
std::vector<ConfidenceMap>& confidenceMaps );

void
updateImpl( std::vector<ConfidenceMap>& confidenceMaps );

...


};

And add it in your TrackerModel : :

//model is the TrackerModel of your Tracker


Ptr<TrackerStateEstimatorMILBoosting>
stateEstimator =
new
TrackerStateEstimatorMILBoosting( params.featureSetNumFeatures );
model->setTrackerStateEstimator( stateEstimator );

参照

TrackerModel, TrackerStateEstimatorMILBoosting, TrackerTargetState

During this step, you should define your TrackerTargetState based on your implementation. TrackerTargetState base class has only the bounding box (upper-left position, width and height), you can enrich it adding scale factor, target rotation, etc.

Example of creating specialized TrackerTargetState TrackerMILTargetState : :

class
TrackerMILTargetState :
public
TrackerTargetState

{

public:

TrackerMILTargetState(
const
Point2f& position,
int
targetWidth,
int
targetHeight,
bool
foreground,
const
Mat& features );

~TrackerMILTargetState();

...



private:

bool
isTarget;

Mat
targetFeatures;

...


};

型定義詳解

ConfidenceMap

typedef std::vector<std::pair<Ptr<TrackerTargetState>, float> > cv::detail::tracking::ConfidenceMap

フレームでのターゲットのモデルを表す(すべての状態とスコア)

参照[AAM].ペアのセット

参照

TrackerTargetState

Trajectory

typedef std::vector<Ptr<TrackerTargetState> > cv::detail::tracking::Trajectory

すべてのフレームの推定状態を表す

[AAM]. は時間までのターゲットの軌跡を表す

参照

TrackerTargetState