Complexity estimation of image sequence for automatic target track
WANG Xiaotian*,ZHANG Kai,and YAN Jie
School of Astronautics,Northwestern Polytechnical University,Xi’an 710072,China
Abstract: In the field of automatic target recognition and tracking,traditional image complexity metrics,such as statistical variance and signal-to-noise ratio,all focus on single-frame images.However,there are few researches about the complexity of image sequence.To solve this problem,a criterion of evaluating image sequence complexity is proposed.Firstly,to characterize this criterion quantitatively,two metrics for measuring the complexity of image sequence,namely feature space similarity degree of global background(FSSDGB)and feature space occultation degree of local background(FSODLB)are developed.Here,FSSDGB reflects the ability of global background to introduce false alarms based on feature space,and FSODLB represents the difference between target and local background based on feature space.Secondly,the feature space is optimized by the grey relational method and relevant features are removed so that FSSDGB and FSODLB are more reasonable to establish complexity of single-frame images.Finally,the image sequence complexity is not a linear sum of the single-frame image complexity.Target tracking errors often occur in high-complexity images and the tracking effect of low-complexity images is very well.The nonlinear transformation based on median(NTM)is proposed to construct complexity of image sequence.The experimental results show that the proposed metric is more valid than other metrics,such as sequence correlation(SC)and interframe change degree(IFCD),and it is highly relevant to the actual performance of automatic target tracking algorithms.
Keywords: complexity of image sequence,feature space similarity degree of global background(FSSDGB),feature space occultation degree of local background(FSODLB),grey relational method,nonlinear transformation based on median(NTM).
1.Introduction
In military applications,the automatic target recognition and tracking technology is one of the most popular research topics around the world.In theoretical research,many latest theories have been applied to the design of the algorithm,e.g.,Wavelet transform,template matching,Kalman filter,particle filter.The bottleneck of algorithm innovation is no longer due to the lack of theoretical support,but how to evaluate the performance of the algorithm[1].
Nowadays,the use of image databases is the most common tool to evaluate the performance of automatic target recognition and tracking algorithms.However,scholars use different image databases to test different algorithms.Therefore,it is extremely difficult to compare the performance of one algorithm to the others due to the fact that the testing results for one algorithm might be only good for a certain image database and poor for the others.Moreover,in most cases,algorithm testing is usually performed over a limited set of scenarios and the image databases generally only include a small set of collected images or artificially generated images[2-6].One way to solve this issue is to correlate the image characteristics with the algorithm performance and measure the complexity of images by using quantitative metrics.
In general,scholars usually define complexity as a measure of the difficulty of an object performing its most important task[3-7].Thus,in the context of automatic target recognition and tracking performance,the image complexity should be a description of the inherent difficulty of fulfilling a target recognition and tracking task for given images.The image complexity metrics can depend on a priori information about the actual target in an image to be characterized.The complexity estimation methods for automatic target track are mainly divided into three classes:(i)global image statistical metrics-derived from the set of all pixels in the image,such as image gray-level standard deviation,image gray-level entropy,gray-level uniformity[8],etc.;(ii)local background statistical metrics[9]-derived from the set of pixels in the local background,such as target to background contrast[10],target interference ratio[7],Kolmogorov-Smirnov statistic[11],target local background entropy difference[11],and target local background gray correlation[12];(iii)target dependent metrics-derived from the set of target features[11],such as target entropy,target standard deviation,target average contour length,pixels on target,target length to width,target co-occurrence matrix[13],and so on.With the deeper comprehension for image complexity,the complexity is measured by two specific indicators:similarity degree of global background-the ability of background to introduce false alarms,and occultation degree of local background-the ability of recognition system to detect the target from the local background[14].In addition,Harper et al.[15]examined the relationship between complexity and various aspects of presentation.Corchs et al.[16]proposed a new complexity measure where different image features,based on spatial,frequency and color properties were linearly combined.Zhou et al.[17]evaluated image complexity by using color feature as well as other spatial features.Ciocca et al.[18]investigated the relationship between image complexity and ten features,such as the gray-level entropy and the target co-occurrence matrix.These metrics are based on various theories,but most of them apply to single-frame images.
Since the characteristics of a single-frame image and the characteristics of an image sequence are different,it is not surprising that the complexity metrics for the single-frame image are not best suited for measuring image sequence.So far,the research on complexity of the image sequence is still in infancy.The representative metrics are the degree of sequence correlation[19]and the inter-frame change degree of target[20,21],but they all have their own inherent flaws.The degree of sequence correlation simply calculates the correlation coefficient of adjacent frames in the image sequence and does not consider the information related to the target.The inter-frame change degree of target has achieved a certain effect,but it only considers the influence of inter-frame target feature change and ignores the influence of intra-frame characteristics.
As mentioned above,we propose a criterion of evaluating image sequence complexity which incorporates the influence of inter-frame target feature change and influence of intra-frame characteristics comprehensively.Firstly,feature space similarity degree of global background(FSSDGB)and feature space occultation degree of local background(FSODLB)are developed to measure the complexity of the image.Secondly,the feature space is optimized by the gray relational method.Finally,a nonlinear transformation based on median is proposed to construct complexity of the image sequence.
The structure of this paper is as follows.Section 1 discusses traditional image metrics.Section 2 defines image complexity and analyses the deficiency of existing metrics for measuring the image sequence complexity.Based on this analysis,new metrics that are more suitable for measuring the image sequence are proposed in Section 3.The proposed metrics are validated by experiments in Section 4.It should be noted that it is the first time that an image database of more than 10 000 image sequences is used for experimental purposes.Section 5 draws the conclusions of this paper.
2.Defniition and drawbacks of the existing image complexity
Step 5 Calculate the values of eight connected domains when Gi＞0.5.
The image complexity metrics that can be found in the current literatures all focus on the single-frame image and do not take into account the characteristics of the image sequence.There are certain drawbacks of traditional metrics for measuring image sequence,for example,
(i)Traditional metrics mainly describe the information of target,background,noise,or the relationship among them in a single-frame image.However,they do not consider the sequence information between different image frames and this will inevitably result in loss of information.
where E(T)and E(B)are the target area entropy and the local background area entropy respectively.
Based on all of the above mentioned,it can be concluded that the current traditional metrics have obvious drawbacks in terms of the measure of the image sequence.The representative complexity metrics of the image sequence,such as the degree of sequence correlation and the inter-frame change degree of target,are either less effective or less thoughtful of intra-frame characteristics.Therefore,it is necessary to propose new metrics suitable for the image sequence that takes into account the influence of inter-frame target feature change and influence of intra-frame characteristics comprehensively.
3.Proposed complexity metric of the image sequence
3.1 Existing image complexity metrics
The complexity of the image sequence is a measure of the inherent difficulty of target recognition and tracking in given image sequences.To design the complexity metric of the image sequence,we need to know how to characterize the image sequence,so that we can relate these characteristics to the performance of the target recognition and tracking algorithm.So far,many complexity metrics have been proposed[22-35].This paper classifies current metrics into three categories based on the scope.The first one is global image statistical metrics,the second one is local background statistical metrics,and the third one is target dependent metrics.
Global image statistical metrics are functions of all the pixels in an image.They give no concern about priori information on the actual target,while the target priori information plays an important role for performance of target recognition and tracking.Therefore,global image statistical metrics have poor robustness and are rarely used in practical application.
The local background statistical metrics are derived from the set of pixels in the local background.The local background region B is a rectangular annulus whose inner boarder coincides with the target rectangle T and whose outside rectangle area is twice the area of the target rectangle,which is shown in Fig.1.
Fig.1 Local background region
It seems reasonable to presume that if a target contrasts highly with its local background or the difference between the distributions of the target area and the local background areas is large,the target will be easier to find.Consequently,a simple measure of image complexity is the contrast between a target and its local background(TBC)[10]:
where μT is the average gray-level of the pixels in the target and μB is the average gray-level of the pixels in the local background.
Lahart et al.[7]described image complexity with the target interference ratio(TIR)based on the above assumption:
where σT and σB are the standard deviations of the target and the local background.
Target-to-background entropy difference(ETB)[11]represents the difference of the target area and the local background in gray-level distribution.If the entropy difference is small,then it is harder to extract the target from the background:
（7）神经网络法，神经网络是通过数据训练、检验、预测为基础的数学模型，具有较强的可行性和可信度，以神经网络法为基础衍生的系列改进神经网络方都可以结合具体工程项目进行选择。
(ii)The aim of traditional metrics is to measure the degree of difficulty of target detection or recognition in images,while image sequence complexity also needs to consider the degree of difficulty of target tracking.
In addition,Beard et al.[11]suggested the use of a Kolmogorov-Smirnov statistic(KSZ),since it is a widelyused measure of the difference between two distributions:
where pi and qi are respectively target area gray-level probability distribution and local background area gray-level probability distribution.
Target local background gray-level distribution correlation(GDC)also represents the difference of the target area and the local background area:
Most image complexity metrics are target-dependent.They require explicit information on the true target in the image,such as the gray-level statistical,location,edge,texture,size,shape,etc.Target standard deviation(TSD)[3,24]is used to compute the standard deviation of the pixel intensity values in the target region and can be denoted by σT.Target entropy(ENT)[11],being similar with TSD,is the entropy of the pixel intensity values in the target.
在大数据技术的整个运行周期种，首要的环节就是数据采集，要依据Map Reduce产生的数据进行有效分类，先分布式局部计算得出结果再统一合并得到计算结果。
An image sequence metric that focuses on measuring the change degree of the target size is developed,and it is calculated as follows:
where N is the frame number of images in the given image sequence,and li and wi are the length and width of the smallest rectangle containing a target in the ith image frame,respectively.
Beard et al.[11]defined a statistic called the average contour length(ACL)which is a measure of the connectivity of edge pixels.The ACL is a measure of edge information in known targets.
Diao et al.[21]proposed an image sequence metric for describing the change degree of target location(IFCDTL).
1.盈利水平与研发支出资本化强度呈负相关关系。企业盈利水平越高，为减少可能承担的社会责任，往往不会将研发支出资本化；而那些盈利水平较低的企业，为了提高企业价值，吸引更多投资，则更趋向于将研发费用资本化。
Based on the IFCDTL,Zheng[25]proposed the interframe change degree of target movement information(IFCDTML),which took the direction information of the target into account,and he proved that the inter-frame change degree of target gray information is also an important metric.
大数据的作用是可以在较短的时间内对所收集的信息进行统计，从而在进行相关研究时得出快速的结论。同时，收集的信息和结果可以应用于多个领域或方面，可以快速实现资源共享，以前这是不可能的。在大学思想政治教育的基础上进行大数据创新是一个好主意。通过研究所收集的信息，并根据不同的结论提出相应的对策，该方案可以很好地实施。同时，通过思想政治教育班级教师，辅导员等组织的时事性，专题网络问卷调查，网络学习和面向沟通的数据采集，实现学生思维行为，数字化表达，两种方式可以有效地实现思想政治教育信息数据库平台的建设。
With deeper comprehension for image complexity,the complexity is measured by two specific indicators:similarity degree of global background-the ability of background to introduce false alarms,and occultation degree of local background-the ability of recognition system to detect the target from the local background.
Chang and Zhang[30,31]developed a metric that is suitable for electro-optical images and named it target structure similarity(TSSIM).This measure estimates the similarity degree of clutter in an image by a simple comparison of luminance,contrast and structure between the target and the background areas.
应用科学、有效的血栓风险评估工具，对下肢深静脉血栓形成的风险进行准确的评估，是护理人员筛选高危人群、实施预防的先决条件。
每上完一个新的内容, 就要及时完成配套的练习, 千万不能拖.课后及时做作业既是对已学知识的巩固, 又是为下一节课所学知识做好铺垫, 如果不能及时把作业做完, 就有可能影响下一节新课的理解和接受, 造成学习困难.再者, 练习也是复习的一种方法, 通过练习来温习所学的新内容, 使自己温故而知新.
Aviram and Rotman[32-34]developed a new texture metric which is called the improved co-occurrence matrix(ICOM).This metric incorporates the attributes of global texture matching and of local texture distinctness by analyzing the texture differences between the target and suspected target areas.
Some target dependent metrics mainly consider the texture of the target.One of the best ways to measure the texture of a true target can be achieved by calculating its cooccurrence matrix(COM)[26-28]which contains information about the intensity-value distribution and the possible transitions among neighbor pixels.The simplest size measure is pixels on target(POT)[29],which could be obtained by counting the number of pixels in a target region.The simplest shape measure is the ratio of target length to width(LTW)[11].
In addition,Diao and Mao proposed two image metrics in the perspective of quantifying the difficulty degree of infrared small target detection[35].One of the metrics is the degree of target being confused(DTC),which quantifies the odds of infrared small target images providing fake targets.The other one is the degree of target being shielded(DTS),which reflects the contribution of the image to shield the target.
In the next section,we review these commonly used image and image sequence metrics.We will integrate these metrics and transform them by constructing feature space so that they will obtain complementary advantages.
3.2 Feature space
The current image complexity metrics are mainly divided into three classes by scholars.Nevertheless,with deeper research and comprehension in the field of image complexity,two specific indicators for measuring complexity are formed:similarity degree of the global background and occultation degree of the local background.Thus,when constructing complexity of the image sequence,one can refer to the experience from previous achievements.The global image statistical metrics have poor robustness and are rarely used in practical application due to the negligence of target priori information.We will use local background statistical metrics and target dependent metrics to build FSSDGB and FSODLB.FSSDGB reflects the ability of the global background to introduce false alarms.We need to compare the features of the true target and false alarms.And the FSSDGB will be built with target dependent metrics.It is worth noting that IFCDTL and interframe change degree of target size(IFCDTS)are replaced by IFCDTML and inter-frame change degree of target gray information(IFCDTGL)considering the latter combination gives better results.Also,TSSIM,DTC,and ICOM are abandoned because they all reflect the ability to detect false alarms in a global background,which overlaps with the functions of FSSDGB.
Fig.2 Feature space for complexity of the image sequence
Thus they are not included in the feature space.Feature space of FSSDGB includes TSD,ENT,ACL,IFCDTML,IFCDTGL,COM,POT,and LTW.The eight types of target dependent metrics need to be comprehensively considered to build FSSDGB.Feature space of FSODLB includes TBC,TIR,ETB,KSZ and GDC.Feature space for complexity of the image sequence is shown in Fig.2.
3.3 Grey relational method
The basic principle of the grey relational method is to determine whether different elements are relevant based on similarity of curve change trend[36].The more similar the curves are,the greater the correlation between the elements is,and vice versa.The grey correlation matrix between different elements could be calculated by the grey relational method shown in Algorithm 1.
Algorithm 1 The grey correlation method
Input: feature data
n:frame number of data sequence,
m:number of features.
Output: grey correlation degree:0-1.
Step 1 All the data sequence should be normalized:
由式(7)可以看出,只需求出dAB(t2n-1,t2n-1)-dAB(t2n-3,t2n-3)的值,再通过线性回归,即可弱化εmotion偏差求得频偏θ.
Step 2 Calculate grey correlation coefficient:
where εij(k)is the correlation coefficient between
and
at point k.
Let N denote the number of images in a given image sequence,let vector di be the target displacement from the frame i-1 to the frame i,and let Δi denote the size of the target in the ith image frame.The target size represents the mean area of the smallest rectangle which contains the target.The IFCDTL is defined as
Step 3 Calculate the grey correlation degree:
where γij is the correlation degree between 
and
Step 4 Set threshold r,require r=0.9,when γij≥r,Xi and Xj is treated as the same kind of feature.
Using the theory mentioned above,we extract feature space of FSSDGB and feature space of FSODLB from 500 randomly selected image sequences in different scenes and calculate the gray correlation degree between each feature and the other features.The final gray correlation matrix can be obtained by calculating the mean value between each feature and the other features in the 500 image sequences,which are shown in Table 1 and Table 2.
(6)检测尾气排放成分：根据MT 220—90规定，在检测尾气中CO，NOx的浓度时，要测定11种运转工况下的浓度，但我国煤矿企业目前难以实现11种模拟检测实验，目前大多数煤矿现场还是采用比较单一的工况。因此，根据实际情况及交管部门关于柴油车尾气检测的经验来看，只以煤矿现场车辆在额定载荷、正常运行的工况条件下作为检测环境，根据其中CO、NOx，烟度值浓度来进行判定实验，也即是采用尾气成分检测仪与BQDC100-8型号机动车流动检测线检测无轨胶轮车车辆正常工况下的尾气成分。
Table 1 Correlation matrix of feature space of FSODLB
Let r =0.9,by observing the correlation matrix in Table 2, we discover that γTBC-TIR = 0.968 4,γETB-KSZ = 0.956 2.Since the results above are all greater than 0.9,according to Step 4 in Algorithm 1 of the grey correlation theory,we know that TBC and TIR are similar in trend and ETB and KSZ are also similar in trend.The feature space of FSODLB can reduce the dimension of measurement metrics,which only includes measurement metrics TBC,ETB and GDC.In the same way,the feature space of FSSDGB only includes TSD,ENT,ACL,IFCDTML,IFCDTGL,COM and POT.Due to the fact that γLTW-POT=0.921 6,the measurement metric LTW is removed.
Table 2 Correlation matrix of feature space of FSSDGB
3.4 Feature space similarity degree of global background
The feature space similarity degree of the global background is a quantitative value,which reflects the ability of the global background to introduce false alarms.The greater the similarity between the target and the global background is,the more difficult it is to recognize the target.In order to construct feature space similarity degree of the global background,there are two factors that need to be considered:the calculation of feature similarity between the target and the global background and the choice of the“target-like”area.Firstly,the normalized cross-correlation is used to match the target region with the global background region,and the“target-like”area can be obtained.Then based on the feature space which is constructed,the FSSDGB can be accurately calculated,the feature space includes TSD,ENT,ACL,IFCDTML,IFCDTGL,COM and POT.We show the details of this algorithm flowchart in Fig.3,and the method to compute the FSSDGB is shown in Algorithm 2.
高层次人才引进依赖于完善的人才政策法规和成熟的人才引进机制。目前，河北省现有政策并没有体现出更多的优越性，相比其他省市较为落后，没有对人才形成强大的吸引力。人才引进渠道较为单一，人才引进需求信息没有得到广泛、全面、有效的传播，限制了河北省人才引进的成效性。政府引进人才的政策信息公开程度不高，政策公布渠道不够完善，公布时间相对滞后，使很多区域外人才难以获取相关信息，影响人才集聚。
Fig.3 Algorithm flowchart of FSSDGB
Algorithm 2 FSSDGB
Input: Evaluated image,template of target.
Output: FSSDGB(0-1).
什么是“怀旧”？按照两位当代旅行文学专家克莱恩·霍普(Clenn Hooper)和蒂姆·杨斯(Tim Youngs)的观点，“怀旧是旅行写作的文化自满和权力差异的标志，历史上正是这种差异使旅行成为可能，并使基于旅行的写作接踵而至。但是，怀旧也是一种文体的标志，这种文体承认它自己无法控制各种姿态，反讽性地通过求助于一个被创造出来的过去来使自己心安理得。旅行作家的追忆经常是精心编织起来的，通过叙事纳入时间和空间，变得栩栩如生——实际上从未发生过。”“怀旧有着一张乌托邦的面孔，这张面孔转向未来的过去，这个过去只在意识形态意义上是现实的”。〔1〕140
Step 1 Input template of target and evaluated image.
Step 2 Template of target as the window slide in evaluated image,step=1.
Step 3 Calculate the normalized cross-correlation Gi,where i is the scanning number.
Step 4 Get the number of“target-like”area when Gi＞0.5.
2018 “13 Impressionen aus CHINA”，Galarie Djesany Weserstr.11·柏林
Image complexity,in the context of ATR performance,is a measure of the inherent difficulty of finding a true target in given images.An ideal image complexity metric is a mapping from the set of all images to a finite real interval.The extreme of the interval indicates the extreme in difficulty.The mapping must be monotonic in probability.
Step 6 Make sure m,the number of“target-like”area,and treat the max normalized cross-correlation in each“target-like”area as the suspected target.
Step 7 Calculate the feature similarity fchar(objj,ck)between the suspected target and real target,objj is the jth suspected target,ck is the kth feature.
where Fcomplex is the image complexity.
Step 8 Calculate fobj(objj),the similarity between the suspected target and the real target.
Step 9 Calculate fimg,the similarity between all the suspected targets and the target in the single-frame image.
Step 10 The feature space similarity degree of the global background is FSSDGB.
3.5 Feature space occultation degree of local background
The feature space occultation degree of the local background is a quantitative value,which is used to reflect the difference between the target and its local background.The definition of the local background is given in Section 3.The smaller the difference between the target and its local background is,the fewer target features can be acquired,and the more difficult it is for the target recognition system to detect the target from the local background.When constructing the feature space occultation degree of the local background,one needs to consider the local background statistical metrics comprehensively.The feature space of FSODLB is made of TBC,ETB and GDC.We show the details of this algorithm flowchart in Fig.4.
Fig.4 Algorithm flowchart of FSODLB
In Fig.4 ET is the feature space value between the local background and the target in the evaluated image.EK is the feature space value when the local background area gray values are set to 0.We calculate the Euclidean distance d(ET,EK)between vector ET and vector EK to get the FSODLB=d(ET,EK)/d(EO,EK),where EO is a zero vector.In some special cases,if the result of FSODLB is greater than 1,we consider it as 1.When the feature space value of ET is the same with the feature space value of EK,d(ET,EK)is 0,the FSODLB is 0,and vice versa.
765 Anti-breast cancer activity of ebracteolatain A
After we obtain FSSDGB and FSODLB,the statistical formula F1-Score is used to establish the relationship between the complexity of the single-frame image and the two indexes.The formula F1-Score can treat the weights of the two indicators equally and plays a role of weighted average.The maximum value is 1 and the minimum value is 0.The F1-Score expression is as follows:
where F1 and F2 present two indicators that need to be processed.
In the proposed method,the image complexity is denoted by
where FT(ck)is the kth feature of real target in feature space,Ff(objj,ck)is the kth feature of jth suspected target.
两根平行的金属轨道水平放置在磁感应强度为B的匀强磁场中(如图1)，导体棒ab在磁场中以v1的速度向右做匀速运动，并且受到向右的外力F外，其中导体棒ab与导轨垂直，其金属轨道间距为L，有一电阻R位于左端，运动过程中导体棒在远离电阻，根据楞次定律，回路中会产生逆时针方向的电流，电流在导体上从a流向b，于是导体棒在磁场中受到向左的安培力F=BIL与外力F外平衡。
3.6 NTM-based image sequence complexity
The complexity metrics of the image sequence should be associated with the tracking algorithm performance and there should be a monotone relationship between a good image sequence metric and actual performance of the tracking algorithm[37].
It is well known that the target tracking process is an accumulation process of tracking errors.As the errors increase,the target will have some jitters in the field of view.When the error reaches a certain threshold,it is easy to lose the target.Therefore,the complexity of the image sequence is not a linear sum of the single-frame image complexity.The complexity of the image is low when background interference is absent,and the increase in time has almost no effect on the tracking.Loss of the target often occurs when the target is covered or if there are too many false alarms.At this moment,there will be a large accumulation of errors,which will have a great impact on the target tracking.
For this problem,a nonlinear transformation based on median(NTM)is proposed to construct complexity of image sequence.The NTM method can be described in Algorithm 3.The complexity of the image sequence needs to consider the deviation accumulation between the single frame complexity and the reference complexity comprehensively.The Sigmoid function is introduced to intensify the high-complexity deviation and weaken the lowcomplexity deviation,which makes sequence complexity consistent with the actual tracking.
Algorithm 3 NTM-based image sequence complexity
Input: Single-frame image complexity: F1,F2,...,FN-1,FN
N:frame number of image sequence.
Output: Image sequence complexity FSC(0-1).
Step 1 Sort all the image sequence complexity in ascending order:F(1),F(2),...,F(N-1),F(N).
Step 2 Calculate the median of image sequence complexity m0.5 and use it as the standard complexity.
Step 3 for i=1:N
Step 4 Calculate the deviation between m0.5 and F(i),which is denoted by ∇F(i).
Step 5 Intensify the high-complexity deviation and weaken the low-complexity deviation with the neural network Sigmoid function.The deviation can be further denoted by ∇S(i),which corresponds to single-frame image complexity F(i).
Step 6 end for
Step 7 Image sequence complexity
4.Experimental results
To verify the performance of the algorithm,single-frame image complexity and sequence complexity are verified separately.Experimental data is divided into two types:real images and simulated images.The single-frame image complexity is almost verified by real images,covering the background of sky,ground,sea clutter,and sea-sky connection.The sequence complexity is verified by synthetic image sequences and real visual and infrared image sequences respectively.
4.1 Validity experiment of single-frame image complexity
To verify the effectiveness and accuracy of FSSDGB and FSODLB,five thousand actual images are used to evaluate the validity of indexes.The size of these images is 640×512 pixels.Due to the numerous experimental data,we will only take four typical images shown in Fig.5 as examples to explain in detail,and their evaluation values of FSSDGB and FSODLB are listed in Table 3.
Table 3 Image complexity metrics
It could be concluded from Table 3 that global background hardly introduces false alarms in Fig.5(a)and Fig.5(b)with FSSDGB=0.054 and FSSDGB=0.157 respectively,while the local background has a certain occultation in Fig.5(a)and Fig.5(b)with FSODLB=0.303 and FSODLB=0.354 respectively.Since the values of FSSDGB and FSODLB are both relatively small,it is less difficult to recognize the target,and the corresponding image complexity values are also small,which are 0.092 and 0.218 respectively.The situation of Fig.5(c)is on the contrary.There are many“target-like”areas and the feature similarity is very high,hence the FSSDGB plays an important role.The FSSDGB value of Fig.5(c)is as high as 0.945,while the value of FSODLB is as low as 0.250.The corresponding image complexity is 0.395.The maximum complexity image is shown in Fig.5(d).Both FSSDGB and FSODLB are very large.This means that not only many false targets exist in the global background,but also the local background generates high disturbance to target detection.The target features cannot be completely extracted,resulting in the phenomenon of misrecognition or loss of the target.The corresponding image complexity is 0.775.
Fig.5 Typical images
Comparing Fig.5(a)-Fig.5(d),it could be concluded that FSSDGB and FSODLB are both effective for the measurement of image complexity,and the specific reasons for why the background influences the target detection are clearly defined.At the same time,the analysis results represented above are consistent with the actual situation.It is worth noting that since the inter-frame feature cannot be calculated,zero-value processing is performed.The FSSDGB value and the image complexity value in Table 3 are slightly smaller than the calculated value of the sequence complexity.
4.2 Validity experiment of image sequence complexity
There should be a monotone relationship between an effective image sequence metric and the actual performance of the tracking algorithm[30].Therefore,to evaluate the effectiveness of image sequence complexity,the actual infrared simulation image sequences and visual image sequences are used for the purpose of verification.There are 1 000 groups of visual image sequences and 11 694 groups of infrared simulation image sequences,and each image sequence contains 600 to 1 000 image frames.The infrared synthetic image sequences are obtained by the simulation platform.The simulation platform uses Microsoft Visual Studio as the development environment and uses common modeling software to create the scene model.In the end,the platform uses OpenSceneGraph(OSG)to complete the final rendering.To test the theory further,one experiment is done on public database of real infrared image sequences.Fig.6 shows some samples of them.Finally,the relationship between the image sequence complexity and actual tracking effect is analyzed.
Fig.6 Samples of image sequences
It is widely accepted that automatic target tracking algorithms can be divided into two classes:“filtering and data association”tracking framework and“target representation and location”tracking framework[38].In this experiment,to ensure the universality of the selected algorithm,the tracking algorithm used here combines the advantages of the two mentioned tracking frameworks:the online learning is used to achieve target modeling,while the pyramidal optical flow method is used for accurate matching.
In this experiment,the tracking error is used to quantify the tracking results in given image sequences,defined as follows:
where N is the number of images in tested image sequences,i is the current frame number,soi and sri are actual target location and target location obtained by the tracking algorithm respectively.
The tracking errors of 11 694 groups infrared simulation image sequences are denoted by SS(j), j =1,2,...,11 694.The normalization processing is carried out for all the tracking errors as follows: 
=SS(j)/max(SS(j)).The tracking errors are divided into five intervals: (0,0.2], (0.2,0.4],(0.4,0.6], (0.6,0.8],(0.8,1].The mean tracking error of each interval is defined by 
[39].
The relationship between the mean tracking error of each interval and the corresponding mean value of the image sequence metric is shown in Fig.7(c).From this curve,we can see that there is a good,monotone relationship between our proposed image sequence metric and the actual tracking performance.Meanwhile,the representative metrics of the image sequence are the degree of sequence correlation and the inter-frame change degree.
Fig.7 Relationship between three methods and performance of the tracking algorithm in infrared simulation image sequences
To compare the performanceof the proposed metric with representative metrics,we have done a bit more research.The relationship between the mean tracking error of each interval and the corresponding mean value of sequence correlation(SC)is shown in Fig.7(a),and the relationship between the mean tracking error of each interval and the corresponding mean value of IFCD is shown in Fig.7(b).It is obvious that the correlations between representative metrics and actual tracking performance are poor.
In a similar way, the tracking errors of 1 000 groups visual image sequences are denoted by SV(k),k = 1,2,3,...,999,1 000.After normalization processing and division for the tracking errors,the mean tracking error of each interval is defined by 
For visual image sequences,we give the relationships between the mean tracking error of each interval and the mean value of our proposed image sequence metric,degree of sequence correlation and inter-frame change degree of target,as shown in Fig.8.We can see that there is also a strong correlation between actual performance and our metric for visual image sequences.
Fig.8 Relationship between three methods and performance of the tracking algorithm in visual image sequences
In order to test the theory further,one experiment was done on public database named The Thermal Infrared Visual Object Tracking Challenge 2016(VOT-TIR2016).There are 25 infrared image sequences and 14 of those can finish the tracking successfully.In a similar way,the tracking errors and normalization processing of 14 infrared image sequences are calculated.The relationships between the mean tracking error of every algorithm are shown in Fig.9.
Fig.9 Relationship between three methods and performance of the tracking algorithm in real infrared image sequences
It can be concluded that there is also a strong correlation between the actual performance and our metric for real infrared image sequences.
Whether the database is visual image sequences or infrared image sequences of real and simulation,the proposed image sequence metrics meet the criterion that there should be a monotone relationship between a good image sequence measure and the tracking algorithm performance.The experimental results show that the proposed metrics are more valid than other metrics,such as sequence correlation and inter-frame change degree,and are highly relevant to the performance of target tracking algorithms.
5.Conclusions
This paper discusses over 20 commonly used image and image sequence metrics and classifies them into three categories in terms of their scope.On this basis,the paper finds that the existing image metrics have obvious drawbacks and are not appropriate for the measurement of the image sequence.
To solve this problem,by reviewing these commonly used image and image sequence metrics,we integrate metrics and transform them by constructing feature space so that they will obtain complementary advantages.And based on feature space,FSSDGB and FSODLB are proposed to evaluate single-frame image complexity.In addition,to remove relevant features,the feature space is optimized by the gray relational method.Finally,NTM is proposed to construct complexity of the image sequence.We prove by experiments that the new metrics are more valid than other metrics,such as sequence correlation and interframe change degree.
In the future,we will improve the image database to better evaluate the performance of the target tracking algorithm.On this basis,we will use deep learning to build the feature space and put forward a new image sequence complexity metric with better performance.
References
[1] DIAO W H,MAO X,CHANG L.A new quality estimation method for infrared target images.Acta Aeronautical Astronautical Ainica,2010,31(10):2026-2033.(in Chinese)
[2] CHEN Y,CHEN G,BLUM R S,et al.Image quality measures for predicting automatic target recognition performance.Proc.of the IEEE Aerospace Conference,2008:1-9.
[3] CLARK L G,VELTEN V J.Image characterization for automatic target recognition algorithm evaluations.Optical Engineering,1991,30(2):147-153.
[4] LIU R,LIU E,YANG J,et al.Point target detection of infrared images with eigentargets.Optical Engineering,2007,46(11):501-503.
[5] MA Y,KONG B.A study of object detection based on fuzzy support vector machine and template matching.Proc.of the World Congress on Intelligent Control and Automation,2004:4137-4140.
[6] YANG L,YANG J.Detection of small targets with adaptive binarization threshold in infrared video sequences.Chinese Optics Letters,2006,4(3):152-154.
[7] PETERS R A.Image complexity metrics for automatic target recognizers.Proc.of the Automatic Target Recognizer System&Technology Conference,1990:1-7.
[8] BHANU B.Automatic target recognition:state of the art survey.IEEE Trans.on Aerospace and Electronic Systems,1986,22(4):364-379.
[9] GAO Z Y,YANG X M,GONG J M,et al.Research on image complexity description methods.Journal of Image and Graphics,2010,15(1):129-135.(in Chinese)
[10] WILSON D L.Image based contrast to clutter modeling of detection.Optical Engineering,2001,40(9):1852-1857.
[11] BEARD J,GLARK L G,VELTEN V J.Characterization of ATR performance in relation to image measurements.Automatic Target Recognizer Report,1985:27-55.
[12] ZHU Y,DUAN J,QIAN X F,et al.Research on the optimal selection method of image complexity assessment model index parameter.Proc.of the Applied Optics and Photonics,2015:96751K.
[13] HALLER R S. Complexity of real images evaluated by densitometric analysis and by psychophysical scaling.Arizona:University of Arizona,1970.
[14] MAO X,DIAO W H.Criterion to evaluate the quality of infrared small target images.Journal of Infrared Millimeter&Terahertz Waves,2009,30(1):56-64.(in Chinese)
[15] HARPER S,JAY C,MICHAILIDOU E,et al.Analysing the visual complexity of web pages using document structure.Behaviour&Information Technology,2013,32(5):491-502.
[16] CORCHS S E,CIOCCA G,BRICOLO E,et al.Predicting complexity perception of real-world images.PLoS One,2016,11(6):e0157986.
[17] CIOCCA G,CORCHS S,GASPARINI F,et al.Does color influence image complexity perception?Proc.of the 5th Computational Color Imaging Workshop,2015:139-148.
[18] ZHOU B,XU S,YANG X X.Computing the color complexity of images.Proc.of the International Conference on Fuzzy Systems&Knowledge Discovery,2016:1942-1946.
[19] LI M.Image measurement research for automatic target recognition performance evaluation.Wuhan:Huazhong University of Science and Technology,2006.(in Chinese)
[20] DIAO W H,MAO X,CHANG L.Quality estimation of image sequence for automatic target recognition.Journal of Electronics&Information Technology,2010,32(8):1779-1785.(in Chinese)
[21] DIAO W H,MAO X,ZHENG H C,et al.Image sequence measures for automatic target tracking.Progress in Electromagnetics Research,2012,130(1):447-472.
[22] SCHMIEDER D E,WEATHERSBY M R.Detection performance in clutter with variable resolution.IEEE Trans.on Aerospace and Electronic Systems,1983,19(4):622-630.
[23] HILGERS J W,WILLIAM P V,WILLIAM R R.Sensor and detection algorithm based clutter metrics.Proc.of SPIE,1997,30(62):267-277.
[24] SADJADI F A,BAZAKOS M E.Perspective on automatic target recognition evaluation technology.Optical Engineering,1991,30(2):1-14.
[25] ZHENG X,PENG Z M,DAI J H.Criterion to evaluate the quality of infrared target images based on scene features.Electronics&Electrical Engineering,2014,20(10):44-50.
[26] SHIRVAIKAR M V,TRIVEDI M M.Developing texturebased image clutter measures for object detection.Optical Engineering,1992,31(12):2618-2639.
[27] HARACLICK R M.Texture features for image classification.IEEE Trans.on Systems,Man,and Cybernetics,1973,3(6):610-621.
[28] WALDMAN G,WOOTTON J,HOBSON G,et al.A normalized clutter measure for images.Computer Vision Graphics&Image Processing,1988,42(2):137-156.
[29] CLARK L G,VELTEN V J.Image characterization for automatic target recognition algorithm evaluations.Proceedings of SPIE,1991,30(30):147-153.
[30] CHANG H,ZHANG J Q.Evaluation of human detection performance using target structure similarity clutter metrics.Optical Engineering,2006,45(9):1-7.
[31] CHANG H,ZHANG J Q.New metrics for clutter affecting human target acquisition.IEEE Trans.on Aerospace and Electronic System,2006,42(1):361-368.
[32] AVIRAM G,ROTMAN S R.Evaluating human detection performance of targets and false alarms,using a statistical texture image metric.Optical Engineering,2000,39(8):2285-2295.
[33] CONNERS R W,HARLOW C A.A theoretical comparison of texture algorithms.IEEE Trans.on Pattern Analysis and Machine Intelligence,2009,2(3):204-222.
[34] LI M,ZHANG G.Image measures for segmentation algorithm evaluation of automatic target recognition system.Proc.of the International Symposium on Systems and Control in Aerospace and Astronautics,2005:674-679.
[35] DIAO W H,MAO X,DONG X Y.Infrared small target image quality evaluation.Journal of Beijing University of Aeronautics and Astronautics,2008,34(11):1335-1338.(in Chinese)
[36] LI Z S,SHI Y Y,XIN H M,et al.Technological parameter optimization of disc-milling grooving of titanium alloy based on grey correlation degree.Journal of Northwestern Polytechnical University,2018,36(1):139-148.(in Chinese)
[37] TRIVEDI M M,SHIRVAIKAR M V.Quantitative characterization of image clutter:problem,progress,and promises.Proc.of the International Society for Optical Engineering,1993:1962-1967.
[38] ZHENG X.The evaluation method and application of infrared image without reference image.Chengdu:University of Electronic Science and Technology of China,2015.(in Chinese)
[39] LI J N,DUAN J,YANG X,et al.An image overall complexity evaluation method based on LSD line detection.Proc.of the IOP Conference Series:Earth and Environmental Science,2017:012162.
DOI: 10.21629/JSEE.2019.04.05
Manuscript received January 03,2019.
*Corresponding author .
This work was supported by the National Natural Science Foundation of China(61703337)and Shanghai Aerospace Science and Technology Innovation Fund(SAST2017-082).
Biographies
WANG Xiaotian was born in 1989.He received his B.S.degree in 2013 from North China Institute of Aerospace Engineering.He received his M.S.degree from School of Electronics and Information,Northwestern Polytechnical University in 2016.He is a Ph.D.candidate with the School of Astronautics,Northwestern Polytechnical University.His research interests are object detection,object tracking and image quality evaluation.
E-mail:18710993786@163.com
ZHANG Kai was born in 1979.He received his Ph.D.degree from School of Astronautics,Northwestern Polytechnical University in 2009.As an associate professor,he is working in School of Astronautics,Northwestern Polytechnical University.His research interests are guidance,navigation and control.
E-mail:singlechip@163.com
YAN Jie was born in 1960.He received his Ph.D.degree from School of Astronautics,Northwestern Polytechnical University in 1988.He is currently working as a professor and Ph.D.candidate supervisor in School of Astronautics,Northwestern Polytechnical University.His research interests are flight control,guidance,system simulation and aircraft design.
E-mail:yanjie@nwpu.edu.cn
标签：complexity论文; of论文; image论文; sequence论文; feature论文; Space论文; simi-larity论文; degree论文; global论文; background论文; (FSSDGB)论文; oc-cultation论文; local论文; (FSODLB)论文; grey论文; relational论文; method论文; nonlinear论文; transformation论文; based论文; on论文; median论文; (NTM)论文; School of Astronautics论文; Northwestern Polytechnical University论文;