一种基于草图的交互式图像搜索与融合方法与流程
本发明涉及一种基于草图的交互式图像搜索与融合方法，属于信息技术领域，特别是属于计算机视觉技术领域。
背景技术：
随着社交网络的普及，人们对图像进行处理的需求越来越旺盛。一些简单易用的图像处理软件变得十分火热。分析其成功背后的原因，无非是为非专业用户提供了简单方便的图像编辑的平台。但目前为止还没有一款软件为用户提供简易的自由图像合成的功能。
早在2010年左右，已经有研究人员针对于方便、快捷的合成一幅图像进行了研究。核心问题是如何方便快捷的获取目标物体以及背景图像？目前可用的检索手段只有两种：基于文本的图像检索以及基于草图的图像检索。单一的依靠文本检索，往往难以获得满足特定形状特征的物体；而仅仅依靠目前的草图检索技术，在检索精度上又会大打折扣。sketch2photo(参见chen,t.,cheng,m.,tan,p.,shamir,a.,hu,s.2009.sketch2photo:internetimagemontage.acmtrans.graph.28,5,article124(december2009),10pages.doi＝10.1145/1618452.1618470)将两种方式相结合，实现了自动由草图合成真实图片。但直接在线处理互联网上的图片，经过层层过滤来找到满足合成需要的图片往往需要很长时间，难以满足用户需求。为了满足时间的需求，就需要对图片提前进行离线的预处理，这就需要自建图像库。photosketcher(参见eitzm,richterr,hildebrandk,boubekeurt,alexam.photosketcher:interactivesketch-basedimagesynthesis.ieeecomputgraphappl.2011nov-dev；31(6):56-66.doi:10.1109/mcg.2011.67)采用了离线式图像库进行图像检索，不需要附加文本信息。这样虽然加快了检索速度，但精度却难以令人满意。其中一个原因是：photosketcher采用的特征提取的方法对于图像的位置、方向、尺寸上存在局限性，在bovw模型下无法考虑特征点的空间位置信息。
目前已有的草图检索技术，图像库中的图片往往是图标型图片或者场景，还没有人对日常生活中自然场景下图片进行过检索。而且就目前的应用场景来说，减少用户检索的次数，为用户返回更加精准丰富的物体素材才是检索的重点。
因此如何通过草图来对日常生活中自然场景图片进行检索，并将检索物体与目标场景合成成为目前计算机视觉领域一个急需要解决的技术问题。
技术实现要素：
有鉴于此，本发明的目的是发明一种方法，实现以日常复杂场景图片为图片库，用户只需要输入某一物体的草图，就能返回出现该物体的场景图像，并将检索物体与目标场景合成。
为了达到上述目的，本发明提出了一种基于草图的交互式图像搜索与融合方法，所述方法包括下列操作步骤：
(1)图像库建立索引文件过程，具体内容是：将图像库中源图像分割为只包含单一物体的子图像，记录其映射关系；获取子图像中物体轮廓，并且利用gf-hog算法计算其对应的特征向量；根据bovw视觉词袋模型，对所得特征向量进行聚类，获得视觉词典；然后对每一子图计算其视觉单词词频的统计直方图；按照倒排索引的方式，建立所述图像库的索引文件；
(2)基于草图获得图像检索结果过程，具体内容是：根据用户输入的草图，计算该草图的特征向量；根据步骤(1)中所得到的视觉词典，获取草图的统计直方图；根据该直方图，利用步骤(1)所得的索引文件，计算草图与各子图像的相似度，对子图像按照相似度进行排序；结合子图像的标签信息，对排序结果进行反馈；根据步骤(1)中所述的映射关系，将子图像所对应的源图像返回给用户；
(3)图像融合过程，具体内容是：从所述步骤(2)获得的检索图像，使用grabcut算法抠出所需要的物体；使用possion融合方法把抠出的物体放进背景图像，实现图像融合。
所述步骤(1)中获取子图像中物体轮廓的具体内容是包括如下操作步骤：
(1101)使用物体检测算法yolo，对图像库中每一幅图像中的物体进行检测，获得只包含单个物体的子图像、其对应的标签信息以及标签准确度；
(1102)对上述的每个子图像，使用显著性区域检测算法saliencycut进行显著性区域检测，将子图像中前景即物体与背景分割开，形成二值化图像；
(1103)对上述的二值化图像，使用canny算法计算得到物体的轮廓。
所述步骤(1)中根据所得到的子图像中物体轮廓，计算其对应的特征向量的具体内容是包括如下操作步骤：
(1201)首先，以二值化轮廓图m作为输入，其中m(x,y)＝1表示轮廓像素点,m(x,y)＝0表示非轮廓像素点，x,y分别表示像素点的行和列坐标，运用以下公式求得轮廓像素点的梯度方向θ(x,y)，从而获得轮廓图m的稀疏梯度方向场ψ：
(1202)在保持轮廓像素点梯度方向不变的情况下，对非轮廓像素点的梯度方向进行插值处理，从而获得稠密梯度方向场θω；同时为使所述的稠密梯度方向场θω在整个图像坐标ω∈r2满足平滑性，需要对稠密梯度方向场θω进行拉普拉斯平滑约束，具体如下式：
该式中，θ表示待求的像素点的梯度方向，ω表示整个图像坐标，∫∫ω是在整个图像坐标系中对运算符内数值求积分操作，
(1203)在满足狄利克雷边界条件的基础上，上式用如下泊松方程来进行求解：
该式中，
其中，对于图像中任一像素点p，np表示像素点p的四个邻域点的集合，在四邻域条件下|np|＝4，q表示np内一点，
(1204)在获得所述的稠密梯度方向场θω后，以轮廓像素点为中心，利用hog算法对θω进行多尺度采样，构造该轮廓图的特征向量。
所述步骤(1)中对所计算得到的物体的特征向量进行聚类所采用方法是k-means聚类方法。
步骤(1)中所述的按照倒排索引的方式，建立所述图像库的索引文件的具体内容是包含如下操作步骤：
(1301)根据bovw模型，将所有子图像的词频统计直方图合并在一起组成一个n行k列的直方图矩阵，其中n为图像库中子图像的个数，k为聚类中心数，将矩阵保存到文件中；
(1302)按列遍历上述直方图矩阵，统计每一列中值不为0的图像的标号，并将统计结果写入文件中，这样就获得了所需要的倒排索引文件。
所述步骤(2)的具体内容是包含如下操作步骤：
(21)按照步骤(1)中所述方法，计算输入草图的特征向量；
(22)利用步骤(1)中获得的视觉词典，统计视觉单词出现的频率，得到草图对应的统计直方图q；
(23)利用步骤(1301)和(1302)获得的倒排索引结构以及矩阵，计算查询草图与子图像的相似度，相似度公式定义如下：
该式中，q表示查询草图的统计直方图，di表示图像库中子图像i的统计直方图，
(24)通过步骤(23)，计算得到子图像i与用户输入草图的相似度si，利用如下公式求得在top-k下出现的类别的反馈值ft：
上式中，ci为yolo返回的子图像i标签的准确度,ti为子图像i的标签，t为某一类别标签；利用上述公式获得的各个类别标签的反馈值ft，然后利用如下公式对top-n下子图像进行相似度重计算，这里n一般取大于等于k的自然数，其中si为反馈前子图像i的相似度，s'i为重新计算获得的子图像i的相似度；
在top-n下对s'i进行重新排序；
(25)利用步骤(1)中的映射关系，返回相似度最高的前k张子图像所对应的源图像。
所述步骤(3)的具体内容是包含如下操作步骤：
(31)对于草图检索返回的结果，使用grabcut算法，将图像中的物体抠出，然后将抠图结果留在备选区待用；
(32)待所有物体都被抠出放入备选区后，将备选区中物体全部放置在背景图片上，调整其大小以及位置，然后使用possion融合，将物体融合到背景中，从而获得一副自然的图片。
本发明的有益效果在于整体处理时间上相对于sketch2photo大大缩短，而且提供了更加自由化的用户交互，而相比于photosketcher，本发明方法能够提供较高的检索精度，并大大减少用户检索的次数，为用户提供更加合理、丰富的素材。
附图说明
图1是本发明提出的一种基于草图的交互式图像搜索与融合方法的流程图。
图2是本发明实施例所用的一个建库图像。
图3是对图2进行步骤(1101)操作获得的结果图。
图4是利用图3中矩形框将图片进行分割得到的只包含单一物体的子图像。
图5是对图4图像进行步骤(1102)操作得到的二值化图像。
图6是对图4进行步骤(1103)操作获得的轮廓图。
图7是对图2中所对应的轮廓图进行步骤(1201)操作获得的稀疏梯度方向场。
图8是对图3中稀疏梯度方向场进行步骤(1203)获得的稠密梯度方向场。
图9是本发明中步骤(1204)中hog算法的示意图。
图10是本发明实施例所用的一个查询图像。
图11是以图10为查询实例，在不加入标签反馈时按相似度排序的top-10结果。
图12是以图10为查询实例，并且加入标签反馈后按相似度排序的top-10结果。
图13是本发明实施例中搜索以及融合图片的一些实例。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面结合附图对本发明作进一步的详细描述。
参见图1，介绍本发明提出的一种基于草图的交互式图像搜索与融合方法，所述方法包括下列操作步骤：
(1)图像库建立索引文件过程，具体内容是：将图像库中源图像分割为只包含单一物体的子图像，记录其映射关系；获取子图像中物体轮廓，并且利用gf-hog算法(参见ruihu,markbarnard,johncollomosse.gradientfielddescriptorforsketchbasedretrievalandlocalization.icip2010.doi:10.1109/icip.2010.5649331)计算其对应的特征向量；根据bagofvisualwords(bovw,参见sivicj,zissermana.videogoogle:atextretrievalapproachtoobjectmatchinginvideos[c]//null.ieeecomputersociety,2003:1470.)视觉词袋模型，对所得特征向量进行聚类，获得视觉词典；然后对每一子图计算其视觉单词词频的统计直方图；按照倒排索引的方式，建立所述图像库的索引文件；
(2)基于草图获得图像检索结果过程，具体内容是：根据用户输入的草图，计算该草图的特征向量；根据步骤(1)中所得到的视觉词典，获取草图的统计直方图；根据该直方图，利用步骤(1)所得的索引文件，计算草图与各子图像的相似度，对子图像按照相似度进行排序；结合子图像的标签信息，对排序结果进行反馈；根据步骤(1)中所述的映射关系，将子图像所对应的源图像返回给用户；
(3)图像融合过程，具体内容是：从所述步骤(2)获得的检索图像，使用grabcut算法(参见carstenrother,vladimirkolmogorov,andrewblake.“grabcut”—interactiveforegroundextractionusingiteratedgraphcuts.siggraph'04acm.doi:10.1145/1186562.1015720)抠出所需要的物体；使用possion融合方法(参见patrickperez,michelgangnet,etal.possionimage
所述步骤(1)中获取子图像中物体轮廓的具体内容是包括如下操作步骤：
(1101)使用物体检测算法yolo(参见josephredmon,santoshdivvala,rossgirshick,alifarhadi.youonlylookonce:unified,real-timeobjectdetection.cvpr.2016.doi:10.1109/cvpr.2016.91)，对图像库中每一幅图像中的物体进行检测，获得只包含单个物体的子图像、其对应的标签信息以及标签准确度；
参见图2，图2为本发明实施例所用一个建库图像，利用yolo算法，可以得到图3所示的结果。从图3中可以看出该算法将图2中的物体准确框出，并且给出了其标签为“horse”，标签的准确度为0.92。利用图3中矩形框的坐标，可以将图2切割为图4所示的结果。
使用上述步骤将物体分割开，便于以单个物体为目标的搜索，减少其他物体所带来的干扰。同时获得物体的标签，能够为草图检索加入语义信息，从而进一步提高草图检索的准确度。
(1102)对上述的每个子图像，使用显著性区域检测算法saliencycut(参见ming-mingcheng,niloyj.mitra,xiaoleihuang,philiph.s.torr,andshi-minhu.globalcontrastbasedsalientregiondetection.ieeetransactionsonpatternanalysisandmachineintelligence.2014.doi:10.1109/tpami.2014.2345401)进行显著性区域检测，将子图像中前景即物体与背景分割开，形成二值化图像；
参见图5，图5为对图4进行sakiencycut算法进行处理后得到的二值化图片，白色部分为物体，黑色部分为背景。
采用显著性区域检测算法能够保留物体基本轮廓的同时，有效滤除背景所带来的干扰，从而获得高质量的轮廓图图片集。
(1103)对上述的二值化图像，使用canny算法(参见cannyj.acomputationalapproachtoedgedetection[j].patternanalysis&machineintelligenceieeetransactionson,1986,pami-8(6):184–203.)计算得到物体的轮廓。
参见图6，图6为对图5中图片进行cany算法提取轮廓后的结果图。
所述步骤(1)中根据所得到的子图像中物体轮廓，计算其对应的特征向量的具体内容是包括如下操作步骤：
(1201)首先，以二值化轮廓图m作为输入，其中m(x,y)＝1表示轮廓像素点,m(x,y)＝0表示非轮廓像素点，x,y分别表示像素点的行和列坐标，运用以下公式求得轮廓像素点的梯度方向θ(x,y)，从而获得轮廓图m的稀疏梯度方向场ψ：
参见图6，图6为输入的二值化轮廓图m,图7为计算得到的稀疏梯度方向场ψ的表示图。
(1202)在保持轮廓像素点梯度方向不变的情况下，对非轮廓像素点的梯度方向进行插值处理，从而获得稠密梯度方向场θω；同时为使所述的稠密梯度方向场θω在整个图像坐标ω∈r2满足平滑性，需要对稠密梯度方向场θω进行拉普拉斯平滑约束，具体如下式：
该式中，θ表示待求的像素点的梯度方向，ω表示整个图像坐标，∫∫ω是在整个图像坐标系中对运算符内数值求积分操作，
(1203)在满足狄利克雷边界条件的基础上，上式用如下泊松方程来进行求解：
该式中，
其中，对于图像中任一像素点p，np表示像素点p的四个邻域点的集合，在四邻域条件下|np|＝4，q表示np内一点，
参见图8，图8为求解得到的稠密梯度方向场θω的表示图。
(1204)在获得所述的稠密梯度方向场θω后，以轮廓像素点为中心，利用hog算法(参见n.dalalandb.triggs,“histogramsoforientedgradientsforhumandetection,”incivr,newyork,ny,usa,2007,pp.401-408,acm)对θω进行多尺度采样，构造该轮廓图的特征向量。
在实施例中，本发明将方向量化为了9个方向，以轮廓点像素为中心，构造3乘3大小的窗口，因此该窗口包含9个子窗口。为了构造尺度不变性，每个子窗口边长分别选取7、11以及15个像素点长度进行方向统计，因此每个子窗口可以得到一个9维的向量。将9个子窗口的向量进行合并，然后将统计结果进行归一化，这样就获得了81维的特征向量。该算法示意图参见图9。这里给出了图8中某个轮廓像素点的3个尺度的特征向量：
a7＝[0,0,0.366116,0.146446,0,0,0,0,0,0,0,0.0313814,0.188288,0.198749,0.0941441,0,0,0,0,0,0,0,0,0.156907,0.355656,0,0,0,0.135986,0.376576,0,0,0,0,0,0,0.0523023,0.115065,0.0732232,0.0313814,0.0313814,0.0523023,0.0732232,0.0627627,0.0209209,0,0,0,0,0,0,0.0836837,0.428879,0,0.0104605,0.0627627,0.0313814,0.0836837,0.135986,0.104605,0.0523023,0,0.0313814,0.0836837,0.135986,0.0418418,0.0418418,0.0523023,0.0418418,0.0732232,0.0104605,0.0313814,0.0523023,0,0,0,0.0104605,0,0.0836837,0.355656,0.0104605]t
a11＝[0,0,0.325097,0.15462,0,0,0,0,0,0,0,0.00792921,0.174443,0.186336,0.111009,0,0,0,0,0,0,0,0,0.0951505,0.356814,0.0277522,0,0,0.0356814,0.416283,0.0277522,0,0,0,0,0,0.0277522,0.0951505,0.0792921,0.0277522,0.0237876,0.0475752,0.0832567,0.0792921,0.0158584,0,0,0,0,0,0,0.0594691,0.420248,0,0.00792921,0.0317168,0.0277522,0.0237876,0.138761,0.162549,0.0673983,0,0.019823,0.130832,0.122903,0.0555044,0.0475752,0.0436106,0.0317168,0.0277522,0,0.019823,0.00792921,0.0039646,0,0,0,0,0.0475752,0.412319,0.00792921]t
a15＝[0,0,0.327283,0.141753,0,0,0,0,0,0,0,0.0020846,0.170938,0.168853,0.122992,0.00416921,0,0,0,0,0,0,0,0.0604535,0.335621,0.0729612,0,0,0.0416921,0.396075,0.0145922,0.00833842,0,0.0020846,0,0.00625381,0.0187614,0.0750458,0.089638,0.0333537,0.0291845,0.0458613,0.0854688,0.079215,0.0125076,0,0,0,0,0,0,0.0416921,0.427344,0,0.0125076,0.0541997,0.0333537,0.0270999,0.0771304,0.183445,0.0729612,0,0.00833842,0.223053,0.0812996,0.0354383,0.0312691,0.0437767,0.0145922,0.020846,0.00833842,0.010423,0.0020846,0.00416921,0,0.0020846,0,0.0020846,0.0125076,0.437767,0.00833842]t
所述步骤(1)中对所计算得到的物体的特征向量进行聚类所采用方法是k-means聚类方法。
在实施例中，本发明选取聚类中心k＝5000，这样就可以获得一个5000行81列的视觉词典矩阵，每一行为一个聚类中心的视觉单词。已知一张图片的所有特征向量，可以通过比较特征向量与聚类中心的距离找到距离最近的聚类中心，从而按照聚类中心进行词频统计，获得一个5000维的词频统计直方图。
步骤(1)中所述的按照倒排索引的方式，建立所述图像库的索引文件的具体内容是包含如下操作步骤：
(1301)根据bovw模型，将所有子图像的词频统计直方图合并在一起组成一个n行k列的直方图矩阵，其中n为图像库中子图像的个数，k为聚类中心数，将矩阵保存到文件中；
在本发明的实施例中，发明人使用了microsoftcoco验证图像库，该图像库参见：http://mscoco.org/dataset/#download，该图像库包含40k张图片，每张图片包含多种物体，经过步骤(1101)切分为88266张子图像，即n＝88266，而聚类中心数k＝5000，这样就构建了一个88266行5000列的矩阵。下面给出了其中一行的数据，该行表示图4子图像所对应的词频统计直方图。
r＝[0,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0.000586166,0.00293083,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00644783,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0,0,0,0,0.00293083,0,0,0.000586166,0,0,0.000586166,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0.000586166,0.00117233,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0,0.000586166,0,0.0017585,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.007034,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0.000586166,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0.00586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0.000586166,0,0,0,0,0,0,0,0.00293083,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00586166,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0.00410317,0,0,0,0,0.00117233,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0.00410317,0,0.0017585,0.00117233,0.00586166,0,0,0,0.003517,0,0.00410317,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0.00644783,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0.00644783,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0,0,0.000586166,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.0017585,0,0,0.000586166,0,0,0,0,0,0,0.00234467,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00410317,0,0.0017585,0.00234467,0,0,0.00293083,0.0017585,0,0,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0.003517,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0.0017585,0,0,0,0,0.0017585,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0.000586166,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0.00117233,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0.000586166,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0052755,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0.0017585,0.0017585,0.003517,0,0,0,0,0,0.00996483,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0.000586166,0,0,0,0,0,0,0,0,0.00410317,0.000586166,0.00410317,0,0,0,0,0,0.0017585,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0.00117233,0,0,0,0,0.00468933,0,0.0017585,0,0,0,0,0,0,0,0.00468933,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.0017585,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0.000586166,0,0.00117233,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0.00293083,0,0,0,0.00117233,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0.0017585,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0.00117233,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.00644783,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0.000586166,0.00293083,0,0,0.00117233,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00410317,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0.00820633,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.00293083,0.000586166,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.010551,0,0,0,0.000586166,0.000586166,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0.0017585,0,0,0,0,0,0.000586166,0,0,0,0.00117233,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0.00117233,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0.00117233,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0.000586166,0,0,0,0.00117233,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0.0269637,0,0,0,0,0,0,0,0.00937866,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.003517,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0.000586166,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0.00644783,0.00234467,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0.00117233,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0.00293083,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0.00234467,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00644783,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0.00586166,0,0,0.00117233,0.00117233,0,0,0.003517,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0.00410317,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0.00410317,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.0134818,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.00234467,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0.000586166,0,0,0.000586166,0,0,0,0,0,0.000586166,0,0,0,0,0.000586166,0,0,0,0,0,0,0.007034,0,0,0.00117233,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00586166,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0.0017585,0.00410317,0,0,0,0,0.00117233,0,0,0.000586166,0,0,0.0087925,0,0,0,0,0,0,0.00410317,0.00762016,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0.00410317,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0.00117233,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00468933,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0.000586166,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0.00117233,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0.0017585,0,0,0.0017585,0.00410317,0.00293083,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0111372,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0.00117233,0,0,0,0,0,0,0,0.00293083,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.0152403,0,0,0,0,0,0,0.0123095,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0.00468933,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0.00234467,0,0.00586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0.00644783,0,0,0,0,0.00996483,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0.00293083,0,0,0,0,0,0,0,0.0169988,0,0,0,0,0.00234467,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0.00586166,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0.00762016,0,0,0,0,0.00234467,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0.000586166,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0.0017585,0.003517,0,0,0,0,0,0,0,0,0,0,0.000586166,0.000586166,0.00117233,0,0,0,0,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0.0052755,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00293083,0,0,0,0,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.010551,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0.00410317,0,0,0,0,0,0,0,0.000586166,0,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0.00117233,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0.0017585,0.000586166,0,0,0,0,0,0,0,0,0,0,0,0.00762016,0,0,0,0.0017585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.00234467,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00762016,0,0,0,0.003517,0,0.003517,0,0,0.003517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.000586166,0,0,0,0,0,0.000586166,0]
(1302)按列遍历上述直方图矩阵，统计每一列中值不为0的图像的标号，并将统计结果写入文件中，这样就获得了所需要的倒排索引文件。
通过观察步骤(1301)获得的结果可知，最后得到的矩阵是一个稀疏矩阵。因此可以通过统计非零值图片标号、建立倒排索引的方式加速计算过程。对直方图矩阵进行统计，下面给出了其中一列的统计结果，其中数字代表图片的序号，该向量表示序号所对应的图片含有该聚类中心所对应的特征向量。
i＝[8689108375383554623706871939967102710301166119612741592160316271697173319221973202320952145217222442383242124632553272228872905291729403046311931873330333933843394340734163632382940284268436245424554455946194640467646914700475049524955496550775144515551845279529253885394544356415665569357205731574357505836593459986018625062596372645065406596659766646758676067776809690370717151719372647283730973617471765376547748776978387854793979817988800480068042806982788391846285148629872888088834883588738982901191479267951295459631969699161003710165102821038810730110111107911096111371124611282113741138011381115131157411668116801171811727117611179011875119561202812216122401226612300123881250912585126111263812692127031274212750127931295813024130281304713058131791320413256133211339113705138321385513881142641429614416145271453114627146311471014858149731499115100151641521015419154281543615521155841559715703157821598116029161571627716431164771648916667167321697417127174191744417557176651767117685177351787517881179231795017992180171802418054182211829318297183081831318482184971855618654186691871318928190251906819216192681927719311193831948119504196241963319659197311988920166204002046520479204882058320752209612117221254213042135121409214622169721757217712179921904222032225222311226542275422786228082281022901229022303923101232612327223373234032346423481235722364723721237812380523839239572398123983240582411724232243462439324682247742482924831248742488824982251402524125299253402539025452254672598526037260482616326285263112631326318264052652926710267122671826842271372720227289273092744527472278192790627951279762799628005280512816028206282862837128496285022856828612286582873128981290562913729165291832939829455294602957729641296502972229744297772981529872299672999430213302703027430303303303055030666309953102231220312603138031422314893149131660318843195731990320193205332081320963210832145321473219532277324513252732686327733279732817328923314133203332523327333322333333336433390334143347333502336383364133847340093415134241342993430934329344653451634541346513475334817349273496735026350343505035100352403524935474355093551635598357093580435890359373596636135361783622436267363683647736547366273666936722368353689036903371233724137251372733733137516375203752737733378173803738120383513846738505385313902739140393553966539685397083979139842399834029740429404744084240903411934148341503415324161641740417464176141808420674213642149421784256042686427834282942926430074306643257438314385644080442694427844411444694450044761447634494145009450794514745190452834530345350453844546045751459324623146265462734645946827468294688746996470164702047102471184713047253473954765847757478214786647924479724798748099484494845448495485274858948617486964878448883489064931049337494154944049471496044964149653496594972049732497964979949824499445024850403504335063250644508315083950868509365129251323513895182751830518525204952086521465224552289523105274052762529635309853111531345315753347533565349153678537765384154041543415455754650549265513255143552115528655695557175574655789559655629756435567155680956844569685710957116571235717457265573915780357848578725794758043581025828558498585505874158921589355893758938591525919859224592605947759493596065961359682597185983660021600396010360239604676058560650606806076260777608246089560935609446096260989610656111361177615826175861790618676197261987621866239462434625276254362701628066283862928629376295562982630386304863228632586340563455634886355263662641056413764148644096453764623650566540565456655176564465646656936587965972659776605166261665656657766784671106714267149677466786868049684936854168734689886905369066690876915669313696866974570293704447065970679707247090671168711797188971980719967215472260722897234972462725977270372724727827282672971733127338073485735737360673612736367430574317743327435174437747927495875295753887553675537755417555475564755687559875703758827614576331763517660276825770657719577370773767752377545776347784877921779277803378243785407860978792788117903879092792427927379646797197979179863800288012280154801878020880421804628079781251813888145981603816558168982004821028212182364824258246682603827748278482961830848316783273832858338283522836588368283869839728402284101843278466584840850678542486210862758630386372863988644886747868358711987268872978742887489876778773287740878628804288200]t
所述步骤(2)的具体内容是包含如下操作步骤：
(21)按照步骤(1)中所述方法，计算输入草图的特征向量；
(22)利用步骤(1)中获得的视觉词典，统计视觉单词出现的频率，得到草图对应的统计直方图q；
参见图10，图10所示图像作为查询草图；
(23)利用步骤(1301)和(1302)获得的倒排索引结构以及矩阵，计算查询草图与子图像的相似度，相似度公式定义如下：
该式中，q表示查询草图的统计直方图，di表示图像库中子图像i的统计直方图，
上式表示两个向量的余弦相似度，在此基础上同时加入了文档检索中常用的tf-idf(termfrequency–inversedocumentfrequency)算法，上式中idfp即tf-idf算法中表述的逆文档频率idf，而fq,p以及
以图10为查询草图，下面给出了top-50的图片的相似度s、图片标签t以及图片序号i，而top-10所对应的图片如图11所示。
s＝[2.19008,1.22887,0.978853,0.915278,0.89948,0.886331,0.884973,0.880953,0.879824,0.838481,0.838048,0.836581,0.8161,0.769038,0.747189,0.711824,0.71155,0.708184,0.703801,0.701853,0.697603,0.694958,0.679824,0.665309,0.664681,0.647052,0.642052,0.634729,0.63425,0.633312,0.633138,0.632802,0.619647,0.619475,0.616024,0.613952,0.60772,0.606761,0.593584,0.593071,0.592695,0.591643,0.590981,0.588569,0.571987,0.571159,0.569874,0.566539,0.564236,0.560989]
t＝[bird,bird,bird,bird,bird,bird,surfboard,motorbike,person,knife,bird,bird,bird,,bird,motorbike,,bear,bird,,bird,person,bird,dog,,bird,person,bird,bird,bird,,bird,bird,bird,bird,bottle,bird,,,bird,bird,bench,carrot,surfboard,bird,elephant,bird,bird,,,]
i＝[8222,2608,1032,4400,3581,9818,1391,3149,7339,4391,3433,4180,9524,1406,8501,8573,68,9558,4947,8923,9411,6145,3008,301,5224,6028,594,9678,4020,2959,6495,5134,3660,4638,8502,8137,4131,7880,8982,1638,9528,4798,9165,6185,2616,6379,4373,3198,7251,7315]
在本实施例中，对于标签准确度ci<0.5的标签，由于标签是不准确的，会对反馈带来误差，因此发明人忽略了其标签信息，所以t中有些标签信息为空白。
(24)通过步骤(23)，计算得到子图像i与用户输入草图的相似度si，利用如下公式求得在top-k下出现的类别的反馈值ft：
上式中，ci为yolo返回的子图像i标签的准确度,ti为子图像i的标签，t为某一类别标签；利用上述公式获得的各个类别标签的反馈值ft，然后利用如下公式对top-n下子图像进行相似度重计算，这里n一般取大于等于k的自然数，其中si为反馈前子图像i的相似度，s'i为重新计算获得的子图像i的相似度；
在本实施例中，发明人选取k＝10，n＝50，发明人在top-10下求取出现类别的反馈值，然后对top-50下图片进行相似度重计算。以图10为查询草图，根据求步骤(23)中所得结果，ft求取过程举例如下，其中c为top-10下图片标签的准确度：
c＝[0.98185,0.978674,0.982609,0.987828,0.881285,0.996606,0.50939,0.540037,0.635041,0.668688]，
对于无标签信息的图片，计算其反馈值时令ft＝0，即令s'i＝si。相似度重计算后的结果s'如下：
s′＝[1.31416,1.07447,0.992579,0.96962,0.963754,0.958821,0.889592,0.885986,0.885792,0.846343,0.940286,0.939712,0.93163,0.769038,0.903424,0.722784,0.71155,0.708184,0.884784,0.701853,0.882061,0.708438,0.874161,0.665309,0.664681,0.859239,0.657248,0.8535,0.853275,0.852835,0.633138,0.852596,0.846377,0.846295,0.844649,0.613952,0.840664,0.606761,0.593584,0.833545,0.833361,0.591643,0.590981,0.602009,0.823097,0.571159,0.822036,0.820356,0.564236,0.560989]
在top-n下对s'i进行重新排序；
对上述s'进行重新排序，得到的相似度s”、图片标签t以及图片序号i'如下，top-10所对应的图片如图12所示。
s″＝[1.31416,1.07447,0.992579,0.96962,0.963754,0.958821,0.940286,0.939712,0.93163,0.903424,0.889592,0.885986,0.885792,0.884784,0.882061,0.874161,0.859239,0.8535,0.853275,0.852835,0.852596,0.846377,0.846343,0.846295,0.844649,0.840664,0.833545,0.833361,0.823097,0.822036,0.820356,0.769038,0.722784,0.71155,0.708438,0.708184,0.701853,0.665309,0.664681,0.657248,0.633138,0.613952,0.606761,0.602009,0.593584,0.591643,0.590981,0.571159,0.564236,0.560989]
t＝[bird,bird,bird,bird,bird,bird,bird,bird,bird,bird,surfboard,motorbike,person,bird,bird,bird,bird,bird,bird,bird,bird,bird,knife,bird,bird,bird,bird,bird,bird,bird,bird,,motorbike,,person,bear,,dog,,person,,bottle,,surfboard,,bench,carrot,elephant,,,]
i′＝[8222,2608,1032,4400,3581,9818,3433,4180,9524,8501,1391,3149,7339,4947,9411,3008,6028,9678,4020,2959,5134,3660,4391,4638,8502,4131,1638,9528,2616,4373,3198,1406,8573,68,6145,9558,8923,301,5224,594,6495,8137,7880,6185,8982,4798,9165,6379,7251,7315]
这样，直观上讲用户并不需要输入文本信息，系统会根据外形判断出对应草图最可能属于的几种类别，并优先返回满足这些类别外形最相似的物体。也就是说，用户画得越像返回结果越令用户满意。这点，可以对照图11以及图12看出，在未加入反馈前，返回结果中存在与鸟类外形相似的物体，而通过对初始结果进行统计，系统能够判断出图5草图更加形似鸟类，于是对相似度进行反馈后，标签为鸟类的图片排序结果会被提前，而其他物体则会被置后。
(25)利用步骤(1)中的映射关系，返回相似度最高的前k张子图像所对应的源图像。
之所以返回源图像，主要基于以下考虑：目前几乎所有草图检索系统都是直接使用单一物体的图标型图片作为图片库，大部分人忽视了物体之间的相关性。而在实际应用中，这些相关物体又极大可能出现在用户所想象的场景中，如：一只跳跃的狗经常与一只飞碟出现在同一场景下，所以当用户画一只跳跃的狗时，如果飞碟也同时在图片中，这样会大大缩减用户的检索次数。本发明为这种情况提供了可能。这是之前任何一种草图检索系统所不具备的。
所述步骤(3)的具体内容是包含如下操作步骤：
(31)对于草图检索返回的结果，使用grabcut算法，将图像中的物体抠出，然后将抠图结果留在备选区待用；
(32)待所有物体都被抠出放入备选区后，将备选区中物体全部放置在背景图片上，调整其大小以及位置，然后使用possion融合，将物体融合到背景中，从而获得一副自然的图片。
参见图13，图13给出了本发明实施例中的搜索以及融合图片的一些实例。
实例a中，用户通过草图搜索“标志牌”以及“汽车”的图片，然后将图片中的“标志牌”以及“汽车”抠出，放置在“街道”背景的图片中，调节其大小以及位置后进行融合，从而得到最后的结果。
实例b中，用户通过草图搜索“展翅飞翔的鸟”以及“吃草的马”的图片，然后将图片中的“展翅飞翔的鸟”以及“吃草的马”抠出，放到“草原”背景的图片中，调节其大小以及位置后进行融合，从而得到最后的结果。
实例c中，用户通过草图搜索“单板滑雪者”的图片，从返回结果中，选取了包含多个“滑雪者”的图片，因为其他姿势的“滑雪者”也是用户所想要的素材，然后用户将图片中的两位“滑雪者”抠出，放到“滑雪场”背景的图片中，调节大小以及位置后进行融合，从而得到最后的结果。
实例d中，假设用户要合成一幅棒球运动员们在棒球场上打棒球的场景。在之前提到的系统中，用户往往需要分别搜索各个位置的运动员，才能得到所需的素材，而这既耗时又耗力。在本发明中，用户画了一张“击球手”的草图，从返回的结果中，选取了一张包含多个位置的运动员的图片，因为这些运动员也是合成图片中所需要的素材，这样省去了反复查找所浪费的时间。将图片中的三位“运动员”抠出放置在“棒球场”背景的图片上。然后利用同样的方法搜索“投球手”，将其抠出放置在背景图片上，调节所有素材的大小以及位置后进行融合，从而得到最后的结果。
发明人在“flickr160”数据库和microsoftcoco验证数据集上进行了大量实验，实验结果证明本发明的方法是非常有效的。
- 还没有人留言评论。精彩留言会获得点赞！