Panoramic stereo vision and depth-assisted edge detection92
上亿文档资料，等你来发现
Panoramic stereo vision and depth-assisted edge detection92
Panoramicstereovisionand；ArisEconomopoulos,Drakou；NationalandKapodistrianU；PeterPeer,FrancSolinaUni；Abstract；Thisarticledealswiththef；1.Introduction；Thispaperdealswithtwodis；1.1.Mo
Panoramicstereovisionanddepth-assistededgedetectionArisEconomopoulos,DrakoulisMartakosNationalandKapodistrianUniversityofAthens,GreecePeterPeer,FrancSolinaUniversityofLjubljana,Slovenia{pathway,martakos}@di.uoa.grAbstractThisarticledealswiththefusionoftwoinitiallyindepen-dentsystems.The?rstsystemutilizespanoramicimagesandrotationalgeometrytocreatestereoimagepairsandrecoverdepthinformationusingasinglecamera.Thesec-ondsystemutilizesdensedepthestimatesinordertodriveaprocess,whichimprovesedgedetectionbyfactoringinthedistance(scale)fromthecameraateachpoint.Dis-tancedataisfedtoasmoothingpre-processor,whichsplitstheimageintolayersbasedonpixeldistanceandindepen-dentlysmootheseachone,utilizingadifferentsigma.Thisvariationofsigmaacrossimagedepthensuresthatnoiseiseliminatedwithminimallossofoveralldetail.Signi?cantimprovementstotheresultsofsubsequentedgedetectionoperationsareobservedforimageswithgreatdepthvari-ations.Themethodsutilizedbybothsystemsareanalysedandtheirbene?tsanddrawbacksareexposed.Theprocessofintegrationisalsodetailed,outliningproblemsandsolu-tionsthatwereadopted.Theapplicationsandconstraintsofthesystemarealsodiscussed.1.IntroductionThispaperdealswithtwodiscretesubjects.Twosep-aratesystemsarepresented,onedealingwithpanoramicstereovisionandtheotherwithstereo-assistededgedetec-tion.Bothsystemsweredevelopedindependently.1.1.MotivationEdgedetectionisoneofthemostbroadlyusedopera-tionsintheanalysisofimages.Therehasbeenaplethoraofalgorithmspresentedwithinthiscontextandseveraldif-ferentapproachestothesubjecthavebeenwelldocumented[4,6,10,13,21].Inthevastmajorityofcases,edgedetectionhasbeenconsideredtobeanearlystepintheprocessofanalysis.Thishasallowedresearcherstoeffectivelyutilizeedgedata{peter.peer,franc.solina}@fri.uni-lj.sitoguidehigher-leveloperations.Inadditiontoitsutilizationinotherareas,edgedetectionhasalsobeenusedinseveralapplicationsofstereoscopyinordertoimprovestereocor-relationalgorithmsandtoaidinoverallsystemrobustness[9,11,21].Thereisverylittleresearchhowever,thatlooksatthere-lationshipofstereopsisandedgedetectionfromadifferentstandpoint.Speci?cally,wesubmitthatinhumans,whosevisionsystemourdisciplineisafteralltryingtomodel,thepresenceofrobustandef?cientstereoscopymayactuallybeimplicitlyaidingtheprocessofedgedetection,ratherthantheotherwayaround.Modernmedicineandneuroscienceareasofyetunabletoprovideapreciseblueprintofthealgorithmsinherentinhumanvisualperception.Wehavepreciouslittleinformationonhowthebrainprocessesvisualstimuli,andtheorderinwhichoperatorsareappliedandthepossibleinterdependenciesthattheyexhibitarenotamongit.Toexplorethishypothesis,andtoestablishwhetheritcouldholdtrueaswellaswhetheritcouldbeappliedtocomputervisionproblems,wedecidedtoanalyzethework-ingsofanexistingedgedetectionoperatorandenhanceitsoastomakeuseofdepthdata.Theassumptionmadehere,ofcourse,isthatarobuststereovisionsystemthatdoesnotdependonedgedetec-tiontechniquesisavailable.Suchasystemispresentedin[23].Analternativeapproachthatisabletoproducedensedepthmaps,andhencedrivetheaforementionedprocess,ispanoramicstereoimaging.Astandardcamerahasalimited?eldofview,whichisusuallysmallerthanthehuman?eldofvision.Becauseofthatfact,peoplehavealwaystriedtogenerateimageswithawider?eldofview,uptoafull360degreespanorama[17].Pointsandlinesshouldbevisibleonallsceneimages.Thisisapropertyofpanoramiccam-erasandrepresentsourfundamentalmotivationforthegen-erationofdepthimagesusingthistechnique.Ifwewouldtrytobuildtwopanoramicimagessimultaneouslybyusingtwostandardcameras,wewouldhaveproblemssincethescenewouldnotbestatic.Inthisarticle,weattempttoshowhowknowledgeofascene’sstereogeometryandcorrespondingdisparitiescanbeusedtoassistinthetaskofedgedetection.Totestourtheories,wechosetoworkwiththeCannyalgorithmforedgedetection[2];anoperatorthathasreceivedmuchat-tention,andrightlyso,foritfaroutperformsmostofitscompetitors.However,theCannyedgedetector,likemanyothers,hasaseriousdrawback,ifnotlimitation.Toachieveusableresults,theprocessofedgedetectionmustbepre-cededbytheapplicationofaGaussiansmoothing?lter,thestandarddeviationσofwhichseverelyaffectsthequalityoftheoutputimage.Morespeci?cally,itisknownthattheprocessofelimi-natingnoisewithoutalsoeliminatingcrucialimagedetailisadelicateone.Themoreanimageissmoothed,themoreits?nerdetailsblendintothebackgroundandfailtobepickedupbytheedgedetector.Thus,abalantraditionally,thisbalancehasbeenachievedbyvary-ingσuntiltheresultsoftheoperationaresatisfactory.1.2.StructureofthearticleInthenextsection,wepresentourmosaic-basedpanoramicstereosystem.Thegeometryofthesystem,theepipolargeometryinvolved,andtheprocedureofstereore-constructionareexplainedindetail.Furthermore,wede-scribethecontributionofourwork,andofferanoverviewofrelatedwork.Thefocusofthesectionisonanalyzingthesystem’scapabilities.Section3givesadetaileddescriptionofourmethodfordepth-assistededgedetectionvialayeredscale-basedsmoothing.Thismethodessentiallyimprovestheresultsofanexistingedgedetectionoperatorbyincorporatingintotheprocessthedepthdataofferedbythestereosystempre-sentedintheprevioussection.Section4givesexperimentalresultforbothsystemsanddescribestheirintegration.The?nalsectionofthearticleisusedtosummarizethemainconclusions.2.Panoramicstereosystem2.1.SystemhardwareInFigure1youcanseethehardwarepartofoursystem:arotationalarmisrotatingaapoleis?xedontherotationalarm,enablingtheoffsetoftheop-ticalcenterofthecamerafromthesystem’srotationalcen-onit,wehave?xedonestandardcolorcameralookingoutwardsfromthesystem’srotationalcenter.Panoramicimagesaregeneratedbymovingtherotationalarmbyananglecorrespondingtoonecolumnofthecapturedimage.2.2.RelatedworkOneofthebestknowncommercialpackagesforcreat-ingmosaicedpanoramicimagesisQTVR(QuickTimeVir-tualReality).Itworksontheprincipleofsewingtogetheranumberofstandardimagescapturedwhilerotatingthecamera[3].Pelegetal.[16]introducedthemethodforcre-ationofmosaicedpanoramicimagesfromstandardimagescapturedwithahandheldvideocamera.AsimilarmethodwassuggestedbySzeliskiandShum[20]whichalsodonotstrictlyconstrainthecamerapathbutassumethatthereisnogreatmotionparallaxeffectpresent.Allthemethodsmen-tionedsofarareusedonlyforvisualizationpurposessincetheauthorsdidnottrytoreconstructthescene.Ishiguroetal.[9]suggestedamethodwhichenablesthereconstructionofthescene.Theyusedastandardcamerarotatingonacircle.Thesceneisreconstructedbymeansofmosaicingtogetherpanoramicimagesfromthecentralcol-umnofthecapturedimagesandmovingthesystemtoan-otherlocationwherethetaskofmosaicingisrepeated.Twocreatedpanoramasarethenusedasinputinstereorecon-structionprocedure.Thedepthofanobjectwas?rstesti-matedusingprojectionsintwoimagescapturedondifferentlocationsofthecameraonthecamera’spath.Butsincetheirprimarygoalwastocreateaglobalmapoftheroom,theypreferredtomovethesystemattachedtotherobotabouttheroom.PelegandBen-Ezra[14,15]introducedamethodforthecreationofstereopanoramicimages.Stereopanoramasarecreatedwithoutactuallycomputingthe3Dstructure―thedeptheffectiscreatedintheviewer’sbrain.In[19],ShumandSzeliskidescribedtwomethodsusedforcreationofpanoramicdepthimages,whichareusingstandardproceduresforstereoreconstruction.Bothmeth-odsarebasedonmovingthecameraonacircularpath.Panoramasarebuiltbytakingonecolumnoutofthecap-turedimageandmosaicingthecolumns.Theycallsuchpanoramasmultiperspectivepanoramas.Thecrucialprop-ertyoftwoormoremultiperspectivepanoramasisthattheycapturetheinformationaboutthemotionparallaxef-fect,whilethecolumnsformingthepanoramasarecapturedfromdifferentperspectives.Theauthorsareusingsuchpanoramasastheinputforthestereoreconstructionpro-cedure.However,multiperspectivepanoramasarenotsomethingentirelyunknowntothevisioncommunity[19].Theyareaspecialcaseofmultiperspectivepanoramasforcellanimation[22],andareverysimilartoimagesgener-atedwithbythefollowingprocedures:multiple-center-of-projection[18],manifoldprojection[16]andcircularpro-jection[14,15].Theprincipleofconstructingthemultiper-spectivepanoramasisalsoverysimilartothelinearpushb-roomcameraprincipleforcreatingpanoramas[7].Inarticlesclosesttoourwork[9,19],wemissedtwothings:systemcapabilitiesanalysisandsearchingforcor-respondingpointsusingthestandardcorrelationtechnique.Thisiswhyinthisarticlewefocusonthesetwoissues.Whilein[9],theauthorssearchedforcorrespondingpointsbytrackingthefeaturefromthecolumnbuildingthe?rstpanoramatothecolumnbuildingthesecondpanorama,theauthorsin[19]usedanupgradedplainsweepstereoproce-dure.2.3.SystemgeometryThegeometryofoursystemforcreatingmultiperspec-tivepanoramicimagesisshowninFigure1.Whencreated,theyareusedasaninputtocreatepanoramicdepthimages.PointCdenotesthesystem’srotationalcenteraroundwhichthecameraisrotated.Theoffsetofthecamera’sopticalcen-terfromtherotationalcenterCisdenotedasr,describingtheradiusofthecircularpathofthecamera.Thecameraislookingoutwardsfromtherotationalcenter.TheopticalcenterofthecameraismarkedwithO.TheselectedcolumnofpixelsthatwillbesewninthepanoramicimagecontainstheprojectionofpointPonthescene.ThedistancefrompointPtopointCisthedepthlandthedistancefrompointPtopointOisdenotedbyd.θde?nestheanglebetweenthelinede?nedbypointCandpointOandthelinede?nedbypointCandpointP.Inthepanoramicimage,θgivesthehorizontalaxisdescribingthepathofthecamera.By?wedenotetheanglebetweenthelinede?nedbypointOandthemiddlecolumnofpixelsofthecapturedimageandthelinede?nedbypointOandselectedcolumnofpixelsthatwillbemosaicedinthepanoramicimage.Angle?canbethoughtofasareductionofthecamera’shorizontalviewangleα.Figure1.Systemhardwareandgeometryforconstructionofamultiperspectivepanorama.2.4.EpipolargeometryItcanbeshownthattheepipolargeometryisverysimpleifwearedoingthereconstructionbasedonasymmetricpairofstereopanoramicimages.Wegetasymmetricpairofstereopanoramicimageswhenwetakesymmetriccolumnsontheleftandontherightsidefromthecapturedimagecentercolumn.Sincewearelimitedwiththelengthofthearticlewewillendthissectionwiththefollowingstatement:epipolarlinesofthesymmetricalpairofpanoramicimagesareimagerows.Thisstatementistrueforoursystemgeometry.Forproofsee[8].2.5.StereoreconstructionLetusgobacktoFigure1.UsingtrigonometricrelationsevidentfromthesketchwecanwritetheequationfordepthestimationlofpointPonthescene.Usingthebasiclawofsinesfortriangles,wecanexpresstheequationfordepthestimationlas:l=r?sin(180o??)r?sin?sin(??θ)=sin(??θ).(1)Fromeq.(1)followsthatwecanestimatethedepthlonlyifweknowthreeparameters:r,?andθ.risgiven.Angle?canbecalculatedwithregardtothecamera’shori-zontalviewangleαas:2?=αW?W2?,(2)whereWisthewidthofthecapturedimageinpixelsandW2?isthewidthofthecapturedimagebetweencolumnsformingthesymmetricalpairofpanoramicimages,givenalsoinpixels.Tocalculatetheangleθwehaveto?ndcor-respondingpointsonpanoramicimages.Oursystemworksbymovingthecamerafortheanglecorrespondingtoonecolumnofcapturedimage.Ifwedenotethisanglewithθ0,wecanwriteangleθas:θ=dx?θ02,(3)wheredxistheabsolutevalueofdifferencebetweencorre-spondingpointsimagecoordinatesonhorizontalaxisxofthepanoramicimages.Weareusingaprocedurecalled“normalizedcorrela-tion”tosearchforcorrespondingpoints.Toincreasethecon?denceinestimateddepthweareusingaprocedurecalled“back-correlation”[6].Withtheback-correlationwearealsosolvingtheproblemofocclusions.2.6.Systemcapabilitiesanalysis2.6.1.ConstrainingthesearchspaceontheepipolarlineKnowingthatthewidthofthepanoramicimageismuchbiggerthanthewidthofthecapturedimage,wewouldhavetosearchforcorrespondingpointsalongaverylongepipo-larline.Thatiswhywewouldreallyliketoconstrainthesearchspaceontheepipolarlineasmuchaspossible.Asideeffectisalsoanincreasedcon?denceinestimateddepthandafasterexecutionofthestereoreconstructionprocedure.Ifwederivefromeq.(1),wecanascertaintwothingswhichnicelyconstrainthesearchspace:(1.)Ifθ0presentstheangleforwhichthecameraisshifted,then2θmin=θ0.Thismeansthatwehavetomakeatleastonebasicshiftofthecameratogetascenepointprojectedinarightcol-umnofthecapturedimageformingthelefteyepanorama,tobeseenintheleftcolumnofthecapturedimageformingtherighteyepanorama.(2.)Theoretically,theestimationofdepthisnotconstrainedupwards,butfromeq.(1),itisevidentthatthedenominatormustbenon-zero.Wecanwritethisfactas:θmax=n?θ20,wheren=?divθwillshow20and?modθ20=0.Inthefollowingsectionswethatwecannottrustthedepthestimatesnearthelastpointoftheepipolarlinesearchspace,butwehaveproventhatwecaneffectivelyconstrainthesearchspace.Toillustratetheuseofthespeci?edconstraintsonrealdata,letuswritethefollowingexamplewhichdescribestheworkingprocessofoursystem:whilethewidthofthepanoramais1501pixels,wehavetocheckonlyn=149pixelsinacaseof2?=29.9625oandonlyn=18inthecaseof2?=3.6125o,whensearchingforacorrespondingpoint.Fromthelastparagraphwecouldconcludethatthestereoreconstructionprocedureismuchfasterforasmallerangle?.Butwewillshowinthenextsectionthatasmallerangle?does,unfortunately,alsohaveanegativeproperty.2.6.2.Themeaningoftheerrorforapixelintheestima-tionofangleθa)2?=29.9625ob)2?=3.6125oFigure2.Graphsshowingdependenceofdepthfunctionlfromangleθwhileradiusr=30cmandusingdifferentvaluesofan-gle?.Toeasethecomparisonoftheerrorforapixelinestimationofangleθweshowedtheintervalofwidthθverticallinesaroundthe20=0.1obetweenthethirdpoint.Beforeweillustratethemeaningoftheerrorforapixelintheestimationofangleθ,letustakealookatthegraphsinFigure2.Thesegraphsareshowingadependenceofthedepthfunctionlfromangleθwhileusingdifferentvaluesofangle?.Fromthegraphsitisevidentthatthedepthfunc-tionlisrisingslowerinthecaseofabiggerangle?.Thispropertydecreasestheerrorindepthestimationlwhenus-ingabiggerangle?,butthisdecreaseintheerrorbecomesevenmoreevidentifweknowthatthehorizontalaxisisdiscreteandtheintervalsontheaxisareθ0degreeswide(seeFigure2).Ifwecomparethewidthof2theintervalonbothgraphswithrespecttothewidthoftheintervalthatθisde?nedonθ(θ∈[0,?]),wecanseethattheintervalwhosewidthis20degrees,ismuchsmallerwhenusingabiggerangle?.Thissubsequentlymeansthattheerrorforapixelintheestimationofangleθismuchsmallerwhenusingabiggerangle?,sinceashiftforangleθ0describestheshiftofthecameraforonecolumnofpixels.Becauseofdiscretehorizontalaxisθ(Figure2)within-tervals,whichareθ20degreeswide(inourcaseθ0=0.2o),thenumberofpossibledepthestimationvaluesispropor-tionaltoangle?:wecancalculate(?divθ0ifweareusingtheangle2=)149differ-entdepthvalues2?=29.9625oandonly18differentdepthvaluesifweareusingtheangle2?=3.6125o.Andthisisthenegativepropertyofusingasmallerangle?.FromthegraphsshownonFigure2wecanconcludethattheerrorismuchbiggerincaseofasmallerangle?thaninthecaseofabiggerangle?.Thesecondconclusionisthatthevalueoftheerrorisgettingbiggerwhenthevalueoftheangleθisgettingclosertothevalueoftheangle?.Thisistrueregardlessofthevalueoftheangle?.Thespeedofthereconstructionprocessisinverselyproportionaltotheaccuracyoftheprocess.Byvaryingtheparametersθ0andrwearechangingthesizeoftheerror.2.6.3.De?nitionofmaximaldepthinwhichwetrustInsection2.6.1wede?nedtheminimalpossibledepthes-timationlminandmaximalpossibledepthestimationlmax,butwedidnotwriteanythingaboutthemeaningoftheerrorforapixelinestimationofangleθforthesetwoestimateddepths.Letusexaminethesizeoftheerror?lforthesetwoestimateddepths:wecalculate?lminastheabsolutevalueofdifferencebetweenthedepthlminandthedepthlforwhichtheangleθisbiggerthanangleθminforangleθ20:?lθmin=|lmin(θmin)?l(θmin+02)|=|lθmin(02)?l(θ0)|.Similarly,wecalculatetheerror?lmaxastheabsolutevalueofdifferencebetweenthedepthlmaxandthedepthlforwhichtheangleθissmallerthanangleθmaxforangleθ20:?lmax=|lmax(θmax)?l(θmax?θ0=|l2)|=max(nθ0?l((n?1)θ2)20)|,wherebythevariablenwedenoteapositivenumberfol-lowingfromtheequation:n=?divθ20.2?=29..6125o?lmin2mm19mm?lmax3mmTable1.Themeaningoftheerror(?l)foronepixelintheestimationofangleθforlminandlmaxregardingtheangle?.InTable1wegatheredtheerrorsizesfordifferentval-uesofangle?.Theresultscon?rmstatementsinSection2.6.2.Wecanalsoaddtwoadditionalconclusions:(1.)Thevalueoftheerror?lmaxisunacceptablyhighandthisistrueregardlessofthevalueofangle?.Thisiswhywehavetosensiblydecreasethemaximalpossibledepthestimationlmax.Thisconclusioninpracticeleadsustode?netheup-perboundaryofallowederrorsize(?l)foronepixelintheestimationofangleθandwithit,wesubsequentlyde?nethemaximaldepthinwhichwetrust.(2.)Angle?alwaysdependsuponthehorizontalviewangleαofthecamera(equation(2)).Andwhiletheangleαislimitedtoaround40oconsideringstandardcameras,oursystemislimitedtoanangleαwhenestimatingthedepth,sinceinthebestcasewehave:?max=α2.Thusoursystemcanreallybeusedonlyforthereconstructionofsmallrooms.3.Depth-assistededgedetection3.1.Scale,sizeandlossofdetailTheoretically,itcouldbearguedthattheapplicationofatwo-dimensional?lterliketheGaussiantoanimagerep-resentingathree-dimensionalsceneheraldsimplicitprob-lems.Whenathree-dimensionalsceneiscapturedusingacamera,itislogicaltoassumethattheavailabledetailthattheresultingimagecanyieldatanypointdependspartiallyonthatpoint’soriginaldistancefromthelens.Thefartherthefeature,thesmalleritsrepresentationintheimagewillbe(scale).Thisdifferenceinscalewillmakeitlesslikelythatadistantfeatureweareinterestedinwillretainallofitsdetailafterbeingsampledandquantized.OnecanthenseethattheapplicationofauniformGaus-sian?lterwithaconstantstandarddeviationσwillcauseadisproportionatelossofdetailacrosstheimage,essen-tiallyresultinginlowoperatorperformanceintheareasofthepicturefarthestfromthelens.Thisismoreevidentinoutdoorandsyntheticscenes,wheredepthcanvarygreatlyfrompointtopoint.Thislossofqualitymaybecounteredifthe?lter’sscalecanbevariedaccordingtoitspointofapplicationintheimage.Anintuitivesolutionhere,then,wouldbetoestablisharelationbetweenthe?lter’sscale(controlledbyσ),andthereal-worldsizeoftheobjectbeingsmoothed.Inordertoachievethiskindofadjustmentatthelevelofprocessingthatprecedesedgedetection,onewouldneedtosomehowbeawareoftherelativesizeofobjectsateverypartoftheimage.Itisknown,however,thatthereisadirectrelation-shipbetweenthesize(inpixels)oftherepresentationofanobjectinanimageandthatobject’soriginal,real-worlddis-tancetothecamera’slens.Assuming,therefore,theavailabilityofarobuststereosystem,wecancomputethedistanceatanypointintheim-age,thusestablishingavariablethatwillallowustoadjusttheGaussian’sσaccordingly.3.2.AlgorithmoverviewThealgorithmthatdealswiththeproblemsdescribedaboveutilizespre-computedpixeldistanceinformationinordertosegmenttheimageintoseveraldifferentlayers.Inthiscapacity,thealgorithmcanbeconsideredtobedepth-adaptive.Theselayersarethensmoothedindependentlyandarerecombinedbeforebeingfedtoanedge-detector.Theentireprocessconsistsof:(1.)Layering―Theim-ageissegmentedintodiscretelayersbasedonpixeldis-tancedataandauser-suppliedparameterthatdenotesthedesirednumberoflayerstobeproduced.(2.)PiecewiseSmoothing―Theresultingsub-imagesaresmoothedin-dependently,utilizinga?lterscaleappropriatetotherele-vantdepthofthelayerintheimage.(3.)Compositing―Thesmoothedsub-imagesarecombinedintooneimageagain.Thatimagewillbetheinputfortheedge-detector.(4.)Edge-detection―Thecompositedimageisfedintoanedge-detector.3.3.LayeringThe?rststepintherealizationofthisalgorithmistosplitagivenimageintomultiplelayers.Eachlayercorre-spondstoacertainrangeofdistancesandconsistsonlyofimagepixelswithrelevantdistancecharacteristics.Thepix-elsthatmakeupeachlayeraresubsequentlysupplementedbyarti?ciallybackground-valuedpixelsinordertoformasub-imagethatisreadyforsmoothing.Thealgorithmdoesnotcurrentlyimposeanycontinuityconstraints,anditisnotuncommonforalayertocontainnon-neighboringpixels.Notethatmosttrialrunswereperformedusingdensedepthmaps.Toaccommodatedifferentmeasurementschemes,thealgorithmdetectstheminimumandmaximumdepthvaluesandnormalizesallpixeldistancestoarelativepercentage.Thepixel(s)withthesmallestdistancetothecameraaregivenavalueof0percent,whilethosefarthestfromthecameraaregivenavalueof100percent.三亿文库包含各类专业文献、幼儿教育、小学教育、中学教育、文学作品欣赏、应用写作文书、生活休闲娱乐、高等教育、Panoramic stereo vision and depth-assisted edge detection92等内容。　
　panoramic camera, panorama camera 011 框幅摄影机 ...stereocopic vision 164 正立体 orthostereoscopy 165...edge enhancement 437 边缘检测 edge detection 438 ...