During the summer of 2014 I began investigating image stitching techniques and technologies for a NOAA sponsored UAS marine survey project.  In the summer of 2015 I was hired by the University of Minnesota Department of Aerospace Engineering and Mechanics to work on a Precision Agriculture project that also involves UAS’s and aerial image stitching.

Over the past few months I have developed a functional open-source image stitching pipeline written in python and opencv.  It is my intention with this series of blog postings to introduce this work and further explain our approach to aerial image processing and stitching.

Any software development project is a journey of discovery and education, so I would love to hear your thoughts, feedback, and questions in the comments area of any of these posts.  The python code described here will be released under the MIT open-source software license (one of my to-do list items is to publish this project code, so that will happen “soon.”)


The world already has several high quality commercial image stitching tools as well as several cloud based systems that are free to use.  Why develop yet another image stitching pipeline?  There are several reasons we began putting this software tool chain together.

  • We are looking at the usefulness and benefits of ‘direct georeferencing.’  If we have accurate camera pose information (time, location, and camera orientation of each image), then how can this improve the image analysis, feature detection, feature matching, stitching, and rendering process?
  • One of the the core strengths of the UMN Aerospace Engineering Department is a high quality 15-state kalman filter attitude determination system.  This system uses inertial sensors (gyros and accelerometers) in combination with a GPS to accurately estimate an aircraft’s ‘true’ orientation and position.  Our department is uniquely positioned to provide a high quality camera pose estimate and thus examine ‘direct georeferencing’ based image processing.
  • Commercial software and closed source cloud solutions do not enable the research community to easily ask questions and test ideas and theories.
  • We hope to quantify the sensor quality required to perform useful direct georeferencing as well as the various sources of uncertainty that can influence the results.
  • We would love to involve the larger community in our work, and we anticipate there will be some wider interest in free/open image processing and stitching tools that anyone can modify and run on their own computer.


I will be posting new tutorials in this series as they are written.  Here is a quick look ahead at what topics I plan to cover:

  • Direct Georeferencing
  • Image stitching basics
  • Introduction to the open-source software tool chain
  • Aircraft vs. camera poses and directly visualizing your georeferenced data set
  • Feature detection
  • Feature matching
  • Sparse Bundle Adjustment
  • Seamless 3D and 2D image mosaics, DEM’s, Triangle meshes, etc.

Throughout the image collection and image stitching process there is art, science, engineering, math, software, hardware, aircraft, skill, and a maybe bit of luck once in a while (!) that all come together in order to produce a successful aerial imaging result.

Software Download

The software referenced in this tutorial series is licensed with the MIT license and available on the University of Minnestoa UAV Lab public github page under the ImageAnalysis repository.


Join the Conversation


Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

  1. Hi Pulkeshian,

    Just my 2 cents here:
    There are commercial software products like pix4d and agisoft that are very high quality and should achieve most of your objectives plus a lot more.
    There is also an open-source solution called Open Drone Map (which is a collection of open-source tools made to work on ubuntu.) ODM probably isn’t as polished as the high end commercial tools, but you can get useful orthophotos and 3d models out of it.
    I believe the cost to develop your own solution from scratch (or even adapt existing work) will far exceed the cost of an existing commercial solution.
    All that said, I think you would need some other objective that these solution do not provide, or some other use case they don’t handle to justify your own development effort.
    Personally, I work full time for the University of Minnesota, so I don’t have hours in the day to pursue independent projects, but all the work I’ve invested into my own image stitching scripts is freely available for examination or reuse.

  2. Dear Sir,

    We are writing you from the desk of Tata Group Technology and Innovation Office. We are looking for a solution for a challenge related to Stitching images captured from drone. The complete problem statement is mentioned below –

    Drones are used to capture images of factory premise for surveillance and record keeping purposes. To capture the images the Drone is flown over the area of interest at an elevation of about 30 to 40 meters. The area is divided into several sectors depending on size of the area. Typically, the sector is divided into 8 to 12 sectors. The Drone flies to the specified location to capture the images. The location in a particular sector, where the image is to be captured and the path to be traversed is determined by a program for autonomous flights. The images can be captured using manual mode as well. All the images captured by Drone are tagged a Geo coordinates along with elevation and inclination data of the point at which the image is captured. Each image is of 4 – 5 MB in size. They are stored in JPEG format.

    This way a lot of Geo referenced images are captured wherein each picture captures only a part of the whole area. To get the image of the entire area these segregated images must be stitched together. While stitching it is to be made sure proper images are put next to each other depending on their Geo reference coordinates.

    We are looking for solutions that will enable us to stitch such images (that we have) to get a holistic 2D images of the area.

    Success Criteria

    Should be able to handle various images formats to produce one holistic 2D image
    Should not be resource intensive. It should run on commercially available laptops
    It will be an added advantage if it can also produce a 3D reconstruction of the area of interest using the same images
    Please suggest if you could help us in solving this challenge.

    Thanks & Regards,
    Pulkeshian Daruka
    Analyst – Open Innovation
    Group Technology and Innovation Office

    Tata Sons Limited
    TMTC Campus | 1 Mangaldas Road |
    Pune 411001 | Tel +91 20 6609 1027 | Mobile +91 9921020630