Howto: Action Cam HUD overlay

NOTICE: This is a draft document and in the process of being written.  It is incomplete and subject to change at any time without notice.  

In this post I share my process and tools for creating HUD overlays on a flight video.  Here is a quick overview of the process: (1) Calibrate your camera, (2) Extract the roll rate information from the flight video, (3) Automatically and perfectly time correlate the video with the flight data, (4) Render a new video with the HUD overlay.

Source Code

Please be aware that this code has not yet gone through a v1.0 …
Read the rest... >>

Autopilot Visualization

Blending real video with synthetic data yields a powerful and cool! way to visualize your kalman filter (attitude estimate) as well as your autopilot flight controller.

screenshot-from-2016-11-30-14-31-14

Conformal HUD Elements

Conformal definition: of, relating to, or noting a map or transformation in which angles and scale are preserved.  For a HUD, this means the synthetic element is drawn in a way that visually aligns with the real world.  For example: the horizon line is conformal if it aligns with the real horizon line in the video.

  • Horizon line annotated with compass points.
  • Pitch ladder.
  • Location of nearby airports.
  • Location of sun,

Read the rest... >>

Mistakes!

dj-mistakes-homer-dohmistake-pano_13891

I make thousands of mistakes a day, mistakes typing, mistakes coding software, mistakes driving, mistakes walking, forgetting to order my sandwich without mayo, etc.  Most of the time they are immediately obvious — a red squiggly line under a word I mistyped, a compiler spewing an error message on line #42, a stubbed toe, my gps suggesting a u-turn at the next intersection, etc.

mistakes

But what happens when the mistake isn’t obvious, isn’t noticed immediately, and doesn’t cause everything around me to immediately fail?  Often these mistakes can have a long lifespan.  Often we discover them when we are looking …
Read the rest... >>

Automated Movie Frame Extracting and Geotagging

This is a short tutorial on an automated method to extract and geotag movie frames.  One specific use case is that you have just flown a survey with your quad copter using a 2-axis gimbal pointing straight down, and a gopro action cam in movie mode.  Now you’d like to create a stitched map from your data using tools like pix4d or agisoft.

The most interesting part of this article is the method I have developed to correlate the frame timing of a movie with the aircraft’s flight data log.  This correlation process yields a result such that for any …
Read the rest... >>

Image Stitching Tutorial Part #2: Direct Georeferencing

What is Direct Georeferencing?

uav-for-PA_01-image-collection

The process of mapping a pixel in an image to a real world latitude, longitude, and altitude is called georeferencing.  When UAS flight data is tightly integrated with the camera and imagery data, the aerial imagery can be directly placed and oriented on a map.   Any image feature can be directly located.  All of this can be done without needing to feature detect, feature match, and image stitch.  The promise of “direct georeferencing” is the ability to provide useful maps and actionable data immediately after the conclusion of a flight.

Typically a collection of overlapping images …
Read the rest... >>

Image Stitching Tutorial Part #1: Introduction

features

Background

During the summer of 2014 I began investigating image stitching techniques and technologies for a NOAA sponsored UAS marine survey project.  In the summer of 2015 I was hired by the University of Minnesota Department of Aerospace Engineering and Mechanics to work on a Precision Agriculture project that also involves UAS’s and aerial image stitching.

Over the past few months I have developed a functional open-source image stitching pipeline written in python and opencv.  It is my intention with this series of blog postings to introduce this work and further explain our approach to aerial image processing and stitching.…
Read the rest... >>

Flight Milestones

Congratulations!

Congrats ATI Resolution 3, Hobby Lobby Senior Telemaster, Hobbyking Skywalker, and Avior-lite autopilot on your recent milestones!

IMG_20130923_183033 IMG_20150804_073522 IMG_20150804_093541 IMG_20150801_135249

Avior-lite (beaglebone + apm2 hybrid) autopilot:

  • 300th logged flight
  • 7000+ logged flight minutes (117.8 hours)
  • 6400+ fully autonomous flight minutes (107.2 hours)
  • 2895 nautical miles flown (3332 miles, 5362 km)

Hobby Lobby Senior Telemaster (8′ wing span)

  • Actively flight testing autopilot hardware and software changes since 2007!
  • 200th logged flight.
  • 5013 logged flight minutes (83.5 hours)
  • 4724 fully autonomous flight minutes (78.7 hours)
  • 2015 nautical miles flown (2319 miles, 3733 km)

Today (October 7, 2015) I logged the 300th avior-lite flight and …
Read the rest... >>

APM2 Sensor Head

apm2

The ardupilot mega is a fairly capable complete autopilot from both the hardware and the software perspective.  But what if you projects needs all the sensors and not the full APM2 autopilot code?

Overview

The apm2-sensorhead project provides a quick, robust, and inexpensive way to add a full suite of inertial and position sensors to your larger robotics project.  This project is a replacement firmware for the ardupilot-mega hardware.  The stock arduplane firmware has been stripped down to just include the library code that interrogates the connected sensors, and also maintains the code that can read your RC transmitter stick …
Read the rest... >>

PBY Catalina

January 27, 2015

All shiny again!

IMG_20150127_174222 IMG_20150127_174150 IMG_20150127_174100

January 27, 2015

Today my replacement plastic parts arrived … a new canopy and a new hull shield.  It wasn’t too hard to pry the old shield off, and the new part is already to slap on.

IMG_20150127_112053

January 26, 2015

IMG_20150126_102541

Here is a quick snapshot showing the new orange color on the engine cowls, the upper side wing tips, and also notice the tail stripes have been repainted orange (originally red.)  I’m just waiting on a replacement canopy which will hopefully arrive tomorrow and I should be able to completeley reassemble the model and …
Read the rest... >>

Sonic 64

Hobbyking Sonic 64

This is 1230mm (48.4″) wing span flying wing.  It is powered by a 64mm electric ducted fan.  It is a simple build and flies great!

Here are a couple pictures before the maiden flight (temperature was about +18F, winds were calm, skies had a medium thin overcast.)

IMG_20150110_143003  IMG_20150110_143017 IMG_20150110_143010

My one gripe about the Sonic 64

Of course we buy airplanes like the Sonic 64 because they are inexpensive, quick to assemble, and fast to get out to the field and up in the air.  It doesn’t hurt that they look great and are a lot of fun to …
Read the rest... >>