Controlling a quad with a wave of your hand

So I just recently purchased a Leap Controller.  For those of you not familiar with it, this is an amazing device that can track your fingers and hands extremely precisely.  So of course, the first thing I wanted to do was use it to control a quadcopter!

The API is actually fairly straightforward to interface to C++ and you can access it here.  I wrote a plugin for Tau Labs GCS which would access the hand position and the roll-pitch-yaw value of the palm of hand. Right now I'm not doing anything with the position (other than checking a hand is there) and the RPY value is packaged into a LeapControl UAVO and relayed via the telemetry link to the FlightController.

On the flight controller side, I extended the ManualControl module to support another flight mode position (LeapControl). When the flight mode switch is in that position, then the LeapControl UAVO is used to set the desired attitude (yaw is used to set the turning rate, not absolute heading) and the throttle from the transmitter is used.  If a hand is missing it turns off the motors (a nice safety feature you'll see in the video).

The code changes can be found here although check the commit history since it requires some hardcoded paths to the Leap libraries.

The flight platform was my IconicX, which is beautifully light, has good flight times and only uses 8 inch props.  All really good things for indoor testing of a brand new and probably terribly advised control scheme.  It used bluetooth for telemetry to get the LeapControl information from GCS (more on that later).

The flight controller is the Sparky board that I designed for Tau Labs:

But of course, you really just want to see the video, so without further ado:

Controlling quad with a wave of the hand from James Cotton on Vimeo.

Conclusions

It's definitely flyable. Especially after a few minutes practice (I'd recommend a larger space).  A few things I need to change or would like to address in the future

• Increase the yaw sensitivity.  Right now the highest value is only a few degrees per second. However, it's a bit tricky because the Leap seems to zero yaw at the something random (related to what it first saw) and I'm finding "zero" seems a bit arbitrary.  Probably a dead band and some exponential on the GCS side would go a long way.
• Lag. This is the biggest blocker I think.  This quad is well tuned and reacts _really_ fast and so I can tell the limitations from hand controller. Of course the Leap itself has some latency, but I think something is going on in terms of the bluetooth serial port backing up data.  When I set the update rate too high there is a very clear backlog that occurs. Using a PipX might help since we have a bit more control about the lower level buffers.  Probably the serial code needs a goo review and for things like this we need a low latency protocol for sending the same object. I did something similar for the Tau Labs android app I wrote which made a huge difference for telemetry control (also bluetooth).
• Use hand position - I feel like tracking the position could allow a stronger communication of "oh shit" when getting near something.  Possibly if the hand moves rapidly then it applies a short lived roll signal to the quad to pull it away from the object but avoid the pilot induced oscillations from the latency.  And of course the Z position can also be used for ...
• Integrate this with the sonar based altitude hold (after some improvement) I described last week so that the hand height directly indicated the altitude.  That could create a really cool control scheme.
• Using overo on Freedom for optical flow then the X and Y position can actually indicate a rate to move at.  That would be really awesome.
• Try this outdoors.  I'm not sure how well the Leap will work there.

But it was fun.  I definitely won't give up my transmitter this week though.  If you are interested in this or our other developments, make sure to drop by Tau Labs and say hi.

Tau Labs Sonar altitude hold

So following up on Stac and scenkov's really nice work I added support for SMD-IO-UART sonar module which I had laying around and connected it to Sparky. This one is a bit different than the HCSR04 which encodes the range in the duration of a positive digital pulse. Instead, it had a negative pulse with a duration of 150µs when it finds an obstacle and the latency from the trigger to the pulse indicates the distance.  For more details check this out.

I mounted the module on the bottom of my silver hornet, which is a bit too big for the landing gear but works for now:

With the module working, I hacked up some code to pass that data into the altitude hold EKF.  I played around with the tuning a bit and can't get to the point where I'm super happy with it.  The next step will be trying a different module (e.g. I have an XL-MaxSonar sitting on my desk right now which people on IRC say good things about).  However it does work.

Sonar altitude hold from James Cotton on Vimeo.

One thing that is a problem with the current implementation is that if you go too high and the sonar goes out of range, then the EKF no longer has anything to correct the altitude. This means it goes into a straight prediction mode based on integrating the accels but depending on the direction of the bias this can make it think it is going down and correct by going up.

I suppose an alternative might be to keep correcting with the previous altitude (or max for that sonar) which should in any case be higher than what you engaged it at and cause it to come back down. A better approach would be to fill in that missing information with the baro.

If anyone wants to play with it, the code is currently at my Tau Labs github fork.