Installation notes for Kinefly

Today I'm going to set up a computer to track average wing stroke envelopes of flies. This is useful because asymmetries in wing stroke amplitude indicate attempted turns by tethered flies, so measuring them gives us a quantitative metric of flight behavior. I'm going to be installing Steve Safarik's Kinefly, which is a ROS-based replacement for Andrew Straw's Strokelitude, both of which were developed in the Dickinson Lab. (Strokelitude is rapidly becoming obsolete.) Here is a screenshot of what Kinefly looks like:

Screenshot of wing amplitude tracking using Kinefly

I'll mostly follow Steve's instructions here, but make notes of any differences. To start off with, I installed 64-bit Ubuntu 14.04 LTS on a new hard drive.

Departing from Steve's instructions, I'll first install ROS Indigo, which is a newer version of ROS and will hopefully be supported for longer. I'll follow the directions here.

I'll be using a firewire (IEEE1394) camera in my rig, not a gigE (Ethernet) camera, so I'll skip the Camera Aravis installation (lines 29-35 of Steve's installation instructions). I also skipped lines 50-53. Instead I installed the ROS drivers for 1394 cameras:

cd catkin/src/
git clone https://github.com/ros-drivers/camera1394.git
cd ~/catkin
catkin_make

Now I'm going to download and install coriander to test that the camera is working:

sudo apt-get install coriander
Then I plugged the camera into the computer and ran
coriander
Under the Services tab, click Display and you should see a live image from the camera.

In line 90, I copied the 'polarization' rig instead of 'thadsrig' because that rig used a 1394 camera:
cp -R polarization yourrigname

We are almost done! Now just three lines of code that need to be changed because ROS indigo uses opencv2 instead of an earlier version (I think):

gedit ~/src.git/Kinefly/nodes/kinefly.py
and edit line 240 to be:
rosimg = self.cvbridge.cv2_to_imgmsg(imgInitial, 'passthrough') # used to be img = np.uint8(cv.GetMat(self.cvbridge.imgmsg_to_cv(rosimg, 'passthrough')))
line 619 to be
img = np.array(self.cvbridge.imgmsg_to_cv2(rosimg, 'passthrough')) # used to be img = np.uint8(cv.GetMat(self.cvbridge.imgmsg_to_cv(rosimg, 'passthrough')))
and line 769 to be
rosimgOutput = self.cvbridge.cv2_to_imgmsg(imgOutput, 'passthrough') # used to be rosimgOutput = self.cvbridge.cv_to_imgmsg(cv.fromarray(imgOutput), 'passthrough')


If you want to test Kinefly without the LED panels attached, you can edit the _main.launch file: 
gedit ~/src.git/Kinefly/launch/yourrigname/_main.launch 
Comment out the following three lines:
<include file= "$(find Kinefly)/launch/$(env RIG)/params_ledpanels.launch" ns="kinefly" />
<node name="ledpanels" pkg="ledpanels" type="ledpanels.py" />
<node name="flystate2ledpanels" pkg="Kinefly" type="flystate2ledpanels.py" ns="kinefly"  />
by inserting   <!-- before each and --> after each. Now test Kinefly by running this command:
roslaunch Kinefly main.launch
Don't forget to click the 'exit' button to close Kinefly--the red x doesn't work!

Plotting strokelitude wing tracking data on fmf frames

I have been working on a Python script to save movies in which strokelitude wingstroke tracking data are plotted on original frames from an .fmf movie. My current version is below.

Load strokelitude data from .bag file into Python

Here is a script I've been using to load the strokelitude data contained in a .bag file. It should be easy to convert it to load data from other ros topics.

Installing fview and Strokelitude from Scratch

Install Ubuntu

This step is not stictly necessary, but everything works in Lucid, so it may save time.

• Download and install Ubuntu 10.04 (Lucid) from http://www.ubuntu.com/download/ubuntu/download
• Often the display will be improved by looking under System>Administration>Hardware Drivers for video card drivers.
• Run the Update Manager (under System>Administration).

Install fview

Andrew Straw wrote fview, a general purpose video acquistion application. Strokelitude (also by Dr. Straw) is a plugin for fview.

• Add the repositories, following the instructions at http://code.astraw.com/projects/motmot/download.html

  sudo add-apt-repository ppa:astraw/ppa
  sudo wget --output-document=/etc/apt/sources.list.d/astraw.list http://debs.astraw.com/sources.list.d/astraw-$(lsb_release -cs).list
  sudo apt-get update && sudo apt-get install astraw-keyring && sudo apt-get update
  

• From synaptic (System>Administration>Synaptic Package Manager) install the following packages with all their dependencies:

  • camiface
  • python-motmot-fview
  • libhdf5-serial-1.8.4
  • hdf5-tools
  • python-tables
  • python-enthought-traits
  • python-enthought-traits-ui
  • python-chaco
  • python-pylibusb
  • pyro
  • python-motmot-fastimage
  • python-motmot-realtimeimageanalysis
  • python-motmot-fviewexttrig
  • git-core

• Test fview by typing fview at a terminal (Applications>Accessories>Terminal) or going to Applications>Sound & Video>fview don't worry about error message:

  While attempting to open the plugin "fview_ext_trig",
  FView encountered an error. The error is:
  
  Cannot find device.
  (Perhaps run with environment
  variable REQUIRE_TRIGGER=0.)
  

  Initialize the camera by going to Camera>initialize camera...

• Note: If you are using a firewire (1394) camera and you cannot initialize it, try running sudo fview instead. Also, run the following command in a terminal:

  sudo adduser $USER video
  sudo chown .video /dev/raw1394
  

  then log out and log back in.

Install strokelitude

Now we are ready to install the plugin, "strokelitude," that actually tracks the wing positions.

• In a terminal, type:

  mkdir src
  cd src
  git clone git://github.com/astraw/remote_traits.git
  cd remote_traits/
  sudo python setup.py install
  cd ..
  git clone git://github.com/motmot/strokelitude.git
  cd strokelitude/
  sudo python setup.py install
  

• Start fview, you should now see strokelitude as an option under Windows.

Install ROS (Electric)

The easiest way to save strokelitude data is to install ROS (Robot Operating System) and save the "topics" that strokelitude "publishes" in a .bag file. In order to do this, we install ROS, then start roscor before opening fview. We record using the rosbag record command.

• Follow the instructions at http://www.ros.org/wiki/electric/Installation/Ubuntu . Briefly, you need to do the following. In a terminal, type:

  sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu lucid main" > /etc/apt/sources.list.d/ros-latest.list'
  wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
  sudo apt-get update
  sudo apt-get install ros-electric-desktop-full
  echo "source /opt/ros/electric/setup.bash" >> ~/.bashrc
  . ~/.bashrc
  

  The fourth line will take some time -- you are installing all of ROS.

• Now we'll need to create a directory in which we can place ROS scripts (see http://www.ros.org/wiki/ROS/Tutorials/InstallingandConfiguringROSEnvironment ). In a terminal, type

  cd ~/src
  mkdir ros_workspace
  cd ros_workspace
  

• We want to tell ROS that this directory will contain ROS scripts. Create a file named setup.sh:

  #!/bin/sh
  source /opt/ros/electric/setup.bash
  export ROS_ROOT=/opt/ros/electric/ros
  export PATH=$ROS_ROOT/bin:$PATH
  export PYTHONPATH=$ROS_ROOT/core/roslib/src:$PYTHONPATH
  export ROS_PACKAGE_PATH=~/src/ros_workspace:/opt/ros/electric/stacks:$ROS_PACKAGE_PATH
  

  then run the file using the command . setup.sh

  To make this change permanent, assuming the setup.sh file is in your home directory, add to the bottom of your .bashrc file:

  source  ~/src/ros_workspace/setup.sh
  

• Now we are ready to install the package that defines the message type strokelitude will publish in ROS. In a terminal type:

  git clone git://github.com/motmot/strokelitude_ros.git
  rospack profile
  rosmake strokelitude_ros
  rosdep install rxtools
  rosmake roscpp_tutorials rospy_tutorials rxtools
  roscore
  

  then open fview by typing fview in a new terminal. Open the strokelitude window and click box next to 'Processing enabled'

• In a new terminal type rxplot /strokelitude/left_wing_angle_radians /strokelitude/right_wing_angle_radians in order to view a realtime plot of the fview data

• In new terminal, navigate to the directory where you want to record data, then type rosbag record -O data /strokelitude (this will save data in data.bag file)

• To import your data into Python, do something like the following in a python script:

  import roslib
  roslib.load_manifest('rosbag') import rosbag
  bag = rosbag.Bag('data.bag')
  
  leftWingRad=list()
  rightWingRad=list()
  for topic, msg, t in bag.read_messages():
      leftWingRad.append(msg.left_wing_angle_radians)
      rightWingRad.append(msg.right_wing_angle_radians)
  

Install strokelitude-jfi-emulator

If you want to do closed-loop experiments using the Reiser panels, probably the simplest way is to use the mcc analog output board to output an analog voltage proportional to each wingstroke. In order to install this, follow these steps:

• Using synaptic package manager, install libmcclibhid

• Plug in the usb1208FS (this used to be called pmd1208FS and those work also) and test it by running the command sudo testusb1208FS in a terminal. Type 'b' to blink the LED.

• Using synaptic, install python-wjul. This allows Python access to the usb1208FS.

• install strokelitude-jfi-emulater by typing the following into a terminal:

  cd src
  git clone git://github.com/motmot/strokelitude-jfi-emulator.git
  cd strokelitude-jfi-emulator
  sudo python setup.py install
  

test strokelitude and ROS

This is useful: rxplot /strokelitude/left_wing_angle_radians