Matplotlib Animations, Using Stored Data, Writing Video

There are some good matplotlib animations examples floating around the internet, including here and here. These were useful and I gained a great deal from them, but my particular problem was not directly addressed.

The scenario we're looking at today is the following. We have some stored data on disk, that we would like to plot using a Matplotlib animation, and save this animation as a video file. This is a reasonably common occurrence for researchers, but I had trouble finding tutorials online for this problem. Many of the tutorials generated their data via simulations on the fly, which didn't directly map to the case of recorded data.

There were two stumbling blocks I ran into, that had non-obvious solutions. First, Matplotlib's animation takes a function pointer to a user-defined function, for example def animate(i), but the animate function is then controlled by the iterator i. The second major stumbling block I hit, is that your animation function def animate(i): should return the data structures that need to be updated.

In this example, I have saved some data to a file in JSON format; each entry has a position and a velocity parameter. I read this file in line-by-line, and create a scrolling plot of the time series. I save this animated scrolling plot as a video file to disk, using ffmpeg and x264 for video encoding.

 import sys  
 import os  
 import json  
 import numpy as np  
 import matplotlib  
 import matplotlib.pyplot as plt  
 import matplotlib.animation as animation  
 from time import time  
 import itertools  
   
 from matplotlib.path import Path  
 import matplotlib.patches as patches  
 import threading  
   
   
   
 def init():  
   plotline.set_data([], [])  
   dplotline.set_data([],[])  
   return plotline,dplotline,  
   
 def animate(i):        
   #ax.clear()  
     
   global frame_num, buffersize  
     
   if (frame_num%100==0):  
             print frame_num    
   line=f.readline()  
   jsondata=json.loads(line)  
   frame_num+=1  
     
   if "Position" in jsondata:  
     pos= jsondata["Position"]  
       
   if "Velocity" in jsondata:  
             vel=jsondata["Velocity"]  
   
   pos_buf.append(pos)  
   vel_buf.append(vel)  
     
   while len(pos_buf)>buffersize:  
     pos_buf.pop(0)  
   while len(vel_buf)>buffersize:  
     vel_buf.pop(0)  
     
   x=np.linspace(-len(pos_buf)/framerate,0,len(pos_buf))  
   dx=np.linspace(-len(vel_buf)/framerate,0,len(vel_buf))        
     
   plotline.set_data(x,headpose_buf)  
   dplotline.set_data(dx,headvel_buf)    
   return plotline, dplotline,  
   
 def folder_from_path(file):  
   split_path=file.split("/")  
   folder=split_path[0]  
   for i in range(1,len(split_path)-1):  
     folder+="/"+split_path[i]  
   return folder  
   
 #Setting up globals  
 file=""    
 frame_num=0    
   
 framerate=20.0  
 buffersize=5*20  
 vel_buf=[]  
 pos_buf=[]  
   
 fig, ax = plt.subplots()  
 ax.set_xlim(-5.0,2.0)  
 ax.set_ylim(-180,180)  
 my_dpi=96  
 fig.set_size_inches(640/my_dpi, 360/my_dpi)    
 plotline, =ax.plot([],[],lw=2,color='b')  
 dplotline, =ax.plot([],[], lw=2, color='r')  
   
 if __name__=='__main__':  
   if len(sys.argv)<2:  
     print "Usage:\nplot_data.py file"  
     exit()  
   #Parsing command-line arguments, checking the input file exists.  
   file= sys.argv[1]  
   print file  
   if (os.path.isfile(file)==False):  
       print "Problem with file"  
       exit()   
   folder=folder_from_path(file)  
   #Writing video to same folder  
   outfile=folder+"/data.avi"  
   print folder  
   #Getting number of data points from file. A little hackey, but I couldn't control animate() with the file open()            
   line_count=0  
   with open(file, 'r') as f:  
     for read_data in f:      
       line_count=line_count+1  
   print line_count  
   f=open(file,'r')  
   Writer = animation.writers['ffmpeg']  
   writer = Writer(fps=25, metadata=dict(artist='Sayanan'), extra_args=['-vcodec','libx264'])      
   anim = animation.FuncAnimation(fig, animate,init_func=init, frames=line_count-1,blit=True, interval=1, repeat=False)      
   anim.save(outfile,writer=writer)  
   f.close()  
   #plt.show()  
     

So there you go. A basic program that plots animation, and saves the resulting video to disk, using recorded data, stored in JSON on disk.

Edit: Just a quick addendum. I also had some trouble plotting animations of a grid of custom polygons, which could be useful for mapping, occupancy grids, or similar data. Here's a quick demo for how to generate animations of this sort:

 "Demo of a grid of PathPatch objects."  
 import numpy as np  
 import matplotlib  
 import matplotlib.pyplot as plt  
 import matplotlib.animation as animation  
 from time import time  
 import itertools  
   
 from matplotlib.path import Path  
 import matplotlib.patches as patches  
   
 def animate(i):       
   prob=np.random.rand(np.size(yr)*np.size(xl))  
   ax.clear()  
   cells=[]  
   for i in range(0,len(P1)):    
     verts=[P1[i], P2[i], P3[i], P4[i], P1[i] ]  
     path = Path(verts, codes)  
     patch = patches.PathPatch(path, color=[1-prob[i],prob[i],0], lw=2)  
     cells.append(ax.add_patch(patch))  
   return cells  
     
   
 fig, ax = plt.subplots()  
 fig.set_size_inches(5,20)  
   
 width=3.5  
 height=5.0  
   
 xl=np.arange(-8.75,8.75,width)  
 yr=np.arange(-50.0,50.0,height)  
 ax.set_xlim(xl[0],xl[np.size(xl)-1]+width)  
 ax.set_ylim(yr[0],yr[np.size(yr)-1]+height)  
 cells=[]  
 print yr[10]  
 print np.size(yr)  
 print np.size(xl)  
 P1=list(itertools.product(xl,yr+height))  
 P2=list(itertools.product(xl+width, yr+height))  
 P3=list(itertools.product(xl+width, yr))  
 P4=list(itertools.product(xl,yr))  
 prob=np.random.rand(np.size(yr)*np.size(xl))  
 codes = [Path.MOVETO,  
      Path.LINETO,  
      Path.LINETO,  
      Path.LINETO,  
      Path.CLOSEPOLY,  
      ]  
 for i in range(0,len(P1)):  
   verts=[P1[i], P2[i], P3[i], P4[i], P1[i] ]  
   path = Path(verts, codes)  
   patch = patches.PathPatch(path, color=[1-prob[i],prob[i],0], lw=2)  
   cells.append(ax.add_patch(patch))  
 anim = animation.FuncAnimation(fig, animate, blit=True, save_count=0,interval=10)  
 plt.show()  
   

Matplotlib Animation Save with FFmpeg Freezes After N Frames

Matplotlib is a go-to plotting tool in Python, which has a lot of useful features, like compatibility with the Scipy/Numpy stack. Sometimes it's useful to visualize time-series data temporally.

One very useful feature that Matplotlib has is the animations module, which allows you to animate plots with changing data [over time, for example], and save the data as a video file using FFmpeg. There's a great tutorial for Matplotlib animations here.

My only complaint about the Matplotlib animations, is that the documentation and examples are not super-comprehensive. I might create a follow-up post with some sample programs, to add to the documented examples floating around the interwebs.

In any case, once I got the animations working, I wanted to save them as a video for later on-demand viewing. I set up an animation writer with ffmpeg, x264, and let it run. Each time I ran the script, and the video saving would freeze after 820 frames.

It took a fair amount of online searching to find the answer here. It turns out that the stock Ubuntu/Debian FFmpeg and LibAV have branched apart at some point. You can obtain the "real" FFmpeg here. Once you install this, the Matplotlib animations save videos just fine.

USB Serial COM Ports and Python

I was working with a simple device that provides data via USB Serial COM. For whatever reason, the C++ library I was using to read from the device was not working consistently on different machines.

USB Serial COM is akin to socket programming. The routines are somewhat platform-dependent, and today's developer doesn't usually care about the details. Just give me an abstraction that lets me communicate!

Then I realized that Python has PySerial. Using this library, I was easily able to communicate with the device, and send it via localhost socket to my C++ program. Done and done.

Check out the super simple source code:

 import serial  
 import socket  
   
 IP="127.0.0.1"  
 PORT=3001  
 sock=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) #Sending data to localhost via UDP  
   
 ser= serial.Serial(2,timeout=2) #COM3 is port number 2.  
 print ser.name  
 line=''  
 while (True):  
      line=ser.readline()  
      print line  
      sock.sendto(line,(IP,PORT))  

Python for the win.

Karate Interpretations: Sport, Fighting, Self-defense

Over the past year, I've gotten back into Shotokan karate. I grew up doing karate, and completed my blackbelt exam when I was 15. There was a great sense of community with my dojo growing up, in Randallstown, Maryland.  The training there was pretty intense, and not geared towards tournaments.

 Over the years, I've been on and off with it. Practical considerations, like time, or distance to the neareset dojo, obviated my formal training when I was in San Diego. I used to train by myself.

It's great for fitness, teaches you about bio-mechanics and anatomy, and it's solid for self-defense. The full system consists of various punches, strikes, kicks, blocks, joint locks, grappling, throws, and take-downs. The traditional style are tailor made for close-range and medium-range hand-to-hand combat.

A problematic aspect arises when a dojo trains with too much emphasis on tournament competition. Tournaments have an artificial set of rules, that really favor high kicks, and fast punches to the body. These techniques alone are fairly impractical for self-defense.

If a dojo trains with too much emphasis on winning tournaments, it tends to ignore the more practical, self-defense oriented styles and techniques. Common moves like arm-bars, wrist-locks, and throws tend to be completely ignored in such a dojo. This is especially problematic for students, who think they are learning how to defend themselves, but are actually learning how to score points in tournament-style sparring.

There are some voices of reason out there, who train traditional karate, and include the grappling and throwing. A couple are Bruce Clayton, and Iain Abernathy. These two have produced books and dvds on the subject, the historical evolution of modern karate, and even lead practical self-defense karate clinics.

No single self-defense system is comprehensive, and so it's good to keep your eyes open. When I watch elite boxers, I see a lot of the same technique and form. When I watch brazilian jiu-jutsu guys, I see a lot of familiar grappling, but enhanced and taken to the ground-fighting scenario. Martial arts are enjoyable for sure.

Winsock TCP Client in C++

Recently I've been looking at a server-client software architecture for machine learning.  A lot of really great machine learning and mathematical modeling toolboxes are now written in Python. However, if you're working directly with a sensor-based system, your sensor capture code is often written in C/C++.

Yes, you could write a wrapper for that code, but it can be a slow and redundant process.

Instead, you can set up a Python TCP server like so , even on the localhost on the same machine, send a JSON structure with your feature vector, do your heavy number-crunching in Python, and send the result back to the C++ client.

There's a lot of documentation out there for TCP servers and clients on *nix in C and C++, but the documentation for Windows is less available, written in pure C, or cryptic to understand. I wanted a class that would abstract away most of the nuts and bolts.

Using the Windows Winsock documentation example, I've written a simple Winsock TCP client class in C++. This code will build with any semi-recent version of Visual Studio Express or Professional. It's really simple stuff, just easier to read.

Here's the header file, which I called TCPClient.h

#ifndef TCP_CLIENT_H
#define TCP_CLIENT_H
#define WIN32_LEAN_AND_MEAN

#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#include <stdlib.h>
#include <stdio.h>
#include <string>
#include <iostream>
#include <sstream>

#pragma comment (lib, "Ws2_32.lib")
#pragma comment (lib, "Mswsock.lib")
#pragma comment (lib, "AdvApi32.lib")


class TCPClient{
public:
    TCPClient(); //Default constructor
    TCPClient(const char* host, const char* port); //Constructor with host-ipaddress, port number
    TCPClient(std::string host, std::string port); //Constructor with host-ipaddress, port number
    TCPClient(const char* host, const char* port, const int buffer_size); //Constructor with host-ipaddress, port number, buffersize [rarely used]
    TCPClient(std::string host, std::string port, const int buffer_size); //Constructor with host-ipaddress, port number, buffersize [rarely used]
    int init(); //initialization, called by constructors
    ~TCPClient(); //clean up winsock environment
    bool open(); //open connection
    bool open(const char* host, const char* port); //open connection with host-ipaddress, port number
    bool open(std::string host, std::string port); //open connection with host-ipaddress, port number
    int send_buf(const char* sendbuf);    //send a message. Returns the number of bytes successfully sent. Returns -1 if failed.
    int send_buf(std::string sendbuffer); //send a message. Returns the number of bytes successfully sent. Returns -1 if failed.
    int receive(std::string& recvbuffer); //receive a message. Returns the number of bytes received.
    bool shutdown_send(); //shutdown the outgoing connection
    bool close(); //close the socket    
private:
    int resolve(); //used internally, resolving host. Returns -1 if failed.
    int connect_to(); //used internally, actually connect. Returns -1 if failed.

    int _buffer_size;    //buffer size in bytes
    int iResult;        //errorcodes from Winsock

    WSADATA wsaData;   //Winsock stuff
    SOCKET ConnectSocket; //Socket
    struct addrinfo *result, *ptr, hints; //Winsock stuff

    std::string host; //host ipaddress
    std::string port; //port number
    bool _connection_open; // Is the connection open?
    bool _ipset; //Did we initialize with host and port number?
    bool _shutdown_sent; //Have we sent a shutdown message to the host?
};

#endif

Here's the class implementation. Again, really simple stuff.

#include "TCPClient.h"

int TCPClient::init(){
    ConnectSocket = INVALID_SOCKET;
    result=NULL;
    ptr=NULL;

    iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
    if (iResult != 0) {
        std::cerr<<"WSAStartup failed with error: "<<iResult<<"\n";
        return -1;
    }
    ZeroMemory( &hints, sizeof(hints) );
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_protocol = IPPROTO_TCP;
    this->_buffer_size=4096;
    this->_shutdown_sent=false;
    return 0;
}
TCPClient::TCPClient(){
    this->_ipset=false;
    if (init()<0){
        std::cerr<<"Error initializing WinSock\n";
        exit(1);
    }
}
TCPClient::TCPClient(const char* host, const char* port){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    if (init()<0){
        std::cerr<<"Error initializing WinSock\n";
        exit(1);
    }
}
TCPClient::TCPClient(std::string host, std::string port){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    if (init()<0){
        std::cerr<<"Error initializing WinSock\n";
        exit(1);
    }
}
TCPClient::TCPClient(const char* host, const char* port, const int buffer_size){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    this->_buffer_size=buffer_size;
    if (init()<0){
        std::cerr<<"Error initializing WinSock\n";
        exit(1);
    }
}
TCPClient::TCPClient(std::string host, std::string port, const int buffer_size){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    this->_buffer_size=buffer_size;
    if (init()<0){
        std::cerr<<"Error initializing WinSock\n";
        exit(1);
    }
}
TCPClient::~TCPClient(){
    WSACleanup();
}

int TCPClient::resolve(){
    // Resolve the server address and port
    iResult = getaddrinfo(host.c_str(), port.c_str(), &hints, &result);
    if ( iResult != 0 ) {
        std::cerr<<"getaddrinfo failed with error: "<<iResult<<"\n";
        WSACleanup();
        return -1;
    }
    return 0;
}
int TCPClient::connect_to(){
    // Attempt to connect to an address until one succeeds
    for(ptr=result; ptr != NULL ;ptr=ptr->ai_next) {
        // Create a SOCKET for connecting to server
        ConnectSocket = socket(ptr->ai_family, ptr->ai_socktype, 
            ptr->ai_protocol);
        if (ConnectSocket == INVALID_SOCKET) {           
            std::cerr<<"socket failed with error: "<<WSAGetLastError()<<"\n";
            WSACleanup();
            return -1;
        }
        // Connect to server.
        iResult = connect( ConnectSocket, ptr->ai_addr, (int)ptr->ai_addrlen);
        if (iResult == SOCKET_ERROR) {
            closesocket(ConnectSocket);
            ConnectSocket = INVALID_SOCKET;
            continue;
        }
        break;
    }
    freeaddrinfo(result);
    if (ConnectSocket == INVALID_SOCKET) {
        std::cerr<<"Unable to connect to server!\n";
        WSACleanup();
        return -1;
    }
    return 0;
}

bool TCPClient::open(){
    if (this->_ipset==false)
        return false;
    if (resolve()<0)
        return false;
    if (connect_to()<0)
        return false;
    return true;
}
bool TCPClient::open(const char* host, const char* port){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    this->_connection_open=this->open();
    return _connection_open;
}
bool TCPClient::open(std::string host, std::string port){
    this->host=host;
    this->port=port;
    this->_ipset=true;
    this->_connection_open=this->open();
    return _connection_open;
}
int TCPClient::send_buf(const char* sendbuf){
    if (_connection_open==false)
        return -1;
    // Send an initial buffer

    iResult = send( ConnectSocket, sendbuf, (int)strlen(sendbuf), 0 );
    if (iResult == SOCKET_ERROR) {
        std::cerr<<"socket failed with error: "<<WSAGetLastError()<<"\n";    
        WSACleanup();
        return -1;
    }   
    return iResult;
}
int TCPClient::send_buf(std::string sendbuffer){
    
    int length=sendbuffer.length();
    if (_connection_open==false)
        return -1;
    // Send an initial buffer
    iResult = send( ConnectSocket, sendbuffer.c_str(), length, 0 );

    if (iResult == SOCKET_ERROR) {
        std::cerr<<"socket failed with error: "<<WSAGetLastError()<<"\n";
        this->close();
        WSACleanup();
        return -1;
    }   
    return iResult;
}
int TCPClient::receive(std::string& recvbuffer){
    // Receive up to _buffer_size bytes
    recvbuffer.clear();
    int total=0;
    char* recvbuf=new char[this->_buffer_size];    
    iResult = recv(ConnectSocket, recvbuf, _buffer_size, 0);
        if ( iResult > 0 ){            
            std::string received_str=recvbuf;
            received_str.resize(iResult);
            recvbuffer+=received_str;
            total+=iResult;            
        }
        else if ( iResult == 0 ){
            std::cout<<"\nConnection closed\n";            
        }
        else            
            std::cerr<<"recv failed with error: "<<WSAGetLastError()<<"\n";
   
    delete [] recvbuf;
    return total;
}
bool TCPClient::shutdown_send(){
     // shutdown the connection since no more data will be sent
    iResult = shutdown(ConnectSocket, SD_SEND);    
    if (iResult == SOCKET_ERROR) {
       std::cerr<<"shutdown failed with error: "<<WSAGetLastError()<<"\n";       
        this->close();
        WSACleanup();
        return false;
    }
    this->_shutdown_sent=true;
    return true;
}
bool TCPClient::close(){
    if (!_shutdown_sent)
        shutdown_send();        
     // cleanup
    closesocket(ConnectSocket);
    return true;
}

And finally, this is the main function, which shows how to invoke the class. I had a very simple TCP server written in Python, running on a different machine on the same local network.

The main program simple gets your keyboard input and sends it to the server. If you input the string "exit", the program exits cleanly.

int main (int argc, char* argv[]){
    TCPClient myClient("192.168.1.2","5000");    
    if (!myClient.open()){
        char a;
        std::cout<<"Enter any key to exit\n";
        std::cin>>a;
        return 0;
    }

    while (true){        
        std::string input; 
        std::cout<<"Enter something to send to the server\n";
        getline(std::cin,input);
        if (!input.compare("exit")){
            myClient.close();
            break;
        }
        if (myClient.send_buf(input)<0)    
            break;    
        std::string received;
        if    (myClient.receive(received)<0)
            break;    
        std::cout<<"Received:\n"<<received<<"\n\n";            
    }
    return 0;
}

Much easier to read and understand, than the original code on the Microsoft documentation site. Cheers!

El Nino, California Drought

NOAA performs a vital a national and international service in monitoring ocean and atmospheric conditions, recognizing patterns, and classifying and predicting significant climate events.

One of the somewhat-well-understood phenomena is the El Nino Southern Oscillation cycle. Characterized by warmer-than-average ocean temperatures across the Pacific Ocean, and a weakening or reversal of the prevailing Pacific Trade winds, El Nino years tend to bring unusual weather and some extreme weather events. In past El Nino years, we've seen unusual shifts in precipitation, including drought in some places, flooding and even snow in others.

All this is of interest to the drought-stricken American West and Southwest, because strong El Nino years tend to correlate with above-average rainfall in these regions. NOAA maintains a public communication, including monthly updates here. Currently, we are in an El Nino advisory state. A more detailed document, detailing various observations and methodologies is here.

Overall, NOAA predicts the probability of an El Nino by fall or winter at about 80%, but the El Nino is likely going to be "weak to moderate'.

I really enjoyed viewing the satellite imagery, and temperature time series in the .pdf document. I also enjoyed the numerous temperature prediction models, split into "Statistical" and "Dynamical" models. They also included an ensemble prediction. Good stuff.

Conferences

Last week, I attended the IEEE Intelligent Vehicles Symposium, hosted this year in Dearborn, Michigan. It was nice to run into old colleagues from UCSD, as well as former collaborators from places like Toyota, and researchers from around the world.

I've always enjoyed attending conferences, as they provide a forum for exchanging new ideas and learning about new developments. In our own labs and workplaces, we can get stuck, recycling the same ideas and approaches, trapped in our echo chambers. At a conference, we can learn from each other.

There was a lot of work presented on autonomous driving. Honestly, the topic is not very exciting anymore. Autonomous driving has been done reliably by many groups, and has really transitioned from a research problem into an engineering problem. Now car companies are figuring out how to bring autonomous driving to the market and make profits from it.

In particular, a former collaborator at UCSD, Eshed Ohn-Bar, presented some of his work on hand-tracking in the vehicle, which he's had quite a bit of experience with. He also clued me in to some new work from Pietro Perona's group on object detection. It turns out that Serge Belongie, who sat on my PhD committee, is also a co-author on the paper. Good stuff.

There was a time when I was sick of doing computer vision. I took a break from it for a little while, working with some other sensors, and learning about some other areas. My plan is to get back into some computer vision on a work project later this year.