Cameras Round 1

On Thursday I decided it was time to hook up my cameras (LinkSprite JPEG camera). I've actually had them for a while now, but unfortunately the JVC cables required to plug it in weren't part of the kit and got delayed in the post. Anyway, I now have all the bits, as you can see below:

LinkSprite JPEG camera with JVC cable plugged in

After a bit of digging I find 2 bits of source code to get this working on an Arduino.One is from the Spark Fun web site at the link I posted earlier, and the other is directly from LinkSprite. Both are designed for an older version of the Arduino libraries, so neither compile, which didn't help with deciding which one to use as a starting point. In the end I choose the Spark Fun one which provides a small library and wrapper class, which in heinsight may have been a mistake.

Anyway, I start by wiring the camera up to one of my spare Arduino Unos and spend an hour updating the example code (this consists mainly of stripping out everything except camera code, replacing use of NewSoftSerial with SoftwareSerial, and fixing a load of include files that have changed name). After some initial confusion resulting from incorrectly coloured wires, I have the camera hooked directly into the Arduino board:

Camera connected to Arduino Uno

Note that the green/blue wires are Vcc and GND, and red/black are Tx/Rx. This would normally be other way round, but I'm guessing LinkSprite didn't go with standard use of a JVC socket. The yellow wire is apparently to attach to a tv, presumably a composite connection.

Next, having got the libraries to compile I place them in the correct folder, and setup the main arduino code. This snippet is roughly what I had at the time:

#include <JPEGCamera.h>
#include <SoftwareSerial.h>

char response[512];
unsigned int count=0;
int size=0;
int address=0;
int eof=0;
JPEGCamera camera;

void setup()
{
     //Setup the camera and serial port
    Serial.begin(9600);
    Serial.println("Hello");
  
    camera.begin();
    camera.reset(response);
    delay(3000);
    
    //take a picture
    Serial.println("Taking picture");
    camera.takePicture(response);
    delay(2000);
    Serial.println("");

    //Get + print the size of the picture
    Serial.println("Size");
    count = camera.getSize(response, &size);
    Serial.println(size);
    Serial.println("");
}

void loop()
{
}

The camera works through serial communication, so internally the JPEGCamera library is using SoftwareSerial to talk to it over IO ports 2 and 3. All this code does is tell the library to send a few commands and wait for the responses - first to reset the camera, then take a picture, and finally retreive and print out it's size. After not too much time I get this being printed out in the serial monitor:

Taking picture
Size
3600

A 3.5k jpeg image has been taken! Hurray :)

OK, time to get some images out of this baby. Problem is, I'm just sending data back to PC as text through the serial monitor, so how to send the jpeg? Well at this point I turn to an old c++ trick which gets used all over the place. After reading the size, I add the following loop to extract actual data:

    //Get + print data
    Serial.println("Data");
    Serial.println("{");
    while(address < size)
    {               
        //pull the data out that we requested earlier
        count=camera.readData(response,address);
        for(int i = 0; i < count; i++)
        {
            Serial.print("0x");
            Serial.print(uint8_t(response[i]),HEX);
            Serial.print(", ");
        }
        Serial.println();
        address+=count;
    }
    Serial.println("}");

This reads data in 32 byte chunks from the camera and sends it as text to the serial monitor. What it actually prints looks something like this:

Hello
Taking picture

Size
3652

Data
{
0xFF, 0xD8, 0xFF, 0xFE, 
.... lots more data here ....
0x58, 0xF, 0xFF, 0xD9, 
}

The nice thing is, the section after data is formatted just like a standard c++ array definition. It can be copy and pasted into any c++ or c# application in it's raw form and compiled into the executable. In games this trick is often used to store start up screens inside the exe, allowing you to get around restrictions on boot time. Here though I'm gonna plonk it into my c# mmbot app and dump it to a jpeg in the first few lines of code:

        byte[] data = new byte[] { ... copy and pasted data here ... }
        
        public MainWindow()
        {
            using (FileStream f = File.OpenWrite("myimg.jpg"))
            {
                f.Write(data, 0, data.Length);
            }

And here it is - the resulting image - me looking pleased with myself:

First photo taken from the camera (me looking smug)

The camera is configurable to do less compression and higher resolution than this image, but it's a good start!

OK, doing well, but the text based image sending is a bit silly frankly. Instead I make 2 tiny tweaks to the Arduino code. First, a little wait at the start for any data to arrive from the serial port:

    //wait for pc
    while(!Serial.available());
    while(Serial.available()) Serial.read();

And the inner loop now posts data in raw binary form back to the pc:

    for(int i = 0; i < count; i++)
    {
      Serial.write(uint8_t(response[i]));
    }

Now I modify my MmBot c# app to read the size and data from the serial port and decode / display it in a window:

       public void DoneJpegRead(byte[] data)
        {
            try
            {
                MemoryStream memstream = new MemoryStream(data);

                JpegBitmapDecoder decoder = new JpegBitmapDecoder(memstream, BitmapCreateOptions.None, BitmapCacheOption.Default);
                MainImage.Source = decoder.Frames[0];
            }
            catch (System.Exception e)
            {
            }
         }

        public void DoPortThread()
        {
            //open serial port
            SerialPort port = new SerialPort("COM9", 9600);
            port.Open();

            //clear any bytes waiting in it
            while (port.BytesToRead > 0)
                port.ReadByte();

            byte[] buff = null;
            while (true)
            {
                port.Write(new byte[] { 0 }, 0, 1);
 
                string val = port.ReadLine().Trim();
                System.Diagnostics.Debug.WriteLine(val);
                if (val == "Size")
                {
                    int size = Convert.ToInt32(port.ReadLine().Trim());
                    buff = new byte[size];
                }
                else if (val == "Data")
                {
                    int idx = 0;
                    int lastdisplay = 0;
                    System.Diagnostics.Debug.WriteLine("Reading " + buff.Length.ToString() + " bytes");
                    while (idx < buff.Length)
                    {
                        idx += port.Read(buff, idx, buff.Length - idx);
                        if (idx > (lastdisplay + 1))
                        {
                            lastdisplay = idx;
                        }
                    }
                    Dispatcher.Invoke(new Action(delegate { DoneJpegRead(buff); }), new object[] { });
                }
            }

            port.Close();
        }

This results in the following screen shot:

Slightly corrupted image sent to and displayed in a c# app

Not entirely sure where that corruption is coming from - images seem to come through fine using my earlier copy-and-paste technique, which suggests a bug in my c# code or some weirdness with the .net jpeg decoder. Edit: I later worked out this was because the serial buffer was overflowing. Calling Serial.flush() before sending data to pc fixed it.

For one final trick, I load up Face SDK - the face recognition system I intend to use (at least to begin with). This is a great little piece of software, which the developers very kindly let me have cheap when I said I just wanted to build a robot. Here it is looking at my photo and identifying my face!

Luxand FaceSDK picking out my face from the first photo

So, all good right? Camera working? Everything wonderful. Well.... this is where things got a bit hairy. After some testing I discover it takes a few seconds to get data back from the camera all the way to the pc. As a first step to speed things up I knock the serial port baud rate up to 115200 and test again. It's a little bit faster but still not great, so I add some code in the Arduino to time things. As it turns out the Camera->Arduino communication is taking about 10 times longer than the Arduino->PC communication. I put this partially down to the fact that I'm still using software serial communication, and decide it's time to move onto MmBot, which has an Arduino Mega in, with 4 hardware serial ports. In theory these should work better and faster right? Well.... No. You can see the camera wired to the robot here:

Camera plugged into MmBot

I'm not going to go through all the random things I tried to get this to work. It seems to communciate for a bit and then give up. Unplugging everything other than camera didn't work. Trying different ports didn't work. Head butting my desk didn't work. After digging around I eventually discover that sometimes I'm receiving the same data twice. The process for getting data from the camera goes something like:

• Send 'read data' command (which includes how much data you want - in this case 32 bytes)
• Receive 5 byte response which is basically 'ok'
• Receive 32 bytes of data

For some reason, having switched to hardware serial communication I am now getting:

• Send 'read data' command (which includes how much data you want - in this case 32 bytes)
• I receive the 5 byte response
• Sometimes I receive the 5 byte response again for no particular reason 
• Receive 32 bytes of data

As a result, every time this happens I get 5 bytes out of sync. Eventually serial buffers overflow, or even if they don't, I just get a completely corrupted image. 

So what to do? Well, I don't know yet. I did all this on Thursday and have basically redone it all up to here again today (Saturday) to see if I made any mistakes. If it was a general serial port issue I'd expect the problem to be fairly random, but it seems specific to 5 bytes after the read data request. This makes me think there's something special about how you need to talk to the camera itself. I've tried both cameras and they both have the same issue, so it's unlikely to be hardware problems. Next I think I'm going to try the simpler code from Link Sprite to see if there's anything the Spark Fun guys missed out. 

Hopefully by the end of the day I'll get the hardware serial working. If not, I'll probably just give up on that and make do with slow software serial comms for now (once I get my raspberry pie this'll all be hi def usb web cams anyway!).