Saturday, 28 April 2012

Cameras Round 1

On Thursday I decided it was time to hook up my cameras (LinkSprite JPEG camera). I've actually had them for a while now, but unfortunately the JVC cables required to plug it in weren't part of the kit and got delayed in the post. Anyway, I now have all the bits, as you can see below:

LinkSprite JPEG camera with JVC cable plugged in

After a bit of digging I find 2 bits of source code to get this working on an Arduino.One is from the Spark Fun web site at the link I posted earlier, and the other is directly from LinkSprite. Both are designed for an older version of the Arduino libraries, so neither compile, which didn't help with deciding which one to use as a starting point. In the end I choose the Spark Fun one which provides a small library and wrapper class, which in heinsight may have been a mistake.

Anyway, I start by wiring the camera up to one of my spare Arduino Unos and spend an hour updating the example code (this consists mainly of stripping out everything except camera code, replacing use of NewSoftSerial with SoftwareSerial, and fixing a load of include files that have changed name). After some initial confusion resulting from incorrectly coloured wires, I have the camera hooked directly into the Arduino board:

Camera connected to Arduino Uno

Note that the green/blue wires are Vcc and GND, and red/black are Tx/Rx. This would normally be other way round, but I'm guessing LinkSprite didn't go with standard use of a JVC socket. The yellow wire is apparently to attach to a tv, presumably a composite connection.

Next, having got the libraries to compile I place them in the correct folder, and setup the main arduino code. This snippet is roughly what I had at the time:


#include <JPEGCamera.h>
#include <SoftwareSerial.h>

char response[512];
unsigned int count=0;
int size=0;
int address=0;
int eof=0;
JPEGCamera camera;

void setup()
{
     //Setup the camera and serial port
    Serial.begin(9600);
    Serial.println("Hello");
  
    camera.begin();
    camera.reset(response);
    delay(3000);
    
    //take a picture
    Serial.println("Taking picture");
    camera.takePicture(response);
    delay(2000);
    Serial.println("");

    //Get + print the size of the picture
    Serial.println("Size");
    count = camera.getSize(response, &size);
    Serial.println(size);
    Serial.println("");
}

void loop()
{
}

The camera works through serial communication, so internally the JPEGCamera library is using SoftwareSerial to talk to it over IO ports 2 and 3. All this code does is tell the library to send a few commands and wait for the responses - first to reset the camera, then take a picture, and finally retreive and print out it's size. After not too much time I get this being printed out in the serial monitor:

Taking picture
Size
3600

A 3.5k jpeg image has been taken! Hurray :)

OK, time to get some images out of this baby. Problem is, I'm just sending data back to PC as text through the serial monitor, so how to send the jpeg? Well at this point I turn to an old c++ trick which gets used all over the place. After reading the size, I add the following loop to extract actual data:

    //Get + print data
    Serial.println("Data");
    Serial.println("{");
    while(address < size)
    {               
        //pull the data out that we requested earlier
        count=camera.readData(response,address);
        for(int i = 0; i < count; i++)
        {
            Serial.print("0x");
            Serial.print(uint8_t(response[i]),HEX);
            Serial.print(", ");
        }
        Serial.println();
        address+=count;
    }
    Serial.println("}");

This reads data in 32 byte chunks from the camera and sends it as text to the serial monitor. What it actually prints looks something like this:


Hello
Taking picture

Size
3652

Data
{
0xFF, 0xD8, 0xFF, 0xFE, 
.... lots more data here ....
0x58, 0xF, 0xFF, 0xD9, 
}


The nice thing is, the section after data is formatted just like a standard c++ array definition. It can be copy and pasted into any c++ or c# application in it's raw form and compiled into the executable. In games this trick is often used to store start up screens inside the exe, allowing you to get around restrictions on boot time. Here though I'm gonna plonk it into my c# mmbot app and dump it to a jpeg in the first few lines of code:


        byte[] data = new byte[] { ... copy and pasted data here ... }
        
        public MainWindow()
        {
            using (FileStream f = File.OpenWrite("myimg.jpg"))
            {
                f.Write(data, 0, data.Length);
            }


And here it is - the resulting image - me looking pleased with myself:

First photo taken from the camera (me looking smug)

The camera is configurable to do less compression and higher resolution than this image, but it's a good start!

OK, doing well, but the text based image sending is a bit silly frankly. Instead I make 2 tiny tweaks to the Arduino code. First, a little wait at the start for any data to arrive from the serial port:

    //wait for pc
    while(!Serial.available());
    while(Serial.available()) Serial.read();

And the inner loop now posts data in raw binary form back to the pc:

    for(int i = 0; i < count; i++)
    {
      Serial.write(uint8_t(response[i]));
    }

Now I modify my MmBot c# app to read the size and data from the serial port and decode / display it in a window:

       public void DoneJpegRead(byte[] data)
        {
            try
            {
                MemoryStream memstream = new MemoryStream(data);

                JpegBitmapDecoder decoder = new JpegBitmapDecoder(memstream, BitmapCreateOptions.None, BitmapCacheOption.Default);
                MainImage.Source = decoder.Frames[0];
            }
            catch (System.Exception e)
            {
            }
         }

        public void DoPortThread()
        {
            //open serial port
            SerialPort port = new SerialPort("COM9", 9600);
            port.Open();

            //clear any bytes waiting in it
            while (port.BytesToRead > 0)
                port.ReadByte();

            byte[] buff = null;
            while (true)
            {
                port.Write(new byte[] { 0 }, 0, 1);
 
                string val = port.ReadLine().Trim();
                System.Diagnostics.Debug.WriteLine(val);
                if (val == "Size")
                {
                    int size = Convert.ToInt32(port.ReadLine().Trim());
                    buff = new byte[size];
                }
                else if (val == "Data")
                {
                    int idx = 0;
                    int lastdisplay = 0;
                    System.Diagnostics.Debug.WriteLine("Reading " + buff.Length.ToString() + " bytes");
                    while (idx < buff.Length)
                    {
                        idx += port.Read(buff, idx, buff.Length - idx);
                        if (idx > (lastdisplay + 1))
                        {
                            lastdisplay = idx;
                        }
                    }
                    Dispatcher.Invoke(new Action(delegate { DoneJpegRead(buff); }), new object[] { });
                }
            }

            port.Close();
        }

This results in the following screen shot:

Slightly corrupted image sent to and displayed in a c# app

Not entirely sure where that corruption is coming from - images seem to come through fine using my earlier copy-and-paste technique, which suggests a bug in my c# code or some weirdness with the .net jpeg decoder. Edit: I later worked out this was because the serial buffer was overflowing. Calling Serial.flush() before sending data to pc fixed it.

For one final trick, I load up Face SDK - the face recognition system I intend to use (at least to begin with). This is a great little piece of software, which the developers very kindly let me have cheap when I said I just wanted to build a robot. Here it is looking at my photo and identifying my face!

Luxand FaceSDK picking out my face from the first photo

So, all good right? Camera working? Everything wonderful. Well.... this is where things got a bit hairy. After some testing I discover it takes a few seconds to get data back from the camera all the way to the pc. As a first step to speed things up I knock the serial port baud rate up to 115200 and test again. It's a little bit faster but still not great, so I add some code in the Arduino to time things. As it turns out the Camera->Arduino communication is taking about 10 times longer than the Arduino->PC communication. I put this partially down to the fact that I'm still using software serial communication, and decide it's time to move onto MmBot, which has an Arduino Mega in, with 4 hardware serial ports. In theory these should work better and faster right? Well.... No. You can see the camera wired to the robot here:

Camera plugged into MmBot

I'm not going to go through all the random things I tried to get this to work. It seems to communciate for a bit and then give up. Unplugging everything other than camera didn't work. Trying different ports didn't work. Head butting my desk didn't work. After digging around I eventually discover that sometimes I'm receiving the same data twice. The process for getting data from the camera goes something like:

  • Send 'read data' command (which includes how much data you want - in this case 32 bytes)
  • Receive 5 byte response which is basically 'ok'
  • Receive 32 bytes of data
For some reason, having switched to hardware serial communication I am now getting:
  • Send 'read data' command (which includes how much data you want - in this case 32 bytes)
  • I receive the 5 byte response
  • Sometimes I receive the 5 byte response again for no particular reason 
  • Receive 32 bytes of data
As a result, every time this happens I get 5 bytes out of sync. Eventually serial buffers overflow, or even if they don't, I just get a completely corrupted image. 

So what to do? Well, I don't know yet. I did all this on Thursday and have basically redone it all up to here again today (Saturday) to see if I made any mistakes. If it was a general serial port issue I'd expect the problem to be fairly random, but it seems specific to 5 bytes after the read data request. This makes me think there's something special about how you need to talk to the camera itself. I've tried both cameras and they both have the same issue, so it's unlikely to be hardware problems. Next I think I'm going to try the simpler code from Link Sprite to see if there's anything the Spark Fun guys missed out. 

Hopefully by the end of the day I'll get the hardware serial working. If not, I'll probably just give up on that and make do with slow software serial comms for now (once I get my raspberry pie this'll all be hi def usb web cams anyway!). 

11 comments:

  1. Hi,

    I am currently messing about with these cameras and the arduino as well - however, I'm tearing my hair out at the moment! Try as I might I cannot get a well formatted jpeg from them - the first line of my jpeg always starts:

    FF EC FF FF ...

    I've tried 5V / 3.3V / the linksprite code / your code / and a hacked version of the sparkfun library.

    I also don't get the reply messages stated in the documentation - for example, when I reset the camera:

    send >> 56 00 26 00

    recv << F6 80 93 80 C0 D6 A1 98 13 69 9 29 F2 C0 C1 35 11 D F1 E4 D8 A0 B5 B7 B3 59 48 19 AF AD 2E D7 88 AC 6A 95 E5 5A E4 B3 B3 5A 2B 45 5 9A 95 DD E6 EF B8 8D 84 9B 99 CD 21 F1 C9 B6 B4 BA 40

    Any ideas?

    Alex

    ReplyDelete
    Replies
    1. Hi Alex

      Sorry for the delay in replies - been busy lately. I've had a lot of trouble with these cameras - after many weeks I eventually got a very slow upload of an image. The majority of my problems were due to the baud rate being incorrect, and losing data in the serial connection. Hard to guess without seeing more details, but if you give me an email address I'm happy to send over all my finished code - a couple of people I know have managed to get it working with this. Personally though, if its web cam data you want, I'd buy a raspberry pi (which I'll be working on with web cams in the near future). My latest post also shows how to connect the pi to an arduino - the 2 combined should make a powerful force :)

      -Chris

      Delete
    2. i need the code please

      yehia.heiba@gmail.com

      thanks in advance

      Delete
  2. Hi Chris,

    I'm using Visual Studio 2010. Could you help me to do make a similar c# app using that?
    I'm pretty new on dealing with these c#...

    ReplyDelete
  3. Hey Amin

    There's not a great deal to the c# side of things - the root of it is to use the Serial interface provided by the .net library. This will allow you to communicate over the serial port with the arduino.

    Once you have the jpeg from the serial port, getting it to display is trickier, but you want to start with researching the wpf imaging libraries in .net - it basically involves a 'JpegImageSource' object, with a load of bits around it you can find in the examples. That gives you an image source you can show in a wpf image box.

    -Chris

    ReplyDelete
  4. Hi Chris,

    would you be so kind to send me your code? I am also trying to connect this cam to an Arduino Mega via the Hardware Serial. Not really working yet...
    my mail is bee.develop (a) yahoo.com

    would be nice of you!
    Cheers!

    ReplyDelete
  5. hello...i want to know about this project coz i have use same camera..this is also video recording right? how you connect camera+arduino with PC?? what is your library for your coding camera?

    ReplyDelete
  6. hi chris,
    I am building quadcopter with camera, and I need the code please.
    here is my Email:
    Alsayed050@stemegypt.edu.eg

    thanks in advance

    ReplyDelete
  7. I know this topic is getting a little long in the tooth, but would it be possible to get the code/app you were using. my email address is hensonnd@gmail.com

    I would appreciate the help!

    thanks,
    Nick

    ReplyDelete
  8. it is possible to connect a webcam ( an old Logitech QuickCam Express cam pro 3000) from the pin to leonardo arduino , as easy as JPEG Color LinkSprite ????

    from Chile
    please answer :)
    david.reyes.orellana@gmail.com

    ReplyDelete
  9. Hello!! I am working on an educational project with the mega arduino and the camera ov7670, is to take a photo when you pass a card to identify the user, and that the photo is a database created in xampp by way of php code, unfortunately not I found a tutorial or code to take the photo and send it to the database. Someone could help me or provide some material to do my work, I will thank you infinitely !!

    My email is gabo_libra_1990@hotmail.com

    ReplyDelete