Quantcast

Overo COM + Caspa Camera for image processing

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Overo COM + Caspa Camera for image processing

sebastien manin
Hi everyone,

I'm working on a video processing system for an UAV. 
The size and weight of the Overo COM is perfect for us, but I saw in the mailing list archives that the FPS with the Caspa cameras are not what the datasheet says (60 FPS), and that the processor load is at 100%.

I'd like to know if I could use an Overo IronSTORM COM + Caspa VL camera board and get at least : 30 frames per second at 752x480 that can be used by my C/C++ image processing program with OpenCV (just an example). 
I heard that the problem could be the driver used with the Caspa board (v4l2 isn't it ?), I'd like to know what is the problem before buying.

Thank you in advance for your help !

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Florian Vaussard
Hello,

Le 17/08/2012 11:34, sebastien manin a écrit :

> Hi everyone,
>
> I'm working on a video processing system for an UAV.
> The size and weight of the Overo COM is perfect for us, but I saw in
> the mailing list archives that the FPS with the Caspa cameras are not
> what the datasheet says (60 FPS), and that the processor load is at 100%.
>
> I'd like to know if I could use an Overo IronSTORM COM + Caspa VL
> camera board and get at least : 30 frames per second at 752x480 that
> can be used by my C/C++ image processing program with OpenCV (just an
> example).
> I heard that the problem could be the driver used with the Caspa board
> (v4l2 isn't it ?), I'd like to know what is the problem before buying.

I streamed images at 752x480 / 30 fps using the Caspa, performing the
mpeg4 compression using the DSP. The CPU usage was kept low in this
case, however I cannot tell the figure by heart. So the problem is
definitely not on the driver side. The driver is using the media
controller framework, thus your application will see it as v4l2 api, but
you will need additional tools to configure the acquisition channel.
See: http://omappedia.org/wiki/Camera-ISP_Driver.
Doing image processing in C++ with OpenCV can be pretty heavy, so
depending on the algorithm you are running, you will be limited by the
CPU, hence limiting your framerate. There is some work to accelerate
OpenCV using the DSP, but I cannot tell the current status. I saw a
white paper from TI on this.

>
> Thank you in advance for your help !
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
> _______________________________________________
> gumstix-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gumstix-users

Cheers,
Florian

--
Florian Vaussard
EPFL - STI - IMT - LSRO1
MEB330 - Station 9
1015 Lausanne / Switzerland

tel: +41 21 693 78 39
fax: +41 21 693 78 07
http://lsro.epfl.ch


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Simon Knopp
Hi,

On 29/08/2012, at 8:12 PM, Florian Vaussard wrote:

> Doing image processing in C++ with OpenCV can be pretty heavy, so
> depending on the algorithm you are running, you will be limited by the
> CPU, hence limiting your framerate. There is some work to accelerate
> OpenCV using the DSP, but I cannot tell the current status. I saw a
> white paper from TI on this.

Just to chime in here, I too saw the whitepaper from TI about OpenCV [1]. Unfortunately I then found a post in their forums stating that they have de-emphasised support for OpenCV and aren't distributing their DSP port [2] :(

If you aren't tied to OpenCV then you could look into IMGLIB, TI's DSP image processing library. It doesn't have nearly the range of functions of OpenCV but it's a start. It's relatively easy to use via C6EZRun.

Simon.

[1]: http://focus.ti.com/lit/wp/spry175/spry175.pdf
[2]: http://e2e.ti.com/support/embedded/bios/f/355/t/125780.aspx


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Steven
In reply to this post by sebastien manin
>From my experience with the capsa camera about an year ago, the driver
overhead was higher than I expected (this might have been fixed
though). *If* you're doing video compression @ 60fps then you might
get close to 100% (even with the DSP).

I also did work with OpenCV using a different USB v4l2 camera and it
worked fine, abet without NEON or DSP acceleration.  You can
definitely capture at 30fps, but no one can tell you if you can
process at that rate since OpenCV contains a wide variety of
algorithms with wildly different time complexities.

One more thing, OpenCV's capture code isn't super well written, so you
might have to write your own.  Luckily there is a v4l2 example that's
pretty easy to follow.
http://libav.org/doxygen/master/v4l2_8c_source.html

On Fri, Aug 17, 2012 at 5:34 AM, sebastien manin
<[hidden email]> wrote:

> Hi everyone,
>
> I'm working on a video processing system for an UAV.
> The size and weight of the Overo COM is perfect for us, but I saw in the
> mailing list archives that the FPS with the Caspa cameras are not what the
> datasheet says (60 FPS), and that the processor load is at 100%.
>
> I'd like to know if I could use an Overo IronSTORM COM + Caspa VL camera
> board and get at least : 30 frames per second at 752x480 that can be used by
> my C/C++ image processing program with OpenCV (just an example).
> I heard that the problem could be the driver used with the Caspa board (v4l2
> isn't it ?), I'd like to know what is the problem before buying.
>
> Thank you in advance for your help !
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> gumstix-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gumstix-users
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Florian Vaussard
In reply to this post by Florian Vaussard
Hi Brahim (cc'ing to the list),

Le 29/08/2012 14:04, Brahim HAMADICHAREF a écrit :
> Hi Florian
>
> Is there a detailed tutorial to acheive this "streamed images at 752x480
> at 30 fps using the Caspa, performing thempeg4 compression using the DSP ?

Unfortunately not for now, I had to do this in a hurry for my work. I will
outline the procedure, and maybe post a more detailed tutorial in the near
future if time permits.

>
> When you say stream, you mean stream over ethernet ?

Yes, over Ethernet, although Wifi was also working. I am using the RTP
facility
provided by gstreamer.

>
> I can capture CAPSA at 752x480 at high rate but to display at 30 fps for
> example is not possible for me yet. My threaded Qt4 program eats all
> CPU usage ...

If you want to process the image before streaming it, you will have to
adapt my
procedure, as I was only broadcasting the compressed multimedia flux.
But gstreamer is quite flexible for that.

>
> Any help will be appreciated
Here for the brief outline (details are hanging around in several
mailing lists
and on the internet, but a lot of stuff is quite out-dated). Sorry, not
an easy
solution, maybe someone will have something better. But performances
were great.

- Compile a recent kernel with the tidsp bridge support. I was using a
3.2 at
that time, but no reason you should not use a 3.5. Here are the
important config
flags to look for:
CONFIG_OMAP_MBOX_FWK
CONFIG_TIDSPBRIDGE

TIDSPBRIDGE was in the staging area at that time.

- I also experienced problems using gstreamer, as it was expecting some
unimplemented APIs. I resolved this using a *very* ugly, yet working, patch
[1]. I hopped to never publish such a bad piece of code, but if it could
help...

- Fetch the DSP binary blob (baseimage.dof mainly) from the TI website,
as well
as the MPEG4 compression cocdecs
(c64xplus_mpeg4enc_02_02_04_production.bin on
the website if I remember). But the .dof and all the .dll64P files into
/lib/dsp in
your Overo rootfs.

- When you boot your Overo, you should see a message stating the successful
initialization of the DSP in the kernel traces, and a /dev/dsp node. If
not, check
if the module is correctly loaded.

- Get the gst-dsp project [2], cross-compile compile and
install it on your Overo. I did native compilation on the Overo.

- Set the pipes according to the doc in [3]. Here is mine:

media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP
CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP
resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'

media-ctl -f '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP
CCDC":2[SGRBG10
752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY
752x480]'

- Set a proper gstreamer pipeline, to capture the image, encode it using the
DSP, put the RTP header and stream it by UDP:

HOST='192.168.1.80'    # adapt this to match your remote computer
PORT=5000

gst-launch v4l2src device=/dev/video6 !
video/x-raw-yuv,width=752,height=480,format='(fourcc)'UYVY ! dspmp4venc !
rtpmp4vpay ! udpsink host=$HOST port=$PORT -v

- On your main computer, run something like this, you should see the stream:

PORT=5000

gst-launch -v udpsrc port=$PORT caps="application/x-rtp,
media=(string)video,
clock-rate=(int)90000, encoding-name=(string)MP4V-ES,
profile-level-id=(string)5,
config=(string)
000001b005000001b50900000100000001200086e0002c352003a00065304ba98517843c1463,
payload=(int)96, ssrc=(guint)285587044, clock-base=(guint)3111327438,
seqnum-base=(guint)60347" ! rtpmp4vdepay  ! ffdec_mpeg4 ! xvimagesink


Cheers,
Florian

[1]
 From 016527d3f47e8ca96edee80b2bb00497d5535379 Mon Sep 17 00:00:00 2001
From: Florian Vaussard <[hidden email]>
Date: Wed, 14 Mar 2012 16:08:00 +0100
Subject: [PATCH] Adding missing ioctl's

---
  drivers/media/video/omap3isp/ispvideo.c |   46
+++++++++++++++++++++++++++++++
  1 files changed, 46 insertions(+), 0 deletions(-)

diff --git a/drivers/media/video/omap3isp/ispvideo.c
b/drivers/media/video/omap3isp/ispvideo.c
index 1452b3a..a15e396 100644
--- a/drivers/media/video/omap3isp/ispvideo.c
+++ b/drivers/media/video/omap3isp/ispvideo.c
@@ -733,6 +733,36 @@ isp_video_get_format(struct file *file, void *fh,
struct
v4l2_format *format)
  }

  static int
+isp_video_enum_format(struct file *file, void *fh, struct v4l2_fmtdesc
*fmtdesc)
+{
+    struct isp_video_fh *vfh = to_isp_video_fh(fh);
+    struct isp_video *video = video_drvdata(file);
+
+    if (fmtdesc->index)
+    {
+//        printk(KERN_ERR "%s: invalid index (%d)\n",__func__,
fmtdesc->index);
+        return -EINVAL;
+    }
+
+    if (fmtdesc->type != video->type)
+    {
+//        printk(KERN_ERR "%s: invalid type (%d). Current type:
%d\n",__func__, fmtdesc->type, video->type);
+        return -EINVAL;
+    }
+
+    // ***Ugly hack***
+    fmtdesc->flags = 0;
+    fmtdesc->description[0] = 'U';
+    fmtdesc->description[1] = 'Y';
+    fmtdesc->description[2] = 'V';
+    fmtdesc->description[3] = 'Y';
+    fmtdesc->description[4] = '0';
+    fmtdesc->pixelformat = V4L2_PIX_FMT_UYVY;
+
+    return 0;
+}
+
+static int
  isp_video_set_format(struct file *file, void *fh, struct v4l2_format
*format)
  {
      struct isp_video_fh *vfh = to_isp_video_fh(fh);
@@ -765,6 +795,8 @@ isp_video_try_format(struct file *file, void *fh, struct
v4l2_format *format)
      u32 pad;
      int ret;

+//    printk(KERN_ERR "%s: Trying format %d\n",__func__,
format->fmt.pix.pixelformat);
+
      if (format->type != video->type)
          return -EINVAL;

@@ -1158,8 +1190,21 @@ isp_video_s_input(struct file *file, void *fh,
unsigned
int input)
      return input == 0 ? 0 : -EINVAL;
  }

+static int
+isp_video_queryctrl(struct file *file, void *fh, struct v4l2_queryctrl
*queryctrl)
+{
+//    printk(KERN_ERR "%s: querry id (%d)\n",__func__, queryctrl->id);
+//    printk(KERN_ERR "%s: querry type (%d)\n",__func__, queryctrl->type);
+
+    if (queryctrl->id < V4L2_CID_BASE)
+        return -EDOM;
+
+    return -EINVAL;
+}
+
  static const struct v4l2_ioctl_ops isp_video_ioctl_ops = {
      .vidioc_querycap        = isp_video_querycap,
+    .vidioc_enum_fmt_vid_cap    = isp_video_enum_format,
      .vidioc_g_fmt_vid_cap        = isp_video_get_format,
      .vidioc_s_fmt_vid_cap        = isp_video_set_format,
      .vidioc_try_fmt_vid_cap        = isp_video_try_format,
@@ -1180,6 +1225,7 @@ static const struct v4l2_ioctl_ops
isp_video_ioctl_ops = {
      .vidioc_enum_input        = isp_video_enum_input,
      .vidioc_g_input            = isp_video_g_input,
      .vidioc_s_input            = isp_video_s_input,
+    .vidioc_queryctrl        = isp_video_queryctrl,
  };

  /*
-----------------------------------------------------------------------------
--
1.7.5.4

[2] https://github.com/felipec/gst-dsp
[3] http://omappedia.org/wiki/Camera-ISP_Driver


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

bhamadicharef
Thank you, i will try this. I look forward to check if this works on our system.

Best regards

Brahim

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Gutembert
In reply to this post by Florian Vaussard
Hi Florian,

I successfully " Streamed video (USB v4l2 WebCam) with Gumstix, using GStreamer and the DSP " but now I would like to broadcast it live using either RTP or RTSP protocol via VLC (url: rtp://Host_IP); or the Web/Mobile phone.

Any help or suggestion is welcome

Thank you in advance,
Gutembert.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

bhamadicharef
Gutembert, could you share the rootfs of this "Streamed video (USB v4l2 WebCam) with Gumstix,
using GStreamer and the DSP" system so we can also try ? which kernel is running on 3.0 or 3.2 ?

Brahim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

Gutembert
Hi Brahim,

Could you please explain what you meant by ' rootfs of this "Streamed video (USB v4l2 WebCam) with Gumstix,
using GStreamer and the DSP" '?
Regarding the kernel, I started with 3.2 but it didn't work and by further research, 2.6.34 was found to be the most suitable kernel to get the DSP and the Camera working. Also, Ubuntu 10.04 LTS seems to be better for OpenEmbedded compare to others (in my OPINION).

Regards,
Gutembert.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Overo COM + Caspa Camera for image processing

zoso1234
I Just bought a Caspa VL and am looking to get into doing image processing as well.

I have been using some TI DSP modules for image processing with some success.

My main question is how hard would it be to get the Caspa to work with 2.6.37 kernel or 39?
The linux-omap_2.6.37 has alot of v4l2 patches and the OMAP3-ISP-driver patch but I don't see anything for the mt9v032. While the 2.6.39 has    file://camera/0002-v4l-Add-mt9v032-sensor-driver.patch  in the recipe..but I don't see much for v4l2 or ISP...I'm assuming it'd be easier to use 2.6.39 but i'd rather use 37. Has anyone had much success with 2.6.37 and Caspa? I did see some old threads regarding 2.6.39 and caspa soo I have something to work off of.
any tips for 2.6.37 would be great
Loading...