diff options
| author | Hans Verkuil <hverkuil@xs4all.nl> | 2017-03-06 11:23:15 -0300 |
|---|---|---|
| committer | Mauro Carvalho Chehab <mchehab@s-opensource.com> | 2017-03-22 10:16:17 -0300 |
| commit | 5086924ad4f219ad6498b2cb217a98637b0b55b9 (patch) | |
| tree | b19234532ffdef9582dc9fbd33a4a53f1cd2e210 /drivers/media/platform/vivid/vivid-vid-cap.c | |
| parent | 733e009c37b6caa219d7057838582d1259c4554f (diff) | |
| download | talos-obmc-linux-5086924ad4f219ad6498b2cb217a98637b0b55b9.tar.gz talos-obmc-linux-5086924ad4f219ad6498b2cb217a98637b0b55b9.zip | |
[media] vivid: fix try_fmt behavior
vivid_try_fmt_vid_cap() called tpg_calc_line_width to calculate the sizeimage
value, but that tpg function uses the current format, not the proposed (tried)
format.
Rewrote this code to calculate this correctly.
The vivid_try_fmt_vid_out() code was completely wrong w.r.t. sizeimage, and
neither did it take the vdownsampling[] factors into account.
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Vincent Abriou <vincent.abriou@st.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Diffstat (limited to 'drivers/media/platform/vivid/vivid-vid-cap.c')
| -rw-r--r-- | drivers/media/platform/vivid/vivid-vid-cap.c | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/drivers/media/platform/vivid/vivid-vid-cap.c b/drivers/media/platform/vivid/vivid-vid-cap.c index a18e6fec219b..01419455e545 100644 --- a/drivers/media/platform/vivid/vivid-vid-cap.c +++ b/drivers/media/platform/vivid/vivid-vid-cap.c @@ -616,7 +616,7 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv, /* This driver supports custom bytesperline values */ mp->num_planes = fmt->buffers; - for (p = 0; p < mp->num_planes; p++) { + for (p = 0; p < fmt->buffers; p++) { /* Calculate the minimum supported bytesperline value */ bytesperline = (mp->width * fmt->bit_depth[p]) >> 3; /* Calculate the maximum supported bytesperline value */ @@ -626,10 +626,17 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv, pfmt[p].bytesperline = max_bpl; if (pfmt[p].bytesperline < bytesperline) pfmt[p].bytesperline = bytesperline; - pfmt[p].sizeimage = tpg_calc_line_width(&dev->tpg, p, pfmt[p].bytesperline) * - mp->height + fmt->data_offset[p]; + + pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) / + fmt->vdownsampling[p] + fmt->data_offset[p]; + memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved)); } + for (p = fmt->buffers; p < fmt->planes; p++) + pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height * + (fmt->bit_depth[p] / fmt->vdownsampling[p])) / + (fmt->bit_depth[0] / fmt->vdownsampling[0]); + mp->colorspace = vivid_colorspace_cap(dev); if (fmt->color_enc == TGP_COLOR_ENC_HSV) mp->hsv_enc = vivid_hsv_enc_cap(dev); |
