Wiki@Home Extension

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Wiki@Home Extension

Michael Dale-4
perhaps if people create a lot of voice overs & ~Kens burns~ effects on
commons images with the occasional inter-spliced video clip with lots of
back and forth editing... and we are constantly creating timely
derivatives of these flattened sequences that ~may~ necessitate such a
system.. because things will be updating all the time ...

... but anyway... yea for now will focus on flattening sequences...

Did a basic internal encoder committed in r54340... Could add some
enhancements but lets spec out want we want ;)

Still need to clean up the File:myFile.mp4 situation. Probably store in
a temp location write out a File:myFile.ogg placeholder then once
transcoded swap it in?

Also will hack in adding derivatives to the job queue where oggHandler
is embed in a wiki-article at a substantial lower resolution than the
source version. Will have it send the high res version until the
derivative is created then "purge" the pages to point to the new
location. Will try and have the "download" link still point to the high
res version. (we will only create one or two derivatives... also we
should decide if we want an ultra low bitrate (200kbs or so version for
people accessing Wikimedia on slow / developing country connections)

peace,
michael

Brion Vibber wrote:

> On 7/31/09 6:51 PM, Michael Dale wrote:
>  
>> Want to point out the working prototype of the Wiki@home extension.
>> Presently it focuses on a system for transcoding uploaded media to free
>> formats, but will also be used for "flattening sequences" and maybe
>> other things in the future ;)
>>    
>
> Client-side rendering does make sense to me when integrated into the
> upload and sequencer processes; you've got all the source data you need
> and local CPU time to kill while you're shuffling the bits around on the
> wire.
>
> But I haven't yet seen any evidence that a distributed rendering network
> will ever be required for us, or that it would be worth the hassle of
> developing and maintaining it.
>
>
> We're not YouTube, and don't intend to be; we don't accept everybody's
> random vacation videos, funny cat tricks, or rips from Cartoon
> Network... Between our licensing requirements and our limited scope --
> educational and reference materials -- I think we can reasonably expect
> that our volume of video will always be *extremely* small compared to
> general video-sharing sites.
>
> We don't actually *want* everyone's blurry cell-phone vacation videos of
> famous buildings (though we might want blurry cell-phone videos of
> *historical events*, as with the occasional bit of interesting news
> footage).
>
> Shooting professional-quality video suitable for Wikimedia use is
> probably two orders of magnitude harder than shooting attractive, useful
> still photos. Even if we make major pushes on the video front, I don't
> think we'll ever have the kind of mass volume that would require a
> distributed encoding network.
>
> -- brion
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>  


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wiki@Home Extension

Gregory Maxwell
On Mon, Aug 3, 2009 at 10:56 PM, Michael Dale<[hidden email]> wrote:
> Also will hack in adding derivatives to the job queue where oggHandler
> is embed in a wiki-article at a substantial lower resolution than the
> source version. Will have it send the high res version until the
> derivative is created then "purge" the pages to point to the new
> location. Will try and have the "download" link still point to the high
> res version. (we will only create one or two derivatives... also we
> should decide if we want an ultra low bitrate (200kbs or so version for
> people accessing Wikimedia on slow / developing country connections)
[snip]


So I think there should generally be three versions, a 'very low rate'
suitable for streaming for people without excellent broadband, a high
rate suitable for streaming on good broadband, and a 'download' copy
at full resolution and very high rate.  (The download copy would be
the file uploaded by the user if they uploaded an Ogg)

As a matter of principle we should try to achieve both "very high
quality" and "works for as many people as possible". I don't think we
need to achieve both with one file, so the high and low rate files
could specialize in those areas.


The suitable for streaming versions should have a limited
instantaneous bitrate (non-infinite buf-delay). This sucks for quality
but it's needed if we want streams that don't stall, because video can
easily have >50:1 peak to average rates over fairly short time-spans.
(It's also part of the secret sauce that differentiates smoothly
working video from stuff that only works on uber-broadband).

Based on 'what other people do' I'd say the low should be in the
200kbit-300kbit/sec range.  Perhaps taking the high up to a megabit?

There are also a lot of very short videos on Wikipedia where the whole
thing could reasonably be buffered prior to playback.


Something I don't have an answer for is what resolutions to use. The
low should fit on mobile device screens. Normally I'd suggest setting
the size based on the content: Low motion detail oriented video should
get higher resolutions than high motion scenes without important
details. Doubling the number of derivatives in order to have a large
and small setting on a per article basis is probably not acceptable.
:(

For example— for this
(http://people.xiph.org/~greg/video/linux_conf_au_CELT_2.ogv) low
motion video 150kbit/sec results in perfectly acceptable quality at a
fairly high resolution,  while this
(http://people.xiph.org/~greg/video/crew_cif_150.ogv) high motion clip
looks like complete crap at 150kbit/sec even though it has 25% fewer
pixels. For that target rate rhe second clip is much more useful when
downsampled: http://people.xiph.org/~greg/video/crew_128_150.ogv  yet
if the first video were downsampled like that it would be totally
useless as you couldn't read any of the slides.   I have no clue how
to solve this.  I don't think the correct behavior could be
automatically detected and if we tried we'd just piss off the users.


As an aside— downsampled video needs some makeup sharpening like
downsampled stills will. I'll work on getting something in
ffmpeg2theora to do this.

There is also the option of decimating the frame-rate. Going from
30fps to 15fps can make a decent improvement for bitrate vs visual
quality but it can make some kinds of video look jerky. (Dropping the
frame rate would also be helpful for any CPU starved devices)


Something to think of when designing this is that it would be really
good to keep track of the encoder version and settings used to produce
each derivative, so that files can be regenerated when the preferred
settings change or the encoder is improved. It would also make it
possible to do quick one-pass transcodes for the rate controlled
streams and have the transcoders go back during idle time and produce
better two-pass encodes.

This brings me to an interesting point about instant gratification:
Ogg was intended from day one to be a streaming format. This has
pluses and minuses, but one thing we should take advantage of is that
it's completely valid and well supported by most software to start
playing a file *as soon* as the encoder has started writing it. (If
software can't handle this it also can't handle icecast streams).
This means that so long as the transcode process is at least realtime
the transcodes could be immediately available.   This would, however,
require that the derivative(s) be written to an accessible location.
(and you will likely have to arrange so that a content-length: is not
sent for the incomplete file).

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wiki@Home Extension

Brion Vibber-3
On 8/3/09 9:56 PM, Gregory Maxwell wrote:
[snip]
> Based on 'what other people do' I'd say the low should be in the
> 200kbit-300kbit/sec range.  Perhaps taking the high up to a megabit?
>
> There are also a lot of very short videos on Wikipedia where the whole
> thing could reasonably be buffered prior to playback.
>
>
> Something I don't have an answer for is what resolutions to use. The
> low should fit on mobile device screens.

At the moment the defaults we're using for Firefogg uploads are 400px
width (eg, 400x300 or 400x225 for the most common aspect rations)
targeting a 400kbps bitrate. IMO at 400kbps at this size things don't
look particularly good; I'd prefer a smaller size/bitrate for 'low' and
higher size/bitrate for "medium" qual.


 From sources I'm googling up, looks like YouTube is using 320x240 for
low-res, 480x360 h.264 @ 512kbps+128kbps audio for higher-qual, with
720p h.264 @ 1024Kbps+232kbps audio available for some HD videos.

http://www.squidoo.com/youtubehd

These seem like pretty reasonable numbers to target; offhand I'm not
sure the bitrates used for the low-res version but I think that's with
older Flash codecs anyway so not as directly comparable.

Also, might we want different standard sizes for 4:3 vs 16:9 material?

Perhaps we should wrangle up some source material and run some test
compressions to get a better idea what this'll look like in practice...

> Normally I'd suggest setting
> the size based on the content: Low motion detail oriented video should
> get higher resolutions than high motion scenes without important
> details. Doubling the number of derivatives in order to have a large
> and small setting on a per article basis is probably not acceptable.
> :(

Yeah, that's way tougher to deal with... Potentially we could allow some
per-file tweaks of bitrates or something, but that might be a world of
pain. :)

> As an aside— downsampled video needs some makeup sharpening like
> downsampled stills will. I'll work on getting something in
> ffmpeg2theora to do this.

Woohoo!

> There is also the option of decimating the frame-rate. Going from
> 30fps to 15fps can make a decent improvement for bitrate vs visual
> quality but it can make some kinds of video look jerky. (Dropping the
> frame rate would also be helpful for any CPU starved devices)

15fps looks like crap IMO, but yeah for low-bitrate it can help a lot.
We may wish to consider that source material may have varying frame
rates, most likely to be:

15fps - crappy low-res stuff found on internet :)
24fps / 23.98 fps - film-sourced
25fps - PAL non-interlaced
30fps / 29.97 fps - NTSC non-interlaced or many computer-generated vids
50fps - PAL interlaced or PAL-compat HD native
60fps / 59.93fps - NTSC interlaced or HD native

And of course those 50 and 60fps items might be encoded with or without
interlacing. :)

Do we want to normalize everything to a standard rate, or maybe just cut
50/60 to 25/30?

(This also loses motion data, but not as badly as decimation to 15fps!)

> This brings me to an interesting point about instant gratification:
> Ogg was intended from day one to be a streaming format. This has
> pluses and minuses, but one thing we should take advantage of is that
> it's completely valid and well supported by most software to start
> playing a file *as soon* as the encoder has started writing it. (If
> software can't handle this it also can't handle icecast streams).
> This means that so long as the transcode process is at least realtime
> the transcodes could be immediately available.   This would, however,
> require that the derivative(s) be written to an accessible location.
> (and you will likely have to arrange so that a content-length: is not
> sent for the incomplete file).

Ooooh, good points all. :D Tricky but not impossible to implement.

-- brion

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wiki@Home Extension

Gregory Maxwell
On Tue, Aug 4, 2009 at 7:46 PM, Brion Vibber<[hidden email]> wrote:
[snip]
> These seem like pretty reasonable numbers to target; offhand I'm not
> sure the bitrates used for the low-res version but I think that's with
> older Flash codecs anyway so not as directly comparable.

I'd defiantly recommend using youtube numbers as a starting point...
Though I think there may be an argument for going lower on the low end
(because we think we care more about compatibility than even they do)
and higher on the high end (because for at least some of our material
decent quality will be important)


> Also, might we want different standard sizes for 4:3 vs 16:9 material?

Yes, probably.

> Perhaps we should wrangle up some source material and run some test
> compressions to get a better idea what this'll look like in practice...

http://media.xiph.org/  < There is something like 100gigs of lossless
test material. Though most of it has sufficiently ambiguous licensing
that we wouldn't want to toss it up on commons. Many of them are
atypically difficult to compress, and almost all are too short for
decent testing of buffering/rate control. But they're still useful.

> Yeah, that's way tougher to deal with... Potentially we could allow some
> per-file tweaks of bitrates or something, but that might be a world of
> pain. :)

Tweaks of resolution are more important than bitrate, in that some
content just needs higher resolutions... (and if we offer a bitrate
tweak we'll probably see everyone with a good net connection turning
it up and everyone with a poor net connection turning it down).

> 15fps looks like crap IMO, but yeah for low-bitrate it can help a lot.
> We may wish to consider that source material may have varying frame
> rates, most likely to be:

Low rate video sucks no matter how you cut it. We just get to pick how
it sucks.

Sadly the best choice depends on the content.

> 50fps - PAL interlaced or PAL-compat HD native
> 60fps / 59.93fps - NTSC interlaced or HD native
> And of course those 50 and 60fps items might be encoded with or without
> interlacing. :)
> Do we want to normalize everything to a standard rate, or maybe just cut
> 50/60 to 25/30?

So, interlacing is a non-issue on the output side: No interlacing
support in Theora. (Compression of interlaced video is an extra
special patent minefield;  We'd want to deinterlace anyways, to
unburden the client). The transcoding tools will deinterlace.

I'd recommend cutting 50/60 to 25/30, deinterlacing as required.
Usually 60fps content isn't really displayed as that on PCs anyways
because of the synchronization between frame update and video readout.

(Deinterlacing also brings up the question of: Do we want to use
computationally expensive motion-compensated deinterlacing.  I could
argue "Interlaced content will be rare so the increase would be
harmless and it looks a little better" or "it's not worth the enormous
CPU usage increase for a small quality boost on content which will be
rare")

Non-integer ratio rate conversions don't look good. Whatever we do I
think we should only do small integer ratios, i.e. 1:1, 2:1, 3:1, in
order to get the rate at or under 30fps.    We certainly want to allow
low frame rates: They'd make a vastly superior replacement to the
enormous animated GIFs in articles, much smaller, easier to produce,
and higher quality.


There is some 50/60p content in the media collection I linked to, so
you see how that looks.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Video Quality for Derivatives (was Re:Wiki@Home Extension)

Michael Dale-4
In reply to this post by Brion Vibber-3
So I committed ~basic~ derivate code support for oggHandler in r54550
(more solid support on the way)

Based input from the wiki@home thread;  here are updated target
qualities expressed via the firefogg api to ffmpeg2thoera

Also j^ was kind enough to run these settings on some sample input files:
http://firefogg.org/j/encoding_samples/ so you can check them out there.

We want to target 400 wide for the "web stream" to be consistent with
archive.orgs which encodes mostly to 400x300 (although their 16:9 stuff
can be up to 530 wide) ...

Updated mediaWiki firefogg integration and the stand alone encoder app
these default transcode settings. in r54552 & r54554 (should be pushed
out to http://firefogg.org/make shortly ... or can be run @home with a
trunk check out at:
/js2/mwEmbed/example_usage/Firefogg_Make_Advanced.html

anyway on to the settings:

$wgDerivativeSettings[ WikiAtHome::ENC_SAVE_BANDWITH ] =
        array(
            'maxSize'        => '200',
            'videoBitrate'    => '164',
            'audioBitrate'    => '32',
            'samplerate'    => '22050',
            'framerate'        => '15',
            'channels'        => '1',        
            'noUpscaling'    => 'true'
        );
$wgDerivativeSettings[ WikiAtHome::ENC_WEB_STREAM ] =
        array(
            'maxSize'        => '400',
            'videoBitrate'    => '544',
            'audioBitrate'    => '96',
            'noUpscaling'    => 'true'
        );
$wgDerivativeSettings[ WikiAtHome::ENC_HQ_STREAM ] =
        array(
            'maxSize'         => '1080',
            'videoQuality'    => 6,
            'audioQuality'    => 3,
            'noUpscaling'    => 'true'
        );

--michael


Brion Vibber wrote:

> On 8/3/09 9:56 PM, Gregory Maxwell wrote:
> [snip]
>  
>> Based on 'what other people do' I'd say the low should be in the
>> 200kbit-300kbit/sec range.  Perhaps taking the high up to a megabit?
>>
>> There are also a lot of very short videos on Wikipedia where the whole
>> thing could reasonably be buffered prior to playback.
>>
>>
>> Something I don't have an answer for is what resolutions to use. The
>> low should fit on mobile device screens.
>>    
>
> At the moment the defaults we're using for Firefogg uploads are 400px
> width (eg, 400x300 or 400x225 for the most common aspect rations)
> targeting a 400kbps bitrate. IMO at 400kbps at this size things don't
> look particularly good; I'd prefer a smaller size/bitrate for 'low' and
> higher size/bitrate for "medium" qual.
>
>
>  From sources I'm googling up, looks like YouTube is using 320x240 for
> low-res, 480x360 h.264 @ 512kbps+128kbps audio for higher-qual, with
> 720p h.264 @ 1024Kbps+232kbps audio available for some HD videos.
>
> http://www.squidoo.com/youtubehd
>
> These seem like pretty reasonable numbers to target; offhand I'm not
> sure the bitrates used for the low-res version but I think that's with
> older Flash codecs anyway so not as directly comparable.
>
> Also, might we want different standard sizes for 4:3 vs 16:9 material?
>
> Perhaps we should wrangle up some source material and run some test
> compressions to get a better idea what this'll look like in practice...
>
>  
>> Normally I'd suggest setting
>> the size based on the content: Low motion detail oriented video should
>> get higher resolutions than high motion scenes without important
>> details. Doubling the number of derivatives in order to have a large
>> and small setting on a per article basis is probably not acceptable.
>> :(
>>    
>
> Yeah, that's way tougher to deal with... Potentially we could allow some
> per-file tweaks of bitrates or something, but that might be a world of
> pain. :)
>
>  
>> As an aside— downsampled video needs some makeup sharpening like
>> downsampled stills will. I'll work on getting something in
>> ffmpeg2theora to do this.
>>    
>
> Woohoo!
>
>  
>> There is also the option of decimating the frame-rate. Going from
>> 30fps to 15fps can make a decent improvement for bitrate vs visual
>> quality but it can make some kinds of video look jerky. (Dropping the
>> frame rate would also be helpful for any CPU starved devices)
>>    
>
> 15fps looks like crap IMO, but yeah for low-bitrate it can help a lot.
> We may wish to consider that source material may have varying frame
> rates, most likely to be:
>
> 15fps - crappy low-res stuff found on internet :)
> 24fps / 23.98 fps - film-sourced
> 25fps - PAL non-interlaced
> 30fps / 29.97 fps - NTSC non-interlaced or many computer-generated vids
> 50fps - PAL interlaced or PAL-compat HD native
> 60fps / 59.93fps - NTSC interlaced or HD native
>
> And of course those 50 and 60fps items might be encoded with or without
> interlacing. :)
>
> Do we want to normalize everything to a standard rate, or maybe just cut
> 50/60 to 25/30?
>
> (This also loses motion data, but not as badly as decimation to 15fps!)
>
>  
>> This brings me to an interesting point about instant gratification:
>> Ogg was intended from day one to be a streaming format. This has
>> pluses and minuses, but one thing we should take advantage of is that
>> it's completely valid and well supported by most software to start
>> playing a file *as soon* as the encoder has started writing it. (If
>> software can't handle this it also can't handle icecast streams).
>> This means that so long as the transcode process is at least realtime
>> the transcodes could be immediately available.   This would, however,
>> require that the derivative(s) be written to an accessible location.
>> (and you will likely have to arrange so that a content-length: is not
>> sent for the incomplete file).
>>    
>
> Ooooh, good points all. :D Tricky but not impossible to implement.
>
> -- brion
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Video Quality for Derivatives (was Re:Wiki@Home Extension)

Gregory Maxwell
On Thu, Aug 6, 2009 at 8:00 PM, Michael Dale<[hidden email]> wrote:
> So I committed ~basic~ derivate code support for oggHandler in r54550
> (more solid support on the way)
>
> Based input from the wiki@home thread;  here are updated target
> qualities expressed via the firefogg api to ffmpeg2thoera

Not using two-pass on the rate controlled versions?

It's a pretty consistent performance improvement[8], and it eliminates
the "first frame blurry" issue that sometimes comes up for talking
heads. (Note, that by default two-pass cranks the keyframe interval to
256 and makes the buf-delay infinite. So you'll need to set those to
sane values for streaming).


[1] For example:
http://people.xiph.org/~maikmerten/plots/bbb-68s/managed/psnr.png

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Video Quality for Derivatives (was Re:Wiki@Home Extension)

Gregory Maxwell
On Thu, Aug 6, 2009 at 8:17 PM, Gregory Maxwell<[hidden email]> wrote:

> On Thu, Aug 6, 2009 at 8:00 PM, Michael Dale<[hidden email]> wrote:
>> So I committed ~basic~ derivate code support for oggHandler in r54550
>> (more solid support on the way)
>>
>> Based input from the wiki@home thread;  here are updated target
>> qualities expressed via the firefogg api to ffmpeg2thoera
>
> Not using two-pass on the rate controlled versions?
>
> It's a pretty consistent performance improvement[8], and it eliminates
> the "first frame blurry" issue that sometimes comes up for talking
> heads. (Note, that by default two-pass cranks the keyframe interval to
> 256 and makes the buf-delay infinite. So you'll need to set those to
> sane values for streaming).

I see r54562 switching to two-pass, but as-is this will produce files
which are not really streamable (because they streams can and will
burst to >10mbits even though the overall rate is 500kbit or whatever
is requested).

We're going to want to do something like -k 64 --buf-delay=256.

I'm not sure what key-frame interval we should be using— Longer
intervals lead to clearly better compression, with diminishing returns
over 512 or so depending on the content... but lower seeking
granularity during long spans without keyframes.  The ffmpeg2theora
defaults are 64 in one-pass mode, 256 in two-pass mode.

Buf-delay indicates the amount of buffering the stream is targeting.
I.e. For a 30fps stream at 100kbit/sec a buf-delay of 60 means that
the encoder expects that the decoder will have buffered at least
200kbit (25kbyte) of video data before playback starts.

If the buffer runs dry the playback stalls— pretty crappy for the
user's experience.  So bigger buff delays either mean a longer
buffering time before playback or more risk of stalling.

In the above (30,60,100) example the client would require 2 seconds to
fill the buffer if they were transferring at 100kbit/sec, 1 second if
they are transferring at 200kbit/sec. etc.

The default is the same as the keyframe interval (64) in one pass
mode, and infinite in two-pass mode.  Generally you don't want the
buf-delay to be less than the keyframe interval, as quality tanks
pretty badly at that setting.

Sadly the video tag doesn't currently provide any direct way to
request a minimum buffering. Firefox just takes a guess and every time
it stalls it guesses more. Currently the guesses are pretty bad in my
experience, though this is something we'll hopefully get addressed in
future versions.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
123