F3nd0
It seems to me that AV1 (the codec used in AVIF) has seen a lot more development in recent years, by virtue of being widely adopted for video. JPEG XL has not seen comparably massive adoption, perhaps owing in large part to Chrome rejecting it on dubious grounds (in spite of eagerly forcing the adoption of both WebP and AVIF earlier on). The pace of the reference implementation’s development has grown very slow, and most of the modest attention it’s been getting seems to have recently moved to the new Rust decoder, once the browsers finally made up their mind that’s what they wanted from the devs.
That considered, I don’t think it’s a fair comparison between the codecs. JPEG XL used to be far more impressive than AVIF; it’s great people (author included) have managed to push AVIF forward, but until similar efforts have been made for JPEG XL in earnest, I don’t feel like a comparison between their encoders says much about the codecs themselves. The author does take a guess on how much the JXL encoder could be improved, but that’s just that: a guess. It won’t be know until it’s been tried. And especially given the whole unfortunate history, I think JPEG XL really, really deserves a try.
computerbusterreply
It is getting its try, actively, in libjxl. People like to pretend AV1 got infinite resources; the reality is myself and one other contributor produced the vast majority of the image gains. I built Iris-WebP and aperture-alpha myself, from scratch. As a compression engineer, I think JXL is way, way harder to work with, and it would've taken me a lot longer. libjxl has community contributors, it is just an uphill battle with a codec like that. Same as WebP is an uphill battle due to its format restrictions.
F3nd0reply
> It is getting its try, actively, in libjxl.
That’s the thing, though: libjxl development hasn’t seemed all that active in recent years. The community contributors you mention seem quite far from driving the development, and some of them say that the usual devs have been busy with other projects (including jxl-rs as of recently).
> People like to pretend AV1 got infinite resources
I would not suggest that, but in another comment here you yourself say that SVT-AV1 (which you compare to JXL favourably in your article) has seen active involvement from ‘Meta, Netflix, Intel, independent contractors, and others’ – not to the community fork SVT-AV1-PSY and its successors contributing their improvements back upstream.
I can believe that JXL may very well be more difficult to work with, but am I wrong to assume you’ve been able to continuously focus on your own encoder (and apply your experiences from your earlier work on SVT-AV1)?
computerbusterreply
I think libjxl's development is stalled because the format is hard to work with. It wasn't super hard to drive meaningful improvements to AVIF.
Yes, SVT-AV1 received and continues to receive development efforts from devs at big companies, but the number of core contributors has always been somewhat small. Definitely more resources, but the entirety of the original AVIF work was done by two people.
I'm able to utilize my experiences generally in image coding to work on my encoders. This should translate to JPEG XL, but I feel held back by how algorithmically complex compelling implementations of the coding tools would be, and how to make those implementations fast. I think if the JPEG XL spec was incredibly intuitive, community contributions would have gotten it a lot further. Heck, my own efforts may have gone to it instead of SVT-AV1-PSY's AVIF encoding.
a-french-anonreply
Yes, but for speed which means a lot of SIMD plumbing, the larger workforce certainly benefited AV1 codecs!
ZeroGravitasreply
Wasn't that also basically one guy from the VLC/ffmpeg community that drove that effort? I think they got funding from Netflix to do it again on ARM after demonstrating the benefits on desktop.
bawolffreply
> It won’t be know until it’s been tried
We can reevaluate when and if improvements are made. The problem with statements like this, is its really easy to miss the showstoppers when imagining what something could be. All plans are amazing until you have to actually do them in the real world.
As the saying goes: Never fall in love with potential.
izacusreply
"We don't want to support another pile of security bug ridden C++ library" absolutely aren't "dubious grounds".
You'd rip a new one to Google if there would be a CVE in a new C++ library in Chrome because of it.
Now that Rust library is available, they will continue adoption, as it should be.
account42reply
> "We don't want to support another pile of security bug ridden C++ library" absolutely aren't "dubious grounds".
It is when that didn't stop them YOLO'ing in webp and then avif support.
Daiz
A potentially major issue I have with AVIF is that because it is based on a video format, any hardware decoding support AVIF will get is likely to be restrained to common video scenarios. This can result in eg. only 4:2:0 YUV being supported by hardware decoders, as that's the upper limit of AV1 Main Profile (and thus the limit of AVIF Baseline Profile). And 4:2:0 is just a poor fit for certain categories of images (like various kinds of illustrations, screenshots, etc). I would not be very happy about a web where lossy 4:4:4 image compression would be heavily discouraged with modern formats.
Also, good lossless compression absolutely does matter for the web as well. Lossy images just cannot be used in all circumstances - like when comparing compression quality (especially of videos), for example!
EDIT: To give an additional example, pixel art is an entire field of art (which is very much still active today) where both lossless compression is a must and where 4:2:0 would be absolutely catastrophic for quality.
torginusreply
I'm not super familiar with AV1/AVIF but I do have extensive experience using h264 for desktop streaming, and all major implementations support 444 in hardware and software, and I doubt more advanced codecs/image formats have a limitation like this. Asked ChatGPT and it also claimed there's no problem with 444 and AVIF/AV1.
lgkrnkglwnreply
Then chatgpt missed an obvious issue, nvidia definitely have had shortcomings in the hardware implementation of coder/decoders only supporting 420 in nvenc/nvdec. One obvious problem with that was that it affected premiere.
torginusreply
We have been using NVENC with 444 and h264 in production going back years and HW as old as Turing.
But it turns out you are right, 444 decoding with H264 doesn't seem to be supported up until recently:
https://developer.nvidia.com/video-encode-decode-support-mat...
Never noticed, seems like a strange omission, but in any case doesn't seem to apply to newer codecs like AV1.
jaffathecakereply
I'm pretty sure 444 h264 fails to decode on iOS Safari due to poor support in the hardware decoder.
MrSqueezlesreply
AVIF's support for non-photographic images is covered in the article. It supports lossless.
Daizreply
Yes, I know the format supports it. The question is, will hardware decoders built primarily around video use cases? And if those kind of hardware decoders end up forming the majority of hardware decoders for AVIF, will we end up in a future where using the full range of the format is heavily discouraged because it won't be supported by hardware decoders?
For comparison, it was possible to do 10-bit video with H.264, but 10-bit video didn't see any mainstream adoption with it because hardware decoders generally didn't support 10-bit H.264 video. Only when newer formats came around and specced 10-bit support as a baseline necessity did we start to see wider adoption for it. And well, the baseline specs (main profile) for AV1, which AVIF is based on, are limited to 4:2:0...
edflsafoiewqreply
AFAIK WebP has never used hardware decoding and no browser uses it for AVIF now.
account42reply
It's not lossless if you have to convert to YUV first.
jaffathecakereply
You are correct. However, AVIF does not require this conversion. For lossless it supports an "identity" matrix coefficient, which means no conversion.
It isn't particularly efficient in terms of file size, but as others have said, lossless images within a web page is extremely niche.
miladyincontrolreply
Wait, DAIZ!? I genuinely did not expect to see you here chiming in. Still appreciate much of your 10 bit anime advocacy from way back.
aniviacat
> Progressive rendering (which AVIF supports) decodes a low-fidelity rendition before the full image arrives. AVIF didn't support progressive rendering for a while, and during that time I believe it was deeply oversold. Now that libavif has implemented it (it was always possible), the conversation appears to be over.
This is false. AVIF does not support progressive decoding/rendering. What the author is referring to here is image layering. A lower resolution image layered below the full resolution image, which is loaded and rendered first. That is not progressive decoding/rendering. It is a thumbnail.
Unlike AVIF's false advertising, JXL truly supports progressive decoding/rendering. With JXL, you do not first load a thumbnail before loading the actual image. The lower-quality image that JXL shows while decoding is derived from the data of the actual, full-resolution image.
computerbusterreply
Not sure if the difference is materially relevant to UX at all. JPEG XL achieves progressive rendering at a great cost to its selection of coding tools, so I side with AVIF's approach.
MiroslavPokornyreply
How does progressive rendering actually that most OS GUIS are not multithreaded ?
The preview in finder is still going to block. Im saying this as a MAC user speaking of experience browsing mounted drives that sometimes are slow and it feels like the window freezes up.
wmfreply
Progressive rendering is intended for the Web and it does work.
brigadereply
Thumbnailing happens in a background process and shouldn’t block the main thread; rather hangs are usually because fetching xattr over SMB is slow and Finder will block until it’s enumerated the metadata of every file in a folder.
Preview actually does use progressive rendering for large enough images, even from SSDs. You can tell because opening an image will be blurry for several seconds, even when the full size image could have been decoded in a quarter second or less.
brigadereply
You're wrong. AVIF does support spatial layering where the full-resolution image is derived from the lower-resolution layer, and cannot be decoded independently.
Yes, the format is not inherently progressive. But having the fine steps in progressive quality that JPEG-2000 and JPEG-XL have has less general usefulness than the amount of words complaining about it.
F3nd0reply
Can a similar technique be employed in JPEG XL, or does the format not allow for it in any way?
lgkrnkglwnreply
Jpeg2000 (and jpeg ls and jpeg xr) thankfully wasn’t implemented in any major way. Keep it simple, stupid.
nneonneo
That JPEG XL prime computation is a pretty ugly DoS. Just selecting it in Finder, with the preview pane open, maxed out every core on my Mac inside a QuickLookSatellite that also ate 4GB of RAM while doing so - for a good 15 seconds. Not bad for a 2KB picture. It seems like Apple did not set sane limits on their JXL previewer.
It is, however, an incredibly cool demo of what the format is capable of. I'm not completely sure if an image format should be that flexible, but I'm impressed nonetheless.
estreply
I wonder if similar hacks apply to zlib and .png as well.
nneonneoreply
Probably not; for PNG, the image size is declared in the header, so a decoder can decide immediately if it wants to decode the image or not. The output is bounded by the size of the image times the bit depth, and decompression runs in time proportional to output size. zlib bombs exist, but they don't affect png because a decoder can simply refuse to decompress past the size of the pixel buffer.
tacomagickreply
The infamous PIL DOS errors, rooting from the library refusing to process images larger than a hard limit. Does this mean though JXL does not have that data accesible quickly?
wmfreply
Zip and PNG bombs have been around for a while: https://github.com/0x48piraj/gz-bomb https://libpng.sourceforge.io/decompression_bombs.html
estreply
I mean "mild" bombs where the final output bytes looks normal, a regular sized image, but the decompressing steps takes unnecessarily long, eating CPU or/and RAM
erureply
You can also have a zip that decompresses to itself: https://research.swtch.com/zip
dylan604reply
When a format has so much flexibility, the world will glom onto the first thing that it solves and put it into the wild to solve that problem. The rest of the capabilities fall by the way side, yet not removed from the format. They're just ignored. MP4 can do so much more than the typical deliverable of a video stream and an audio stream. The spec allows for multiple video streams, multiple audio streams, subtitles, Flash like interactivity to allow self contained DVD style programming of menus to allow for chapter navigation, audio/sub selection, multiangle, etc.
fc417fc802reply
> MP4 can do so much more
None of that has fallen by the wayside though? I have seen examples of all of those in the wild except for interactive menus and multiangle. Multiple video streams is incredibly rare to encounter but I have run into it a few times.
The argument in favor of flexibility (and jxl) is that if you optimize things for the "average" web user (as the essay seems to be suggesting) then fairly mundane usecases require you to start juggling formats, support becomes spotty, and things start breaking. It's nice to have generous limits within which you can be confident that things will "just work" for the end user. Even just on my own system I'd much rather use a single format rather than dealing with app x not supporting format y.
tl;dr jxl is the mp4 of image formats and that's exactly why I like it.
The only things I agree with the essay about are progressive decoding and decoding speed. Particularly the latter badly needs to be improved.