From:  Gregory Szorc <gszorc@mozilla.com>
Date:  23 Mar 2017 22:22:43 Hong Kong Time
Newsgroup:  news.mozilla.org/mozilla.tools
Subject:  

Re: Using zstandard for compression

NNTP-Posting-Host:  63.245.214.181


> On Mar 23, 2017, at 06:10, Chris AtLee  wrote:
> 
> Great write-up, thanks for pointing it out!
> 
> I'm really curious about the possible usage for omni.ja, or even for updates themselves. There is work ongoing in bug 641212 to add XZ compression support to our MAR files. I did a quick test, and xz still beats zstd by an additional 10% for complete MARs, and a little less for partial MARs. It probably still makes sense to use xz for the MAR files in that case.

Yes, I would expect xz (uses lzma) to have better compression than zstd in many scenarios. Zstd's big fat sweet spot is use as a non-specialized general purpose algorithm. MARs are extremely optimized for size, so xz being specialized for that use case is fine. Same goes for the other end of the spectrum: speed. For that, LZ4 is probably your bet (at least for now - zstd apparently wants to close that gap).

As I said in the post, I'm not convinced adding zstd to Firefox just for omni.ja makes much sense when brotli is already in Firefox. I do think the topic should be explored more.

> 
>> On 22 March 2017 at 16:52, Gregory Szorc  wrote:
>> ICYMI, I wrote a blog post about zstandard and compression a few weeks ago:
>> 
>> http://gregoryszorc.com/blog/2017/03/07/better-compression-with-zstandard/
>> 
>> tl;dr zstandard is generally amazing and any process currently using other
>> compression algorithms (like zlib, bzip2, and lzma) should seriously
>> consider switching to zstandard for performance reasons. As I state at the
>> end of the post, I used to think zlib was "good enough" for many use cases.
>> But after spending more time with zstandard, it's obvious how inferior
>> other compression algorithms are.
>> _______________________________________________
>> tools mailing list
>> tools@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/tools
>