[0/2] Support compressed firmwares

  Fedora 34 only provides compressed firmwares.

Introduce an internal driver helper to handle transparently compression.

I chose libarchive for decompressing as it seems widely available and
DPDK had used it in the past.

Windows support only matters for net/ice and firmware loading was skipped
in this driver before this series. Since I don't know if/how we want to
load firmwares on Windows, I let an empty stub for this OS.

This series has been compile tested on Linux (I'll trust the CI for
others OSes).
I only tested basic init with a net/ice device (no DCF test).

So please drivers maintainers, check nothing is broken.
  

> Fedora 34 only provides compressed firmwares.
> 
> Introduce an internal driver helper to handle transparently compression.
> 
> I chose libarchive for decompressing as it seems widely available and
> DPDK had used it in the past.
> 
> Windows support only matters for net/ice and firmware loading was skipped
> in this driver before this series. Since I don't know if/how we want to
> load firmwares on Windows, I let an empty stub for this OS.
> 
> This series has been compile tested on Linux (I'll trust the CI for
> others OSes).
> I only tested basic init with a net/ice device (no DCF test).
> 
> So please drivers maintainers, check nothing is broken.

Hi David,

We (Marvell QED) already provide packed version of FW in linux-firmware:
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/qed 

But thats a custom name, and zlib.

I'm just wondering if its a good solution to try transparently load .xz variant?
User may not expect that. M.b. through this api, give a driver an option to specify
archive format? Or even autodetect it from content?

Regards,
  Igor
  

On Wed, Jun 2, 2021 at 12:38 PM Igor Russkikh <irusskikh@marvell.com> wrote:
> We (Marvell QED) already provide packed version of FW in linux-firmware:
> https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/qed
>
> But thats a custom name, and zlib.

Whatever binary blob is available in linux-firmware upstream repo, it
ends up compressed on Fedora 34 (and later).

# fc33
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115649&filename=/usr/lib/firmware/qed/qed_init_values-8.10.9.0.bin
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115649&filename=/usr/lib/firmware/qed/qed_init_values_zipped-8.10.10.0.bin
# fc34
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115322&filename=/usr/lib/firmware/qed/qed_init_values-8.10.9.0.bin.xz
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115322&filename=/usr/lib/firmware/qed/qed_init_values_zipped-8.10.10.0.bin.xz
# fc35
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115234&filename=/usr/lib/firmware/qed/qed_init_values-8.10.9.0.bin.xz
https://koji.fedoraproject.org/koji/fileinfo?rpmID=26115234&filename=/usr/lib/firmware/qed/qed_init_values_zipped-8.10.10.0.bin.xz

Did you try the qede pmd on fc34?


>
> I'm just wondering if its a good solution to try transparently load .xz variant?

The linux kernel (since v5.2, I think) uncompresses this transparently
without kernel drivers knowing.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=82fd7a8142a10b8eb41313074b3859d82c0857dc


> User may not expect that. M.b. through this api, give a driver an option to specify
> archive format? Or even autodetect it from content?

User should (do ?) not care about firmwares, only drivers do.

If not doing transparently, each driver will have to implement (or ask
a common helper for) support of compressed blobs.
While this issue is common to all drivers.

As for autodetecting compression, libarchive can do this, I added only
xz support because this is the only supported compression in the Linux
kernel loader.
  

On 6/2/2021 1:05 PM, David Marchand wrote:
> On Wed, Jun 2, 2021 at 12:38 PM Igor Russkikh <irusskikh@marvell.com> wrote:
>> We (Marvell QED) already provide packed version of FW in linux-firmware:

>>
>> But thats a custom name, and zlib.
> 
> Whatever binary blob is available in linux-firmware upstream repo, it
> ends up compressed on Fedora 34 (and later).
> 
> Did you try the qede pmd on fc34?

Got it, thanks.
No, have not tried, but think qede pmd will fail now on fedora without your patch.

> The linux kernel (since v5.2, I think) uncompresses this transparently
> without kernel drivers knowing.

Was not aware of that, thanks.
In that perspective your patch fits good with the existing linux kernel behavior, agree.

Adding more people from my team.

Regards,
  Igor