Plugin API extension - specify the compression for decoder

[bradh]

As part of the nvidia hardware decoder implementation, I have the case where a decoder can potentially handle multiple compression types (e.g. AV1, HEIC, AVC), depending on runtime capabilities - effectively which video card is used.

That works for the does_support_format design - I can query the hardware.

However when it actually gets around to decoding, I'll need to know what compression type is being used. That is used to set up the decoder before processing data. Options would include a new_decoder2 which takes a compression type, or a set_compression_type plugin API extension. Thoughts on that?

[farindk]

Yes, I think that new_decoder2 would be the right place. Maybe we can pass some (versioned) decoder options object to be able to pass more options in the future. I'm not sure yet what kind of options that may be for a decoder, but still.

[bradh]

Yes, I think that new_decoder2 would be the right place. Maybe we can pass some (versioned) decoder options object to be able to pass more options in the future. I'm not sure yet what kind of options that may be for a decoder, but still.

So something like:

--- a/libheif/api/libheif/heif_plugin.h
+++ b/libheif/api/libheif/heif_plugin.h
@@ -34,15 +34,15 @@ extern "C" {
 
 // API versions table
 //
-// release    decoder   encoder   enc.params
-// -----------------------------------------
-//  1.0          1        N/A        N/A
-//  1.1          1         1          1
-//  1.4          1         1          2
-//  1.8          1         2          2
-//  1.13         2         3          2
-//  1.15         3         3          2
-
+// release    decoder  dec.config  encoder   enc.params
+// -----------------------------------------------------
+//  1.0          1        N/A        N/A        N/A
+//  1.1          1        N/A         1          1
+//  1.4          1        N/A         1          2
+//  1.8          1        N/A         2          2
+//  1.13         2        N/A         3          2
+//  1.15         3        N/A         3          2
+//  1.19         4         1          3          2
 
 // ====================================================================================================
 //  Decoder plugin API
@@ -50,6 +50,14 @@ extern "C" {
 //  added as plugins. A plugin has to implement the functions specified in heif_decoder_plugin
 //  and the plugin has to be registered to the libheif library using heif_register_decoder().
 
+struct heif_decoder_configuration
+{
+  int version; // current version: 1
+
+  // --- version 1 fields ---
+  heif_compression_format compression_format;
+};
+
 struct heif_decoder_plugin
 {
  ...
 
-  // --- version 4 functions will follow below ... ---
+  // --- version 4 functions follow below ... ---
+
+  // Create a new decoder context for decoding an image
+  struct heif_error (* new_decoder2)(void** decoder, heif_decoder_configuration *compression_format);
+
+  // --- version 5 functions will follow below ... ---
 };

[farindk]

Yes. I think that in theory, we could even leave away the version field as the plugin object already has a version and libheif will always send options matching the plugin version, but it is probably better to tell this explicitly. What do you think?
The versionin heif_encoder_parameter also wouldn't have been necessary, but it's too late to remove that.

[bradh]

I think explicit is better, and potentially the plugin version could change independently of the configuration version, although as you noted - its not clear that there will ever be a lot of configuration options for decode side - maybe a GPU number if you have more than one.

I also thought about treating it as a flags value instead of an integer, but it looked like more complexity than we need.