Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oss-fuzz reports stack overflow in libheif #1336

Open
bobfriesenhahn opened this issue Oct 13, 2024 · 12 comments
Open

oss-fuzz reports stack overflow in libheif #1336

bobfriesenhahn opened this issue Oct 13, 2024 · 12 comments

Comments

@bobfriesenhahn
Copy link

As part of GraphicsMagick oss-fuzz fuzz testing, oss-fuzz has detected an unusual stack overflow in latest libheif code.

The issue will eventually be visible to all at graphicsmagick:coder_AVIF_fuzzer: Stack-overflow in ImageItem_Grid::get_decoder.

The report by oss-fuzz is not very conclusive, but one of the stack traces appears to show that ImageItem_Grid::get_decoder() is making a recursive call and has called itself 240 times. It is possible that additional context due to AddressSanitizer might limit how much can be put on the stack.

This is the file which provokes the issue (with .jpg added to satisfy github):

clusterfuzz-testcase-minimized-coder_AVIF_fuzzer-5900685314424832

@farindk
Copy link
Contributor

farindk commented Oct 13, 2024

I cannot reproduce this. I've tested at several commits.
Can you include which libheif commit was used for this fuzzer run?

@bobfriesenhahn
Copy link
Author

Git clones of libheif are reporting recent revisions de48209f29be, 2f470644d34e, and 2595e649e943, but I am not sure how these correspond to test reports.

The job metadata says this:

[2024-10-10 10:38:43 UTC] oss-fuzz-linux-zone1-host-8r5t-15: Fuzz task : Fuzzer libFuzzer_graphicsmagick_coder_AVIF_fuzzer generated testcase crashed in 191.0 seconds (r202410090607).
[2024-10-10 10:41:35 UTC] oss-fuzz-linux-zone8-host-93q7-5: Minimize task started.
[2024-10-10 11:06:53 UTC] oss-fuzz-linux-zone8-host-93q7-11: Minimize task started.
[2024-10-10 11:37:52 UTC] oss-fuzz-linux-zone8-host-93q7-15: Minimize task started.
[2024-10-10 12:28:20 UTC] oss-fuzz-linux-zone8-host-93q7-15: Minimize task errored out: LibFuzzer minimization failed.
[2024-10-10 12:34:39 UTC] oss-fuzz-linux-zone3-host-94sr-5: Regression task started.
[2024-10-10 13:10:08 UTC] oss-fuzz-linux-zone8-host-96kn-4: Regression task started.
[2024-10-10 13:45:38 UTC] oss-fuzz-linux-zone7-host-9fxs-11: Regression task started.
[2024-10-10 19:16:24 UTC] oss-fuzz-linux-zone7-host-9fxs-5: Regression task started.
[2024-10-10 19:52:43 UTC] oss-fuzz-linux-zone7-host-9fxs-12: Regression task started.
[2024-10-10 23:34:40 UTC] oss-fuzz-linux-zone5-host-904f-5: Regression task started.
[2024-10-11 06:58:50 UTC] oss-fuzz-linux-zone5-host-904f-5: Regression task finished: regressed in range 202310160611:202312060606.
[2024-10-13 08:13:45 UTC] oss-fuzz-linux-zone3-host-94sr-2: Progression task started.
[2024-10-13 08:13:50 UTC] oss-fuzz-linux-zone3-host-94sr-2: Progression task finished: Still crashes on latest revision r202410090607.

and it says that the same reproducer creates crashes for four different job types.

afl_asan_graphicsmagick - Stack-overflow
honggfuzz_asan_graphicsmagick - Stack-overflow
libfuzzer_msan_graphicsmagick - Null-dereference READ
libfuzzer_ubsan_graphicsmagick - Null-dereference

@farindk
Copy link
Contributor

farindk commented Oct 13, 2024

Git clones of libheif are reporting recent revisions de48209f29be, 2f470644d34e, and 2595e649e943

These are no libheif commit IDs.

Let's postpone this one a bit. I am currently doing a lot of internal restructuring. Thus, changes are high that it was just an intermediate commit that had this issue.

@bobfriesenhahn
Copy link
Author

The most recent oss-fuzz test run with the problematic test case results in totally different behavior. This "recursive_mutex lock failed: Invalid argument" issue has been seen before:

 	+----------------------------------------Release Build Stacktrace----------------------------------------+
	Command: /mnt/scratch0/clusterfuzz/resources/platform/linux/unshare -c -n /mnt/scratch0/clusterfuzz/bot/builds/clusterfuzz-builds_graphicsmagick_364babd4f1406e0e2b68256230a27a3911dd0072/revisions/coder_AVIF_fuzzer -rss_limit_mb=2560 -timeout=60 -runs=100 /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/crash-0544689f851087964574e8fb1510006edcb16b1b
	Time ran: 0.716010570526123
	
	INFO: Running with entropic power schedule (0xFF, 100).
	INFO: Seed: 529734401
	INFO: Loaded 1 modules   (402630 inline 8-bit counters): 402630 [0x5832e598f930, 0x5832e59f1df6),
	INFO: Loaded 1 PC tables (402630 PCs): 402630 [0x5832e59f1df8,0x5832e6016a58),
	/mnt/scratch0/clusterfuzz/bot/builds/clusterfuzz-builds_graphicsmagick_364babd4f1406e0e2b68256230a27a3911dd0072/revisions/coder_AVIF_fuzzer: Running 1 inputs 100 time(s) each.
	Running: /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/crash-0544689f851087964574e8fb1510006edcb16b1b
	07:36:08 0:0.227980  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.258748  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.259972  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.260831  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.261681  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.262488  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.263332  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.264096  0.060u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.264940  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.265770  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.266541  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.267293  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.268100  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.268911  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.269696  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.270473  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.271307  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.272057  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.272936  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.273736  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.274543  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.275279  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.275975  0.070u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.276664  0.080u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.277420  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.278155  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.278847  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.279586  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.280319  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.280958  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.281620  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.282343  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.283001  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.283683  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.284391  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.285062  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.285777  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.286493  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.287175  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.287839  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.288486  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.289124  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.289818  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.290489  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.291171  0.090u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.291890  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.292546  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.293186  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.293860  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.294488  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.295151  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.295929  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.296600  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.297301  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.297961  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.298572  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.299186  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.299903  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.300584  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.301202  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.301834  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.302449  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.303052  0.100u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.303832  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.304517  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.305130  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.305770  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.306395  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.307046  0.110u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.307738  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.308373  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.308998  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.309659  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.310279  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.310890  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.311558  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.312324  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.312933  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.313569  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.314271  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.314885  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.315573  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.316226  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.316859  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.317512  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.318119  0.120u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.318712  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.319336  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.319981  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.320616  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.321262  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.321879  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.322549  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.323142  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.323785  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.324437  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.325077  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.325683  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.326284  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.326925  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	07:36:08 0:0.327572  0.130u 20746 heif.c/ReadHEIFImage/568/CorruptImage:
	  An error has occurred reading from file ()
	Executed /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/crash-0544689f851087964574e8fb1510006edcb16b1b in 158 ms
	***
	*** NOTE: fuzzing was not performed, you have only
	***       executed the target code on a fixed set of inputs.
	***
	libc++abi: terminating due to uncaught exception of type std::__1::system_error: recursive_mutex lock failed: Invalid argument
	AddressSanitizer:DEADLYSIGNAL
	=================================================================
	==20746==ERROR: AddressSanitizer: ABRT on unknown address 0x05390000510a (pc 0x7b8a19d9b00b bp 0x7fff595f8f00 sp 0x7fff595f8b70 T0)
	SCARINESS: 10 (signal)
	    #0 0x7b8a19d9b00b in raise /build/glibc-LcI20x/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:51:1
	    #1 0x7b8a19d7a858 in abort /build/glibc-LcI20x/glibc-2.31/stdlib/abort.c:79:7
	    #2 0x5832e334df75 in abort_message
	    #3 0x5832e339ea8a in demangling_terminate_handler() cxa_default_handlers.cpp:0
	    #4 0x5832e334dad2 in std::__terminate(void (*)())
	    #5 0x5832e334da87 in std::terminate()
	    #6 0x5832e2ac88ed in __clang_call_terminate
	    #7 0x5832e2af697c in Magick::MagickCleanUp::~MagickCleanUp() /src/graphicsmagick/Magick++/lib/Image.cpp:4274:3
	    #8 0x7b8a19d9e8a6 in __run_exit_handlers /build/glibc-LcI20x/glibc-2.31/stdlib/exit.c:108:8
	    #9 0x7b8a19d9ea5f in exit /build/glibc-LcI20x/glibc-2.31/stdlib/exit.c:139:3
	    #10 0x5832e296d448 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:0:5
	    #11 0x5832e2998692 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
	    #12 0x7b8a19d7c082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/libc-start.c:308:16
	    #13 0x5832e295fb3d in _start
	
	AddressSanitizer can not provide additional info.
	SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x4300b) (BuildId: 0702430aef5fa3dda43986563e9ffcc47efbd75e)
	==20746==ABORTING

@farindk
Copy link
Contributor

farindk commented Oct 16, 2024

There is only one recursive_mutex in libheif. That is used in the library initialization to make it thread-safe. However, nothing changed in that code part during the past months.

There is a new mutex (not recursive) introduced in 009c34e for the decoding progress reporting. Maybe it is related to that? But just from looking at the source, I do not see any issues.

Are you sure that this mutex is in libheif?

@bobfriesenhahn
Copy link
Author

bobfriesenhahn commented Oct 16, 2024 via email

@bobfriesenhahn
Copy link
Author

To be clear

Magick::MagickCleanUp::~MagickCleanUp() /src/graphicsmagick/Magick++/lib/Image.cpp

does not intentionally use any mutexes. The core GraphicsMagick C library only uses simple mutexes (my recollection of use of recursive mutexes was from maybe 20 years ago).

The MagickCleanUp() method calls "extern C" MagickPlusPlusDestroyMagick(), which then invokes the C function DestroyMagick() in the GraphicsMagick library. But we do not see mention of DestroyMagick() in the call stack.

Since GraphicsMagick is not using recursive mutexes, it must be some C++ code which is performing static destruction since this is a very static C/C++ program. It is difficult to assure correct ordering while a static C++ program is destroying its objects.

@farindk
Copy link
Contributor

farindk commented Oct 16, 2024

The (only) recursive_mutex in libheif is in

libheif/libheif/init.cc

Lines 88 to 92 in 1e9ccf1

static std::recursive_mutex& heif_init_mutex()
{
static std::recursive_mutex init_mutex;
return init_mutex;
}

Your error output says that is crashes when it tries to lock it.
It is locked during heif_init() and heif_deinit() (and during calls for explicit plugin loading which you probably do not use).

@bobfriesenhahn
Copy link
Author

I see that your use of a recursive mutex is a C++ thing and it is using a static object, which would be involved with C++ static construction and destruction. Static destruction of C++ objects would occur after main() returns.

Unfortunately, linking against, and initializing, so many 3rd-party libraries, has been using more execution time than a fairly substantial operation involving reading/writing an image file.

In order to lessen the overhead, GraphicsMagick is currently using lazy initialization of libheif like this:

At global scope:

#ifdef HAVE_HEIF_INIT
static MagickBool heif_initialized = MagickFalse;
#endif /* ifdef HAVE_HEIF_INIT */

And in ReadHEIFImage(), which may be called many times:

#ifdef HAVE_HEIF_INIT
  if (!heif_initialized)
    {
      /* heif_init() accepts a 'struct heif_init_params *' argument */
      heif_init((struct heif_init_params *) NULL);
      heif_initialized = MagickTrue;
    }
#endif /* HAVE_HEIF_INIT */

and in UnregisterHEIFImage(), which should be called once:

#ifdef HAVE_HEIF_DEINIT
  if (heif_initialized)
    heif_deinit();
#endif /* HAVE_HEIF_DEINIT */

When Magick++ calls DestroyMagick() then UnregisterHEIFImage() would be called.
Other than a possible multi-thread reentrancy issue if heif_init() is called by multiple threads at once, is there an ordering issue caused by this?

@bobfriesenhahn
Copy link
Author

I will create a new issue regarding the recursive_mutex destruction issue since there is another oss-fuzz issue open (since May!) specifically about that.

@farindk
Copy link
Contributor

farindk commented Oct 17, 2024

libheif is also doing "reference counting" in heif_init() and heif_deinit(). When you call it several times, it will only initialize it once. Same for destruction. It will only release everything when the number of calls to heif_deinit() matches the number of calls to heif_init(). Thus, I don't think the "heif_initialized" optimization is needed.

I'll try replacing the recursive_mutex with an atomic integer counter...

@bobfriesenhahn
Copy link
Author

I opened issue 1345 specifically about the recursive_mutex issue. It includes data from a different oss-fuzz report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants