nixpkgs
qt6.qtwebengine: fix build on darwin, disable metal shader compilation
#359755
Merged
qt6.qtwebengine: fix build on darwin, disable metal shader compilation #359755
There are commits from after the Metal shader build was introduced that introduce a proper flag for it, so we shouldn’t need to patch the build unless we maybe need to backport the upstream commits that added that flexibility. (Apparently Google’s toolchain itself can’t use the shader compiler.)
IIRC @niklaskorz (?) got as far as you previously. The Bluetooth thing seemed like it might be a compiler version issue (maybe try on staging with LLVM 19?) or an SDK version issue, but I don’t have any particular insight beyond that.
IIRC @niklaskorz (?) got as far as you previously. The Bluetooth thing seemed like it might be a compiler version issue (maybe try on
stagingwith LLVM 19?) or an SDK version issue, but I don’t have any particular insight beyond that.
My only encounter with the Metal shader compiler was in the context of zed-editor (not chromium), which I also solved by contributing runtime (instead of build time) shader compilation as an optional flag: zed-industries/zed@6dca609
Sorry, must have mixed you up with whoever was trying to get it building :)
I’m not sure about the Bluetooth thing here but hopefully we can figure out how to plumb down the flag to disable the shader compilation from the Qt WebEngine build system down to ANGLE.
There are commits from after the Metal shader build was introduced that introduce a proper flag for it
Base on the history of metal_backend.gni https://github.com/google/angle/commits/main/src/libANGLE/renderer/metal/metal_backend.gni, I don't see any gnFlags we can use to disable metal shader compilation.
I believe qt6.qtwebengine in nixpkgs already has the latest metal_backend.gni file.
google/angle@a9b0174 added the flag I’m thinking of. Since it’s apparently used by Chrome, it should be quite reliable.
I don't think metal_internal_shader_compilation_supported is a proper gn arg, since it's not in declare_args, unlike angle_enable_swiftshader and others:
Ah, okay, I didn’t realize. So I guess the only way to override things like angle_has_build is patching the source? (I don’t really understand the Chromium build system.)
Anyway, my best guess for the Bluetooth error is that LLVM 16 is too old. So I’d try on staging.
I've used this diff on master to test:
-qtModule {
+let
+ stdenv' = if stdenv.cc.isClang then llvmPackages_18.stdenv else stdenv;
+in
+(qtModule.override { stdenv = stdenv'; }) {But get this error:
FAILED: obj/third_party/zlib/bundled_zlib/zutil.o
../../../../../../../../rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/bin/clang -MMD -MF obj/third_party/zlib/bundled_zlib/zutil.o.d -DTOOLKIT_QT -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -DCR_XCODE_VERSION=0010 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DZLIB_IMPLEMENTATION -DADLER32_SIMD_NEON -DINFLATE_CHUNK_SIMD_NEON -DINFLATE_CHUNK_READ_64LE -DDEFLATE_SLIDE_HASH_NEON -DCRC32_ARMV8_CRC32 -DARMV8_OS_MACOS -Igen -I../../../../../src/3rdparty/chromium -I../../../../../src/3rdparty/chromium/third_party/zlib -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -Wno-unknown-argument -Wno-unknown-attributes -Wno-unknown-warning-option -Wno-ignored-attributes -Wno-predefined-identifier-outside-function -Wno-self-assign -Wno-unknown-pragmas -fcolor-diagnostics -fmerge-all-constants --target=arm64-apple-macos -mno-outline -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -O2 -fno-math-errno -fno-omit-frame-pointer -g0 -isysroot ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -mmacos-version-min=11.0 -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wall -Wno-unused-variable -Wno-c++11-narrowing -Wno-unused-but-set-variable -Wno-misleading-indentation -Wunguarded-availability -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-deprecated-declarations -Wenum-compare-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno-vla-extension -Wno-thread-safety-reference-return -O3 -fno-math-errno -Wno-incompatible-pointer-types -Wunused-variable -std=c11 -c ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c -o obj/third_party/zlib/bundled_zlib/zutil.o
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:8:
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:185:11: warning: 'OS_CODE' macro redefined [-Wmacro-redefined]
185 | # define OS_CODE 19
| ^
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:156:11: note: previous definition is here
156 | # define OS_CODE 7
| ^
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:10:
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/gzguts.h:20:
In file included from ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/stdio.h:61:
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: error: expected identifier or '('
314 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
162 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:10:
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/gzguts.h:20:
In file included from ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/stdio.h:61:
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: error: expected ')'
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
162 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: note: to match this '('
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
162 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:15: note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
Same error when using llvmPackages_19.stdenv, but llvmPackages_17.stdenv seems to work, building now.
@emilazy Do you recognize this error?
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 359755
aarch64-darwin
✅ 2 packages built:
- qt6.qtwebengine
- qt6.qtwebengine.dev (qt6.qtwebengine.dev.dev)
| 143 | 152 | --replace "AppleClang" "Clang" | |
| 144 | 153 | substituteInPlace cmake/Functions.cmake \ | |
| 145 | 154 | --replace "/usr/bin/xcrun" "${xcbuild}/bin/xcrun" | |
| 155 | # Disable metal shader compilation, Xcode only |
Mic92153 days agoNitpick. Can we make this substitution and the AppleClang one running on all platforms? Than we won't break macos if the package is updated.
azuwis153 days agoDone
anything preventing this from being merged?
Well, merge conflicts for one :)
Sorry for forgetting about this.
@azuwis That‘s a known issue; you can look at e.g. pkgs/development/web/nodejs/v20.nix for the upstream Zlib patch we‘ve applied to handle it in previous cases. But I suspect Chromium has probably applied it upstream by now and you can get it from there instead to avoid messing with the paths? Patching would be preferable to using the old LLVM in any case.
There‘s also some other LLVM 19‐related V8 backports in there we might need. It’s possible there’s stuff elsewhere in Chromium too but hopefully upstream patches shouldn’t be too hard to find.
qt6.qtwebengine: make the `AppleClang` substitution unconditional
Looks like the vendored libpng needs patching (or preferably we would just convince the build system to devendor it as we do on Linux).
With latest push, buildPhase and installPhase run fine, but fixupPhase hangs.
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindNodejs.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindNinja.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindGn.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindSnappy.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindGPerf.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindPkgConfigHost.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6WebEngineCoreTools/Qt6WebEngineCoreToolsAdditionalTargetInfo.cmake
installPhase completed in 36 seconds
Running phase: glibPreFixupPhase
Running phase: fixupPhase
checking for references to /nix/var/nix-build-qtwebengine-6.8.1.drv-0/ in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1...
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
patching script interpreter paths in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/tools/scripts/make_archive.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/bash"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/infra/recipes.py: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/tools/update_reference.sh: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/tools/run_formatter.sh: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
stripping (with command strip and flags -S) in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/libexec
signing /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1
autoSignDarwinBinariesHook should no longer be necessary, try dropping it.
It seems sigtool is processing every files in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/ dir.
└─ nix -L build -f . qt6.qtwebengine
└─ /nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/bash -e /nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh
├─ find /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1 -type f -print0
└─ /nix/store/6j9b3vzj6wppcyrbw96qxrk1qrx36dpz-sigtool-0.1.3/bin/sigtool --file /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/chromium/third_party/blink/common/shared_s...
I was wrong about the fixupPhase hang, the phase finally finished, just slow.
aarch64-darwin
✅ 2 packages built:
- qt6.qtwebengine
- qt6.qtwebengine.dev (qt6.qtwebengine.dev.dev)
Wonderful! I love how straightforward the diff is. Really nice that we can build big things like this again now.
I’ll try to remember to test the build on x86_64-darwin and then merge tomorrow; feel free to ping me if I don’t.
@emilazy Gentle ping.
It builds 🎉
Backport failed for release-24.11, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin release-24.11
git worktree add -d .worktree/backport-359755-to-release-24.11 origin/release-24.11
cd .worktree/backport-359755-to-release-24.11
git switch --create backport-359755-to-release-24.11
git cherry-pick -x af397f4a337dcdea43583ab12ef31c0fcdbdcb31 85c1c3ca3f214ee465a63bd6a5a41a7a9ebe191b
Do you mind handling the backport? It’d be great if we got this fixed on 24.11.
Do you mind handling the backport? It’d be great if we got this fixed on 24.11.
See #372849
Login to write a write a comment.
Currently qt6.qtwebengine failed on Hydra with this log:
It seems that is a Xcode only feature introduce by google/angle@a9b0174
I was trying to workaround that by disabling metal shader compilation:
It seems the workaround did the job (failed at [15832/29764] instead of [5019/29770]), but encountered another error:Edit: The workaround works, and the following error can be fixed using clang 17.
Error log
Full log https://gist.githubusercontent.com/azuwis/fff9587f6c9e55798fb0baf711a4a432/raw/5bc8795fce95340b5e0d350b22ff136f82a7afa2/qtwebengine.log
Any idea? @emilazy @K900
Fixes #353924
ZHF: #352882
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.