10 hours of debugging: x264 not found using pkg-config (FFmpeg, Emscripten and WebAssembly)8 min read

I’d spent a few days developing a feature to record the animation in our Jen application. Basically, there is an image port with some image, and then there is a panel to apply the effects on the image (such as swirling, rotating, or spinning, etc). Because all of these functions are done in the C++ backend, we use Emscripten to compile the native code so that the sandbox environment of the web browser can be used by the front-end client (here is the React app) for computationally intensive tasks.

Writing the video recorder by hand is challenging; there are too many low-level details such as color representation, video signal properties, data compression, codec, file format, codec containers, and so much more hard-earned knowledge that would require years of development, not days. Luckily, the open source community has given us FFMpeg, the free software solution that solves virtually all kinds of problems for video and audio processing.

So I try to find out a simple solution for the recording feature using FFmpeg, at the same time, we want the output file size to be not so gigantic, and ideally, we want to have a good video quality. Since the raw video file or audio could be really large. Suppose we have the raw video with the following properties:

• Duration: 30 seconds
• Frame rate: 30 FPS
• Resolution: 1920×1080
• Color model: RGB (3 bytes per pixel)

We can do some simple math to roughly estimate the size of the video as:

We see that the raw video size is too big for just 30 seconds; that’s why we need compression in this case. We need to compress the video in some format to reduce the size of the original video, with or without losing information when decompressing it again. Here, H. 264 is the standard codec to compress video with a good compression ratio that FFMpeg already supports, so I use this inside the WebM container (the container contains codecs for different digital data formats: text, audio, video).

Enough background, now let’s move closer to the debugging problem that we have in the title. Because the app is running in the browser, we have 2 options to run the FFMpeg with our WebAssembly environment:

• Build the FFMpeg binary code locally, and then use this directly as part of the backend code, and then expose desired recording functions to the front end (e.g, start_recording, stop_recording, etc)
• Using the FFMpeg WebAssembly version directly, so we can work with FFMpeg APIs in the WebAssembly environment right in the client code, without updating our backend.

I chose the first option in this case, as I want to have more control over different ways of doing media processing with our application rather than using what’s already there in the FFMpeg Wasm has offered, and also I want to experiment for myself what kind of issues I would have if I try to build it locally. Turned out to be a very frustrating issue!

I went to the FFMpeg wiki page to see how I can build the binary file for FFMpeg with all the codecs and containers that I want. I thought this would be straightforward, and here I ran the ./configure command inside the FFMpeg with --enable-libx264 option to make the H-264 codec available in my executable, because x264 (a library to support the H-264 codec) is an external dependency, so I need to have the pkg-config working for the library so that FFMpeg with the desired codec support could be built. First, I cloned and built the x264 for Emscripten:

# Clone and build x264 for WebAssembly
git clone https://code.videolan.org/videolan/x264.git
cd x264
emconfigure ./configure --prefix=/path/to/emscripten_x264_install --host=i686-gnu --enable-static --disable-asm --disable-thread
emmake make -j4
emmake make install

Then, setting the correct path for the X264_DIR, I run the main step in the Makefile to build the FFMpeg:

setup-ffmpeg:
	@echo "Setting up FFmpeg for Emscripten..."
	cd $(FFMPEG_DIR) && \
	PKG_CONFIG_PATH=$(X264_DIR)/lib/pkgconfig \
	emconfigure ./configure \
		--prefix=$(FFMPEG_DIR) \
		--cc="emcc" \
		--cxx="em++" \
		--ar="emar" \
		--ranlib="emranlib" \
		--enable-cross-compile \
		--target-os=none \
		--arch=x86_32 \
		--disable-x86asm \
		--disable-autodetect \
		--enable-libx264 \
		# ...other configuration flags...

Then I tried to run the make command to build my executable, and this error is consistently thrown at my terminal window:

ERROR: x264 not found using pkg-config
If you think configure made a mistake, make sure you are using the latest
version from Git. If the latest version fails, report the problem to the
[email protected] mailing list or IRC #ffmpeg on irc.libera.chat.
Include the log file "ffbuild/config.log" produced by configure as this will help
solve the problem.

I checked that all the paths are correct, and the packages are properly installed in the location that I specified. I ran the pkg-config --list-all | grep x264 and pkg-config --libs --cflags x264 the library was there in my host machine. Then I tried all sorts of different configurations I came up with, from changing the library location, setting the PKG_CONFIG_PATH, installing the pkg-config and x264 (for H-264 codec) with and without Homebrew. All of the attempts failed, and the error persisted.

After some hours of frustration, the next day I continued with this problem. I’m realizing that I’m building the code from the Emscripten command, not directly with the configure command that we often see when building code in C++.

Here, the Emscripten compiler provides us with the emconfigure command, which is essentially a wrapper around the regular configure command from C/C++. Even though the compile environment is on my machine (macOS), setting the PKG_CONFIG_PATH is not effective since the emconfigure itself will override or even ignore my variables. Turned out that if I want to define a custom pkg-config location for Emscripten, I need to override the EM_PKG_CONFIG_PATH variable, as stated in this doc (yeah, with the EM_ prefix!), so I update the environment variable:

export EM_PKG_CONFIG_PATH=/path/to/emscripten_x264_install/lib/pkgconfig:$EM_PKG_CONFIG_PATH

Then, running the Makefile again (we don’t have to provide the --extra-ldflags or the --extra-cflags flags, setting the EM_PKG_CONFIG_PATH is enough):

emconfigure ./configure \
  --prefix=$(FFMPEG_DIR) \
  --target-os=none \
  --arch=x86_32 \
  --enable-cross-compile \
  --disable-x86asm \
  --disable-inline-asm \
  --disable-stripping \
  --disable-programs \
  --disable-doc \
  --disable-debug \
  --enable-gpl \
  --enable-small \
  --enable-libx264

Now the build for FFMpeg with x264 library dependency on WebAssembly environment works like a charm!

From this debugging section, I also learn an important concept about cross compilation. As discussed in the post, I’m trying to build the FFmpeg library so it can be run on the WebAssembly environment. It’s worth noting that if you try to build this natively on your machine, then this version will not work on the WebAssembly sandboxed environment because of binary incompatibility for different environment architectures. The host machine and WebAssembly are two different environments; the latter has no knowledge about my host machine, like dependencies, libraries, or environments I had there. WebAssembly has a whole different architecture from instruction set architecture (ISA), system call interface, memory model, and binary format, also it has its own virtual file system running on the browser.