Building Software against EGLIBC

From Qi-Hardware
Jump to: navigation, search

Some software like numpy doesn't build against uClibc, which is the C library used in the default software image - see below for details. Embedded GLIBC (EGLIBC) offers a potential solution because it provides the missing floating point-related functionality, but it appears that OpenWrt is not routinely or, at least, widely used with that particularly C library, currently.


[edit] Configuring OpenWrt for EGLIBC

To build the software using EGLIBC, it is necessary to run make menuconfig and choose...

  1. Advanced configuration options (for developers)
  2. Toolchain Options
  3. C Library implementation
  4. Then, choose eglibc from the list.
  5. Now, in the Toolchain Options menu, choose a library version.
  6. Finally, disable Use setjump()/longjump() exceptions. (See below for the consequences of this option being left enabled.)

(If available, the OpenWrt "trunk" build of EGLIBC doesn't seem to work: toolchain/eglibc/Makefile happily puts trunk into its configuration but doesn't tell EGLIBC, so it then seems to invent its own identifier (so-version) in libc/Makeconfig that the installation operation of base-packages isn't aware of. To summarise: don't bother with the "trunk" option; choose a real version if a choice exists.)

[edit] Building the Toolchain

It seems possible to be able to make just the toolchain as follows:

make tools/install
make toolchain/install

[edit] Building Packages

Once the toolchain is available, the configuration can be changed to add packages like pygame and numpy. If this is done using the make menuconfig command, it should be possible to then perform a build; otherwise, having edited the .config directly, try running make oldconfig first.

To compile individual packages and their dependencies - for example, numpy - issue a command like the following:

make package/feeds/packages/numpy/compile

This should put the package files in the bin/xburst/packages directory.

[edit] Building EGLIBC Packages

It appears possible to build the necessary libc and related packages by building the base-files package:

make package/base-files/compile

However, since we will test these packages in a chroot environment as described below, it is also necessary to compile busybox and its dependencies:

make package/busybox/compile

[edit] Installing the Packages

You can now copy the packages to a location on your NanoNote, perhaps using Ethernet over USB.

Since we do not want to install the packages to their normal locations - doing so will overwrite existing files and may make your NanoNote unusable - you must create a special configuration file for the package manager.

On the NanoNote, copy the following into a new file called eglibc_opkg.conf (for example):

dest root /root/eglibc
dest ram /usr/share/opkg/tmp
lists_dir ext /usr/share/opkg
option overlay_root /overlay

This should resemble your system's /etc/opkg.conf file but use a different dest root setting. Here, we define a root in /root/eglibc, so this means that we should also set up such a directory to hold our test packages:

mkdir /root/eglibc

You can now install the packages by issuing a command like the following:

opkg -f eglibc_opkg.conf install packages/*.ipk

Here, packages is a directory containing all the packages built against EGLIBC. It is essential that you use the -f option and that the specified file contains a modified dest root setting; otherwise, you could overwrite various critical libraries and end up with an unbootable NanoNote.

[edit] Testing the Packages

To test the packages, you now need to enter the package area using the chroot command:

 chroot /root/eglibc

This should start a shell and everything you see should originate from the package area but appear as if it were a system of its own. If this works it means that the software is using EGLIBC instead of uClibc. Exiting this shell will release you from the chroot environment.

Some packages will need to see devices, and an easy way to make them available is to bind mount the necessary filesystems. Do this before running chroot:

mount -o bind /dev /root/eglibc/dev
mount -o bind /proc /root/eglibc/proc
mount -o bind /sys /root/eglibc/sys

If you decide to remove the /root/eglibc directory, it is essential that you unmount these filesystems first:

umount /root/eglibc/dev
umount /root/eglibc/proc
umount /root/eglibc/sys

Otherwise, you may end up removing important files.

[edit] Test Results

Testing in a chroot as described above indicates that numpy and pygame.surfarray work when EGLIBC is used, although numpy needs to be changed to import fewer subpackages with EGLIBC 2.15 in order to avoid exhausting the available memory on the NanoNote (without swap in use).

Pygame works with numpy built against EGLIBC: the pygame.surfarray module works as intended.

[edit] Possible Errors

Initially, when compiling with the default NanoNote configuration adjusted to use EGLIBC (2.14 or 2.15), Pygame/SDL didn't release the framebuffer upon exit due to an error: must be installed for pthread_cancel to work.

Although this appeared to be a linker search path issue - the linker apparently not finding the library file, and the absence of --sysconfdir=/etc when configuring EGLIBC might have contributed to such a problem - in fact, the linker was finding the library, but was unable to find certain symbols.

[edit] Missing Symbols in libgcc_s

Activating LD_DEBUG=all gave the following details:

/lib/ error: symbol lookup error: undefined symbol: _Unwind_Resume (fatal)

It actually looks as if the given symbol is missing from /lib/ which does itself seem to be found, so the original error is misleading.

There does seem to be a configuration option to indicate whether gcc will support SJLJ (set-jump, long-jump) exceptions or frame unwinding. This option is controlled in OpenWrt using the SJLJ_EXCEPTIONS setting in the file for the gcc package. See also --enable-sjlj-exceptions in the libstdc++ manual. The configuration and the build are connected via the Makefile for the final gcc build, in particular the following code:


This was added in the following commit: toolchain/gcc: add option SJLJ_EXCEPTIONS to select gcc's exception handling. There's also a related bug, #9185, in OpenWrt.

It looks like this option is set in the default configuration:


The solution to the error involves unsetting this option (and thus removing --enable-sjlj-exceptions from the GCC-related configuration). With a rebuilt toolchain, /lib/ gains the necessary symbols, and rebuilt packages then appear to work correctly, with SDL releasing the framebuffer successfully on exit (presumably because an exception is properly handled).

[edit] uClibc Issues

The numpy code includes the fenv.h header file, but this is not available for most architectures as of uClibc 0.9.33. Adding UCLIBC_HAS_FENV to the OpenWrt configuration allows numpy to be built, but importing it causes a segmentation fault on the NanoNote.

To fix this, architecture-specific work is most likely to be required, although looking at some workarounds might be sufficient.

Personal tools