Where's That Shared Library
18 days ago
- #Python
- #Dynamic Linking
- #Packaging
- The article discusses the challenges of packaging Python applications with dependencies on imaging and AI libraries, which often involve C extensions.
- It explores dynamic linking issues on macOS and Linux, focusing on how linkers resolve dependencies and the problems encountered with absolute paths in dependencies.
- The author shares a personal experience with an error related to `libavcodec.58.dylib` not being found, illustrating common issues developers face with dynamic linking.
- Solutions like using `DYLD_LIBRARY_PATH` on macOS and `LD_LIBRARY_PATH` on Linux are discussed as temporary fixes, but they come with caveats.
- The article delves into the concept of `@loader_path`, `@executable_path`, and `@rpath` in macOS, which allow for more flexible and relocatable dependency paths.
- A comparison between macOS's `dyld` and Linux's `ld.so` linker behaviors is provided, highlighting differences in symbol resolution and search orders.
- The author proposes a packaging strategy inspired by `pnpm` for node_modules, using symlinks and a structured directory layout to manage dependencies without duplication.
- Challenges specific to Python applications, such as `sys.path` manipulation and the need to bundle shared libraries, are addressed.
- A step-by-step guide for creating relocatable Python applications is outlined, emphasizing the importance of tracking `dlopen` calls and managing the dependency graph.
- References for further reading on macOS and Linux linking mechanisms are provided.