[SOLVED] Building python from source, _sqlite3 built but not imported

Issue

I have a Rocky Linux 9.0 server and we use modules on it. I'm trying to compile different versions of python from source and to pack them into modules that can be loaded by the user as needed. But, unfortunately, I've been struggling to build them from source.

First off, if anybody has a good alternative on how to build moduls containing a specific python version (i.e. without compilation), feel free to let me know, I would appreciate some hints on that as well, but that is not the issue of this post.

I have the following script to download and build python and its dependency sqlite:

#!/bin/bash

# build a python module
# based on: https://stackoverflow.com/questions/43993890/modulenotfounderror-no-module-named-sqlite3

# Save current working directory
CURR_DIR="$(pwd)"

# Set working directory to the location of this file
cd "$(dirname "${BASH_SOURCE[0]}")"

# config
PYTHON_VERSION=3.11.7
INSTALL_BASE_PATH="/modules/shared/apps/python/${PYTHON_VERSION}"
SQLITE_VERSION=3450000

# clear the install directory
rm -rf ${INSTALL_BASE_PATH}

# download python
mkdir build
cd build
[ -f "Python-${PYTHON_VERSION}.tgz" ] || wget --no-check-certificate "https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz"
tar -zxf "Python-${PYTHON_VERSION}.tgz"

# download sqlite
# https://www.sqlite.org/2024/sqlite-autoconf-3450000.tar.gz
SQLITE_URL="https://www.sqlite.org/2024/sqlite-autoconf-${SQLITE_VERSION}.tar.gz"
echo "Downloading SQLite from ${SQLITE_URL}"

[ -f "sqlite-autoconf-${SQLITE_VERSION}.tar.gz" ] || wget --no-check-certificate "${SQLITE_URL}"
tar -zxf "sqlite-autoconf-${SQLITE_VERSION}.tar.gz"

# install sqlite
cd sqlite-autoconf-${SQLITE_VERSION}
./configure --prefix=${INSTALL_BASE_PATH}
make
make install

# install python
cd "../Python-${PYTHON_VERSION}"
export PKG_CONFIG_PATH="${INSTALL_BASE_PATH}/lib/pkgconfig"
./configure --prefix=${INSTALL_BASE_PATH}  --enable-loadable-sqlite-extensions LDFLAGS="-L${INSTALL_BASE_PATH}/lib" CPPFLAGS="-I${INSTALL_BASE_PATH}/include" CFLAGS="-I${INSTALL_BASE_PATH}/include"

make -j 8
read -p "Compiled python, check for errors and press key to continue..."
make install

# create a symlink for python
(cd "${INSTALL_BASE_PATH}/bin"; ln -s python3 python)

# set working directory back to original
cd $CURR_DIR

read -p "Done, press key to finish..."

When I run the script and check for errors after make -j 8, I see the following:

*** WARNING: renaming "_sqlite3" since importing it failed: /path/to/python/build/Python-3.11.7/build/lib.linux-x86_64-3.11/_sqlite3.cpython-311-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

The necessary bits to build these optional modules were not found:
_gdbm                 _tkinter              nis                
readline                                                       
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Following modules built successfully but were removed because they could not be imported:
_sqlite3 

I'm trying to leave the underlying OS untouched, so installing the usual pre-compiled sqlite-devel is not really an option.

There is a similar issue here, but not much of an answer that is helping me. Does anybody have an idea what the issue could be?

Solution

You need to define LD_LIBRARY_PATH:

# install python
cd "../Python-${PYTHON_VERSION}"
export LD_LIBRARY_PATH=${INSTALL_BASE_PATH}/lib
export PKG_CONFIG_PATH="${INSTALL_BASE_PATH}/lib/pkgconfig"

After you've built Python, you need to put in ~/.bashrc :

export LD_LIBRARY_PATH=/modules/shared/apps/python/3.11.7/lib

Answered By - Philippe

Answer Checked By - Katrina (WPSolving Volunteer)