Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gathering fizzles while arranging opencv_hdf with MPI #3869

Open
jakaria143j opened this issue Jan 11, 2025 · 1 comment
Open

Gathering fizzles while arranging opencv_hdf with MPI #3869

jakaria143j opened this issue Jan 11, 2025 · 1 comment

Comments

@jakaria143j
Copy link

When compiling OpenCV with HDF (Hierarchical Data Format) support and MPI (Message Passing Interface) enabled, the compilation might fail due to various issues. Here are some common reasons and steps to troubleshoot and resolve the problem:

1. MPI Libraries or Configuration

OpenCV may not find the MPI libraries or might not be properly linked to them. Make sure that the MPI libraries are correctly installed and the environment is set up for OpenCV to detect them.

  • Ensure that you have installed MPI (e.g., OpenMPI or MPICH) on your system.
  • Check if the environment variables MPI_HOME, LD_LIBRARY_PATH, and PATH are set correctly to include MPI directories.

Example (Linux):

export MPI_HOME=/path/to/mpi
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/mpi/lib
export PATH=$PATH:/path/to/mpi/bin

Also, ensure that the OpenCV build process is finding MPI. You can pass the correct paths to CMake using the following options:

cmake -D MPI_INCLUDE_DIRS=/path/to/mpi/include -D MPI_LIBRARIES=/path/to/mpi/lib

2. Missing HDF Dependencies

OpenCV with HDF support relies on the HDF5 library. If the HDF5 development files are missing, OpenCV will fail to compile when HDF support is enabled.

  • Install HDF5 development libraries.

    On Ubuntu/Debian:

    sudo apt-get install libhdf5-dev

    On CentOS/Red Hat:

    sudo yum install hdf5-devel
  • Verify that OpenCV’s CMake configuration is detecting HDF5 correctly:

    cmake -D WITH_HDF=ON -D HDF5_INCLUDE_DIR=/path/to/hdf5/include -D HDF5_LIBRARIES=/path/to/hdf5/lib

3. OpenCV CMake Configuration

If you're using MPI and HDF together, make sure that your CMake configuration is correctly enabling both features.

  • If you're building OpenCV manually using CMake, you can specify the options like this:
    cmake -D WITH_HDF=ON -D WITH_MPI=ON -D HDF5_INCLUDE_DIR=/path/to/hdf5/include -D HDF5_LIBRARIES=/path/to/hdf5/lib

4. Parallel Compilation with MPI

If OpenCV uses MPI for parallelization during the build process, there might be conflicts with other flags or options. You can try disabling parallelism temporarily to see if that resolves the issue.

To disable MPI parallelism in OpenCV, you can pass the following to CMake:

cmake -D WITH_MPI=OFF

5. [Out-of-date CMake or OpenCV Version](https://newsmediapublishers24.blogspot.com/)

@Kumataro
Copy link
Contributor

I'm sorry but it is difficult to read the purpose of this issue correctly for me.
I suggest to append startup tutorial for HDF module if needed.

https://docs.opencv.org/4.x/d3/d00/tutorial_table_of_content_hdf.html

OpenCV may not find the MPI libraries or might not be properly linked to them.

I fixed this problem with #3859 .

sudo apt-get install libhdf5-dev

On ubuntu, we can use libhdf5-mpich-dev or libhdf5-openmpi-dev for using MPI.

-D MPI_INCLUDE_DIRS=/path/to/mpi/include -D MPI_LIBRARIES=/path/to/mpi/lib
-D HDF5_INCLUDE_DIR=/path/to/hdf5/include -D HDF5_LIBRARIES=/path/to/hdf5/lib

These definitions will be detected at cmake, so we do not have to set manually.

https://cmake.org/cmake/help/latest/module/FindHDF5.html
https://cmake.org/cmake/help/latest/module/FindMPI.html

WITH_MPI=OFF

I cannot find this option on https://github.com/opencv/opencv/blob/4.x/CMakeLists.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants