large_image_wheels

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #android #machine-learning #linux #docker #windows #awesome #cli #vue #pytorch #ios #openai

large_image_wheels

(★ 22)

manylinux wheel files for girder/large_image dependencies

large_image_wheels 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# large_image_wheels

manylinux wheel files for girder/large_image dependencies.

## Use

You can install from the wheels in this repository using a command like:
```
pip install pylibtiff openslide_python pyvips gdal mapnik glymur javabridge -f https://girder.github.io/large_image_wheels
```

## Building

The wheels are all generated with the accompanying Dockerfile.

Building the wheels:
```
docker build --force-rm -t girder/large_image_wheels .
```

To extract the wheel files from the docker image:
```
mkdir -p wheels
docker run -v wheels:/opt/mount --rm --entrypoint bash girder/large_image_wheels -c 'cp /io/wheelhouse/*many* /opt/mount/.'
```

This will use the last recorded versions (stored in the `versions.txt` file).  To update the versions that are used, you can do (before the docker build):
```
python3 -u check_versions.py > versions.txt
```

## Results

This makes wheels for the main libraries:
- GDAL
- Glymur
- mapnik
- openslide_python
- pylibmc
- pylibtiff
- python_javabridge
- pyvips

Currently, wheels are built for Python 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 and for architectures x86_64 and aarch64.  Some wheels are also being built for 3.13t and 3.14t.  Some libraries have older versions available for older versions of Python that were built before support for those versions was ended.  There are also some libraries from before their projects published binary wheels (psutil, pyproj, ujson).

This also builds some non-version specific libraries to ensure they have recent dependencies:
- bioformats

In general version numbers have some additional component to the base library (e.g., an extra .1 added to the end) to ensure that they do not have conflicts with exact version numbers on pypi.

## Extras

Various related executables are bundled with the Python packages.  These are added as package data in a `bin` directory within the main package.  There is a python wrapper script exposing these in the Python binary path.  For instance, `gdalinfo` is available from the `GDAL` package.  It is located in the Python site-packages `osgeo/bin` directory.  To access it directly (rather than through the wrapper script), the appropriate directory can be gleaned from Python as part of a bash command, e.g., `` `python -c 'import os,sys,osgeo;sys.stdout.write(os.path.dirname(osgeo.__file__))'`/bin/gdalinfo --version``.

## Issues

In order to find the built libraries, this modifies how pylibtiff, openslide_python, and pyvips load those libraries.  The other libraries are patched in place.  There is probably a better way to do this.

There are some differences in features exposed on different architectures; this is largely based on the source libraries and not on how they are built.

## Example Use

This makes is more convenient to use large_image.  For instance, you can create a Jupyter Notebook with large_image.

```
docker run --rm -p 8888:8888 jupyter/minimal-notebook bash -c 'pip install git+git://github.com/girder/large_image.git#egg=large_image[openslide,mapnik] -f https://girder.github.io/large_image_wheels matplotlib && start.sh jupyter notebook --NotebookApp.token="" --ip=0.0.0.0'
```

In the Jupyter interface, create a new notebook.  In the first cell run:
```
import large_image
import requests
import matplotlib.pyplot as plt
wsi_url = 'https://data.kitware.com/api/v1/file/5899dd6d8d777f07219fcb23/download'
wsi_path = 'TCGA-02-0010-01Z-00-DX4.07de2e55-a8fe-40ee-9e98-bcb78050b9f7.svs'
open(wsi_path, 'wb').write(requests.get(wsi_url, allow_redirects=True).content)
ts = large_image.getTileSource(wsi_path)
tile_info = ts.getSingleTile(
    tile_size=dict(width=1000, height=1000),
    scale=dict(magnification=20),
    tile_position=1000
)
```
And in the second cell run:
```
plt.imshow(tile_info['tile'])
```

## Rationale

It is an inconvenience to have to add `--find-links https://girder.github.io/large_image_wheels` to pip install commands to use these wheels.  There are alternatives: (a) convince upstream repositories to publish wheels, or (b) publish these under unique names (e.g., large_image_dependency_gdal).  None of these are perfect solutions.

Using `--find-links` requires modifying pip install commands.  Further, if a newer non-wheel version of a package is published on pypi, it will be used in favor of the wheel unless the explicit version is used.  Since this repository won't maintain old versions, this means that the wheels must be rebuilt as soon as new versions are released.

It isn't practical to produce wheels for some upstream packages.  For instance, GDAL can be compiled with optional components that require licensing.  A publicly published wheel can't contain such components, and, therefore, it makes sense for the package on pypi to expect that the library has been installed separately from the python module.

If these wheels were published under alternate names, they could be published to pypi.  However, this would require a wheel for every supported OS or have conditionals in downstream packages' setup.py install_requires.  Further, it would preclude using custom-built libraries (such as GDAL with licensed additions).

Using `--find-links` seems like the best choice of these options.  Downstream packages can list the expected modules in `install_requires`.  Installation doesn't become any harder for platforms without wheels, and custom-built libraries are supported.

Newer versions usually have an extra `.1` added to the original python package's version number to ensure that pip and other installers will resolve to our published wheel uniquely.

## Special use cases

### GeoDjango

GeoDjango expects libgdal and libgeos_c to be in standard locations.  These libraries can be installed by installing the GDAL wheel.  Starting with GDAL 3.1.2, the `osgeo` module from these wheels exposes two values that can be used [to tell Django where these libraries are located](https://docs.djangoproject.com/en/5.2/ref/contrib/gis/install/geolibs/).  Specifically, Django can be configured using these values:
```
import osgeo

django.conf.settings.configure()
django.conf.settings.GDAL_LIBRARY_PATH = osgeo.GDAL_LIBRARY_PATH
django.conf.settings.GEOS_LIBRARY_PATH = osgeo.GEOS_LIBRARY_PATH
```

Inside a Django application's `settings.py` file, this is somewhat simpler::
```
import osgeo

GDAL_LIBRARY_PATH = osgeo.GDAL_LIBRARY_PATH
GEOS_LIBRARY_PATH = osgeo.GEOS_LIBRARY_PATH
```

## Future Work

- More optional libraries

Several packages (GDAL, ImageMagick, vips, and probably others) could have additional libraries added to the built wheels.  As time permits, those libraries that are appropriately licensed may be gradually added.  See the Dockerfile for comments on what else could be built.  If there is a specific need for a library that hasn't yet been included, please create an issue for it.

- More executables

A variety of executables are bundled with the wheels in appropriate `bin` directories.  There are additional tools that could be added.  If there is a specific need for an executable that hasn't been included, please create an issue for it.

- Fully automate releases

The wheels should be published from a successful CI run rather than from a user commit.

large_image_wheels

large_image_wheels 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

large_image_wheels

large_image_wheels 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README