Pip install из GitHub

Введение

Устанавливать Python пакеты можно не только из PyPi но и непосредственно из GitHub репозиториев

Синтаксис

Синтаксис запроса похож на обычную установку. Вместо имени пакета нужно написать git+url_репозитория

Например:

python -m pip install git+https://github.com/AccountName/repo_name.git

Пример пакета для установки

Для того, чтобы пакет можно было установить из GitHub репозитория, у него должен быть setup.py файл.

Минималистичный пример такого проекта:

demo_py_package/ |-- demo_package | |-- __init__.py | `-- demo_module.py `-- setup.py

setup.py

from setuptools import setup, find_packages setup(name='package_name', version='0.1.2', packages = find_packages(), )

demo_module.py

def demo_function(service: str) -> str: service_dict = {"hosting": "https://beget.com"} if service in service_dict: return service_dict[service] else: return "Unknown service"

Чтобы использовать этот пакет, нужно установить его из GitHub

python -m pip install git+https://github.com/AndreiOlegovich/demo_py_package.git

Если это первая попытка, то в логах будет что-то вроде

Collecting git+https://github.com/AndreiOlegovich/demo_py_package.git Cloning https://github.com/AndreiOlegovich/demo_py_package.git to c:\users\Andrei\appdata\local\temp\pip-req-build-64_hb37e Running command git clone --filter=blob:none --quiet https://github.com/AndreiOlegovich/demo_py_package.git 'C:\Users\Andrei\AppData\Local\Temp\pip-req-build-64_hb37e' Resolved https://github.com/AndreiOlegovich/demo_py_package.git to commit 2f15fae5000064862c552ed60eae4f045a99185f Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: package_name Building wheel for package_name (pyproject.toml) ... done Created wheel for package_name: filename=package_name-0.1-py3-none-any.whl size=1395 sha256=30675cdca4217d613e0b650181cbefa459d2c32eff0743412112a3e4933150a5 Stored in directory: C:\Users\Andrei\AppData\Local\Temp\pip-ephem-wheel-cache-u2jcd9xl\wheels\d0\d8\63\23164b91e34ef750c2ee8a30fe00ddbebdd59cc29495311af7 Successfully built package_name Installing collected packages: package_name Successfully installed package_name-0.1

Если вы вносите какие-то изменения в свой пакет - не забывайте обновлять версию в setup.py Если не менять версию пакет не обновится.

Допустим нужно изменить имя пакета на demo_package тогда версию нужно будет обновить например с 0.1.2 на 0.1.3

from setuptools import setup, find_packages setup(name='demo_package', version='0.1.3', packages = find_packages(), )

python -m pip install git+https://github.com/AndreiOlegovich/demo_py_package.git

Collecting git+https://github.com/AndreiOlegovich/demo_py_package.git Cloning https://github.com/AndreiOlegovich/demo_py_package.git to c:\users\Andrei\appdata\local\temp\pip-req-build-0yz3ggox Running command git clone --filter=blob:none --quiet https://github.com/AndreiOlegovich/demo_py_package.git 'C:\Users\Andrei\AppData\Local\Temp\pip-req-build-0yz3ggox' Resolved https://github.com/AndreiOlegovich/demo_py_package.git to commit a4a5c5197cb47d55a9fbd712939d1196a6d7457c Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: demo_package Building wheel for demo_package (pyproject.toml) ... done Created wheel for demo_package: filename=demo_package-0.1.3-py3-none-any.whl size=1370 sha256=f2a222f71ee2f11689776cf2f14b273e36eaedd13d07c6d0ff7eccd6eeba38eb Stored in directory: C:\Users\Andrei\AppData\Local\Temp\pip-ephem-wheel-cache-t5l27wsj\wheels\d0\d8\63\22164b91e34ef750c6ee8a30fe00ddbebdd59cc29495311af7 Successfully built demo_package Installing collected packages: demo_package Successfully installed demo_package-0.1.3

Проверить установку пакета можно с помощью pip list

python -m pip list

Package Version ------------ ------- demo_package 0.1.3 pip 24.0 setuptools 56.0.0

После того как пакет установлен, его можно импортировать в скрипт

from demo_package import demo_module print(demo_module.demo_function("hosting")) print(demo_module.demo_function("tickets"))

https://beget.com Recommended service for tickets is https://aviasales.com

Установить из определённого коммита

Если нужно установить не последнюю версию пакета, а из какого-то определённого коммита - его нужно указать после символа @

Допустим, вы хотите откатиться до версии, где в словаре была всего одна пара ключ-значение

Если уже установлена другая версия пакета - её нужно удалить. Чтобы это сделать нужно знать актуальное имя пакета. Оно содержится в актуальной версии файла setup.py

vi setup.py

from setuptools import setup, find_packages setup(name='demo_package', version='0.1.5', packages = find_packages(), )

В этом примере я удаляю demo_package версии 0.1.5

python -m pip uninstall demo_package

Found existing installation: demo_package 0.1.5 Uninstalling demo_package-0.1.5: Would remove: c:\users\Andrei\sandbox\python\from_github_demo\venv\lib\site-packages\demo_package-0.1.5.dist-info\* c:\users\Andrei\sandbox\python\from_github_demo\venv\lib\site-packages\demo_package\* Proceed (Y/n)? Y Successfully uninstalled demo_package-0.1.5

Нужно найти подходящий коммит на GitHub, например 470a6844f71eb8872174d8fe28629337c3d69fab

python -m pip install git+https://github.com/AndreiOlegovich/demo_py_package.git@470a6844f71eb8872174d8fe28629337c3d69fab

python example.py

https://beget.com Unknown service

Как видно из второй строки вывода, мы откатились к версии, которая не знает о сервисе tickets. Обновиться с неё до последней версии можно сразу без удаления.

python -m pip install git+https://github.com/AndreiOlegovich/demo_py_package.git

requirements.txt

Если зависимости устанавливаются через файл requirements.txt то там тоже можно использовать ссылки на репозитории

git+https://github.com/AndreiOlegovich/demo_py_package.git

Или с указанием коммита

git+https://github.com/AndreiOlegovich/demo_py_package.git@470a6844f71eb8872174d8fe28629337c3d69fab

Подробнее про работу с requirements.txt можно прочитать здесь

Также можно устанавливать вполне стандартные пакеты, которые доступны и через PyPi.

Например, установим Pydantic из его репозитория

python -m pip install git+https://github.com/pydantic/pydantic.git

Collecting git+https://github.com/pydantic/pydantic.git Cloning https://github.com/pydantic/pydantic.git to c:\users\Andrei\appdata\local\temp\pip-req-build-6ub9od6s Running command git clone --filter=blob:none --quiet https://github.com/pydantic/pydantic.git 'C:\Users\Andrei\AppData\Local\Temp\pip-req-build-6ub9od6s' Resolved https://github.com/pydantic/pydantic.git to commit 9fc5e18209ac484ceaecbb270d22c4f93929699b Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting annotated-types>=0.4.0 (from pydantic==2.7.0a1) Downloading annotated_types-0.6.0-py3-none-any.whl.metadata (12 kB) Collecting pydantic-core==2.16.2 (from pydantic==2.7.0a1) Downloading pydantic_core-2.16.2-cp38-none-win_amd64.whl.metadata (6.6 kB) Collecting typing-extensions>=4.6.1 (from pydantic==2.7.0a1) Downloading typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB) Downloading pydantic_core-2.16.2-cp38-none-win_amd64.whl (1.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 12.1 MB/s eta 0:00:00 Downloading annotated_types-0.6.0-py3-none-any.whl (12 kB) Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB) Building wheels for collected packages: pydantic Building wheel for pydantic (pyproject.toml) ... done Created wheel for pydantic: filename=pydantic-2.7.0a1-py3-none-any.whl size=403185 sha256=7c96b46bcc6267ea843d53790431c3604aa37148ba05844352d759708054a571 Stored in directory: C:\Users\Andrei\AppData\Local\Temp\pip-ephem-wheel-cache-yuqu_a_n\wheels\64\fb\3d\4363fe96d47e52046f0dbf01541cdee62d2d334fca9794b430 Successfully built pydantic Installing collected packages: typing-extensions, pydantic-core, annotated-types, pydantic Successfully installed annotated-types-0.6.0 pydantic-2.7.0a1 pydantic-core-2.16.2 typing-extensions-4.9.0

Однако можно столкнуться с проблемами. Наткнулся в интернете на пример, где нужно было ставить cachesim, но не получилось сбилдить wheel.

from cachesim import CacheSimulator, Cache, MainMemory mem = MainMemory() l3 = Cache("L3", 20480, 16, 64, "LRU") # 20MB: 20480 sets, 16-ways with cacheline size of 64 bytes mem.load_to(l3) mem.store_from(l3) l2 = Cache("L2", 512, 8, 64, "LRU", store_to=l3, load_from=l3) # 256KB l1 = Cache("L1", 64, 8, 64, "LRU", store_to=l2, load_from=l2) # 32KB cs = CacheSimulator(l1, mem) cs.load(2342) # Loads one byte from address 2342, should be a miss in all cache-levels cs.store(512, length=8) # Stores 8 bytes to addresses 512-519 # will also be a load miss (due to write-allocate) cs.load(512, length=8) # Loads from address 512 util (exclusive) 520 (eight bytes) cs.force_write_back() cs.print_state()

python test.py

Traceback (most recent call last): File "test.py", line 1, in <module> from cachesim import CacheSimulator, Cache, MainMemory ModuleNotFoundError: No module named 'cachesim'

python -m pip install git+https://github.com/RRZE-HPC/pycachesim.git

Collecting git+https://github.com/RRZE-HPC/pycachesim.git Cloning https://github.com/RRZE-HPC/pycachesim.git to c:\users\Andrei\appdata\local\temp\pip-req-build-1xr14t9l Running command git clone --filter=blob:none --quiet https://github.com/RRZE-HPC/pycachesim.git 'C:\Users\Andrei\AppData\Local\Temp\pip-req-build-1xr14t9l' Resolved https://github.com/RRZE-HPC/pycachesim.git to commit 1741d353473206f4322760335ba43855673ed43a Preparing metadata (setup.py) ... done Building wheels for collected packages: pycachesim Building wheel for pycachesim (setup.py) ... error error: subprocess-exited-with-error × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [47 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.8 creating build\lib.win-amd64-3.8\cachesim copying cachesim\cache.py -> build\lib.win-amd64-3.8\cachesim copying cachesim\__init__.py -> build\lib.win-amd64-3.8\cachesim copying cachesim\backend.h -> build\lib.win-amd64-3.8\cachesim running build_ext building 'cachesim.backend' extension creating build\temp.win-amd64-3.8 creating build\temp.win-amd64-3.8\Release creating build\temp.win-amd64-3.8\Release\cachesim C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\Andrei\sandbox\python\from_github\venv\include -IC:\Users\Andrei\AppData\Local\Programs\Python\Python38\include -IC:\Users\Andrei\AppData\Local\Programs\Python\Python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /Tccachesim/backend.c /Fobuild\temp.win-amd64-3.8\Release\cachesim/backend.obj -std=c99 cl : Command line warning D9002 : ignoring unknown option '-std=c99' backend.c cachesim/backend.c(110): warning C4244: 'return': conversion from '__int64' to 'long', possible loss of data cachesim/backend.c(896): warning C4244: 'function': conversion from '__int64' to 'long', possible loss of data cachesim/backend.c(1157): error C2057: expected constant expression cachesim/backend.c(1157): error C2466: cannot allocate an array of constant size 0 cachesim/backend.c(1157): error C2133: 'cacheSim': unknown size cachesim/backend.c(1159): error C2057: expected constant expression cachesim/backend.c(1159): error C2466: cannot allocate an array of constant size 0 cachesim/backend.c(1159): error C2133: 'load_from_buff': unknown size cachesim/backend.c(1161): error C2057: expected constant expression cachesim/backend.c(1161): error C2466: cannot allocate an array of constant size 0 cachesim/backend.c(1161): error C2133: 'store_to_buff': unknown size cachesim/backend.c(1163): error C2057: expected constant expression cachesim/backend.c(1163): error C2466: cannot allocate an array of constant size 0 cachesim/backend.c(1163): error C2133: 'victims_to_buff': unknown size cachesim/backend.c(1165): error C2057: expected constant expression cachesim/backend.c(1165): error C2466: cannot allocate an array of constant size 0 cachesim/backend.c(1165): error C2133: 'linkcounter': unknown size cachesim/backend.c(1190): warning C4013: 'strtok_r' undefined; assuming extern returning int cachesim/backend.c(1190): warning C4047: '=': 'char *' differs in levels of indirection from 'int' cachesim/backend.c(1194): warning C4047: '=': 'char *' differs in levels of indirection from 'int' cachesim/backend.c(1201): warning C4047: '=': 'char *' differs in levels of indirection from 'int' cachesim/backend.c(1275): warning C4047: '=': 'char *' differs in levels of indirection from 'int' cachesim/backend.c(1350): warning C4018: '<': signed/unsigned mismatch cachesim/backend.c(1373): warning C4133: '=': incompatible types - from 'Cache *' to 'PyObject *' cachesim/backend.c(1378): warning C4133: '=': incompatible types - from 'Cache *' to 'PyObject *' cachesim/backend.c(1383): warning C4133: '=': incompatible types - from 'Cache *' to 'PyObject *' cachesim/backend.c(1447): warning C4133: 'function': incompatible types - from 'PyObject *' to 'Cache *' cachesim/backend.c(1449): warning C4133: 'function': incompatible types - from 'PyObject *' to 'Cache *' cachesim/backend.c(1451): warning C4133: 'function': incompatible types - from 'PyObject *' to 'Cache *' error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe' failed with exit status 2 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pycachesim Running setup.py clean for pycachesim Failed to build pycachesim ERROR: Could not build wheels for pycachesim, which is required to install pyproject.toml-based projects