DRFのTutorialを実行していく中で起こった内容の記録と暫定対処法
結論
- これを読む人は、これが記事執筆時点の正攻法ではないことを認識ください
MULTIDICT_NO_EXTENSIONS=1 pip install multidictpip install httpie- を実行することで、インストール可能になった
エラー内容
私はpip install httpieを実行した際に次のようになりました。
(env) tatsuya@MacBook-Pro-3 drf-tutorial-main % pip install httpie
Collecting httpie
Downloading httpie-3.2.2-py3-none-any.whl.metadata (7.6 kB)
Requirement already satisfied: pip in ./env/lib/python3.12/site-packages (from httpie) (23.3.2)
Collecting charset-normalizer>=2.0.0 (from httpie)
Downloading charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (33 kB)
Collecting defusedxml>=0.6.0 (from httpie)
Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting requests>=2.22.0 (from requests[socks]>=2.22.0->httpie)
Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: Pygments>=2.5.2 in ./env/lib/python3.12/site-packages (from httpie) (2.17.2)
Collecting requests-toolbelt>=0.9.1 (from httpie)
Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB 7.5 MB/s eta 0:00:00
Collecting multidict>=4.7.0 (from httpie)
Downloading multidict-6.0.4.tar.gz (51 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.3/51.3 kB 6.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting setuptools (from httpie)
Using cached setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB)
Collecting rich>=9.10.0 (from httpie)
Downloading rich-13.7.0-py3-none-any.whl.metadata (18 kB)
Collecting idna<4,>=2.5 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie)
Downloading idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie)
Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie)
Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Collecting PySocks!=1.5.7,>=1.5.6 (from requests[socks]>=2.22.0->httpie)
Downloading PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=9.10.0->httpie)
Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=9.10.0->httpie)
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Downloading httpie-3.2.2-py3-none-any.whl (127 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 127.4/127.4 kB 8.4 MB/s eta 0:00:00
Downloading charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl (119 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119.4/119.4 kB 14.6 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 7.3 MB/s eta 0:00:00
Downloading rich-13.7.0-py3-none-any.whl (240 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.6/240.6 kB 22.9 MB/s eta 0:00:00
Using cached setuptools-69.0.3-py3-none-any.whl (819 kB)
Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 7.5 MB/s eta 0:00:00
Downloading idna-3.6-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 kB 8.1 MB/s eta 0:00:00
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 9.0 MB/s eta 0:00:00
Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 12.1 MB/s eta 0:00:00
Building wheels for collected packages: multidict
Building wheel for multidict (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for multidict (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [78 lines of output]
*********************
* Accelerated build *
*********************
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-universal2-cpython-312
creating build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/_multidict_py.py -> build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/_abc.py -> build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/__init__.py -> build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/_multidict_base.py -> build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/_compat.py -> build/lib.macosx-10.9-universal2-cpython-312/multidict
running egg_info
writing multidict.egg-info/PKG-INFO
writing dependency_links to multidict.egg-info/dependency_links.txt
writing top-level names to multidict.egg-info/top_level.txt
reading manifest file 'multidict.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files found matching 'multidict/_multidict.html'
warning: no previously-included files found matching 'multidict/*.so'
warning: no previously-included files found matching 'multidict/*.pyd'
warning: no previously-included files found matching 'multidict/*.pyd'
no previously-included directories found matching 'docs/_build'
adding license file 'LICENSE'
writing manifest file 'multidict.egg-info/SOURCES.txt'
/private/var/folders/73/0yft0xs11fndmt7p68_6xypc0000gn/T/pip-build-env-0pud64ky/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'multidict._multilib' is absent from the `packages` configuration.
!!
********************************************************************************
############################
# Package would be ignored #
############################
Python recognizes 'multidict._multilib' as an importable package[^1],
but it is absent from setuptools' `packages` configuration.
This leads to an ambiguous overall configuration. If you want to distribute this
package, please make sure that 'multidict._multilib' is explicitly added
to the `packages` configuration field.
Alternatively, you can also rely on setuptools' discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
If you don't want 'multidict._multilib' to be distributed and are
already explicitly excluding 'multidict._multilib' via
`find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
you can try to use `exclude_package_data`, or `include-package-data=False` in
combination with a more fine grained `package-data` configuration.
You can read more about "package data files" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/datafiles.html
[^1]: For Python, any directory (with suitable naming) can be imported,
even if it does not contain any `.py` files.
On the other hand, currently there is no concept of package data
directory, all directories are treated like packages.
********************************************************************************
!!
check.warn(importable)
copying multidict/__init__.pyi -> build/lib.macosx-10.9-universal2-cpython-312/multidict
copying multidict/py.typed -> build/lib.macosx-10.9-universal2-cpython-312/multidict
running build_ext
building 'multidict._multidict' extension
creating build/temp.macosx-10.9-universal2-cpython-312
creating build/temp.macosx-10.9-universal2-cpython-312/multidict
clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/tatsuya/drf-tutorial-main/env/include -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c multidict/_multidict.c -o build/temp.macosx-10.9-universal2-cpython-312/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for multidict
Failed to build multidict
ERROR: Could not build wheels for multidict, which is required to install pyproject.toml-based projects
httpieが依存しているmultidictがインストールできず、エラーになっていました。
調査したこと
multidictのGitHubをみてみたところ、Python3.12は2024年1月8日時点では、正式な対応前の状態でした。
- https://github.com/aio-libs/multidict/issues/887
- https://github.com/aio-libs/multidict/pull/877/files
また、Library Installationには次のような記述がありました。
PyPI contains binary wheels for Linux, Windows and MacOS. If you want to install multidict on another operating system (or Alpine Linux inside a Docker) the tarball will be used to compile the library from source. It requires a C compiler and Python headers to be installed.
To skip the compilation, please use the MULTIDICT_NO_EXTENSIONS environment variable, e.g.: $ MULTIDICT_NO_EXTENSIONS=1 pip install multidict
そこで、コンパイルをスキップしてみる方法を取ることにしました。
(env) tatsuya@MacBook-Pro-3 drf-tutorial-main % MULTIDICT_NO_EXTENSIONS=1 pip install multidict Collecting multidict Using cached multidict-6.0.4.tar.gz (51 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: multidict Building wheel for multidict (pyproject.toml) ... done Created wheel for multidict: filename=multidict-6.0.4-py3-none-any.whl size=9708 sha256=f98015dd832bbcb0c10f9383b4c33ea04f539baef44449867d14e059b62dce29 Stored in directory: /Users/tatsuya/Library/Caches/pip/wheels/f6/d8/ff/3c14a64b8f2ab1aa94ba2888f5a988be6ab446ec5c8d1a82da Successfully built multidict Installing collected packages: multidict Successfully installed multidict-6.0.4
まず、上記の対応でmultidictのinstallは成功して、その後のhttpieのinstallも成功しました。
(env) tatsuya@MacBook-Pro-3 drf-tutorial-main % pip install httpie Collecting httpie Using cached httpie-3.2.2-py3-none-any.whl.metadata (7.6 kB) Requirement already satisfied: pip in ./env/lib/python3.12/site-packages (from httpie) (23.3.2) Collecting charset-normalizer>=2.0.0 (from httpie) Using cached charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (33 kB) Collecting defusedxml>=0.6.0 (from httpie) Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB) Collecting requests>=2.22.0 (from requests[socks]>=2.22.0->httpie) Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB) Requirement already satisfied: Pygments>=2.5.2 in ./env/lib/python3.12/site-packages (from httpie) (2.17.2) Collecting requests-toolbelt>=0.9.1 (from httpie) Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB) Requirement already satisfied: multidict>=4.7.0 in ./env/lib/python3.12/site-packages (from httpie) (6.0.4) Collecting setuptools (from httpie) Using cached setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB) Collecting rich>=9.10.0 (from httpie) Using cached rich-13.7.0-py3-none-any.whl.metadata (18 kB) Collecting idna<4,>=2.5 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie) Using cached idna-3.6-py3-none-any.whl.metadata (9.9 kB) Collecting urllib3<3,>=1.21.1 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie) Using cached urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB) Collecting certifi>=2017.4.17 (from requests>=2.22.0->requests[socks]>=2.22.0->httpie) Using cached certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB) Collecting PySocks!=1.5.7,>=1.5.6 (from requests[socks]>=2.22.0->httpie) Using cached PySocks-1.7.1-py3-none-any.whl (16 kB) Collecting markdown-it-py>=2.2.0 (from rich>=9.10.0->httpie) Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB) Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=9.10.0->httpie) Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB) Using cached httpie-3.2.2-py3-none-any.whl (127 kB) Using cached charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl (119 kB) Using cached requests-2.31.0-py3-none-any.whl (62 kB) Using cached rich-13.7.0-py3-none-any.whl (240 kB) Using cached setuptools-69.0.3-py3-none-any.whl (819 kB) Using cached certifi-2023.11.17-py3-none-any.whl (162 kB) Using cached idna-3.6-py3-none-any.whl (61 kB) Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB) Using cached urllib3-2.1.0-py3-none-any.whl (104 kB) Installing collected packages: urllib3, setuptools, PySocks, mdurl, idna, defusedxml, charset-normalizer, certifi, requests, markdown-it-py, rich, requests-toolbelt, httpie Successfully installed PySocks-1.7.1 certifi-2023.11.17 charset-normalizer-3.3.2 defusedxml-0.7.1 httpie-3.2.2 idna-3.6 markdown-it-py-3.0.0 mdurl-0.1.2 requests-2.31.0 requests-toolbelt-1.0.0 rich-13.7.0 setuptools-69.0.3 urllib3-2.1.0
ただし、今回対応したビルドしない方法は非常に遅くなるようでGitHubのREADMEには以下の説明がありました。
Please note, the pure Python (uncompiled) version is about 20-50 times slower depending on the usage scenario!!!
https://github.com/aio-libs/multidict?tab=readme-ov-file#library-installation
今回はDRFのTurotialを実行しているだけだった、ということもあるので特に大きな問題も考えられないためこの方法を取りました。 そもそも、Python3.12が正式対応されていないので、本来は3.11以下を利用する、又は別の方法を取るというのも良いかと思います（DRFのTutorialではcurlにも触れられていたので、curlを利用しましたがどうやったら利用できるか気になったので調べてみた具合でした）。
