Compare commits

..

618 Commits

Author SHA1 Message Date
Gabriel Somlo
0c060512c1 Fix undefined type error in 3rdparty/json11/json11.cpp
Under certain conditions (e.g., building on Fedora 42
using gcc-15.0.1), compilation fails with the following
error:

    "error: ‘uint8_t’ does not name a type"

Explicitly include <cstdint> to prevent that situation.

Signed-off-by: Gabriel Somlo <gsomlo@gmail.com>
2025-01-23 14:42:14 +01:00
Miodrag Milanovic
d673d04ff3 CMake: fix windows BBA resource embedding 2025-01-23 11:15:34 +00:00
Miodrag Milanović
e12093201a
CMake: Add include guards when IMPORT_BBA_FILES is used (#1438) 2025-01-23 10:54:37 +01:00
Catherine
1623243d50 CMake: disallow in-tree builds.
In-tree builds pollute the source directory and make version control
more difficult to use effectively.
2025-01-23 07:57:41 +00:00
Catherine
90d746f79e CMake: add support for exporting and importing .bba files.
This is useful for certain cross-compilation workloads, and to cache
rarely changing build products.

To use this functionality, build e.g. as follows:

    cmake . -B build-export -DEXPORT_BBA_FILES=../bba-files -DARCH=all
    cmake --build build-export -t nextpnr-all-bba

    cmake . -B build-import -DIMPORT_BBA_FILES=../bba-files -DARCH=all
    cmake --build build-import
2025-01-23 07:49:12 +00:00
Catherine
fac934bd2d 3rdparty: upgrade pybind11 to v2.12.1. 2025-01-22 21:48:55 +00:00
Catherine
6855b558ac CMake: use imported target for pybind11 (in the GUI).
See commit 43b2f385.
2025-01-22 21:48:40 +00:00
Catherine
17943a51cb CMake: remove -DSERIALIZE_CHIPDBS= option.
The impetus for this commit is the fact that it causes rare but
build-breaking race conditions when used with `make -jN` with `N > 1`.
These race conditions are difficult to track down or fix because of
the very rudmentary debugging tools provided by `make` and opaque
semantics of CMake's Makefile generator. They break the build by
running two `.bba` generation processes, then one of them renaming
the `.bba.new` file once it's done, leaving the other one to fail.

After reflection (as the author of this code path) and discussion with
community members who use it, I've concluded that this isn't the right
approach.
1. In practice, on targets where `-DSERIALIZE_CHIPDBS=` matters, you
   also care about other build steps, like linking nextpnr, which
   are not serializable this way. So you use a workaround anyway, like
   `make`ing individual targets instead.
2. The way to serialize the build with Make is the `-j1` option. Trying
   to work around `-jN` to make it work like `-j1` is inherently error
   prone. While there is some utility in not serializing C++ compilation
   this utility could be more easily achieved by providing a single
   target that builds all chipdbs, running `make <chipdb-target> -j1`,
   then running `make -jN` for the rest of the build.
2025-01-21 17:13:03 +00:00
Catherine
dbba1328bf Allow splitting nextpnr-himbaechel per microarchitecture.
This is added primarily for YoWASP.
2025-01-21 17:13:03 +00:00
Catherine
cd7f7c12f1 CMake: refactor architecture-specific build system parts.
Two user-visible changes were made:
* `-DUSE_RUST` is replaced with `-DBUILD_RUST`, by analogy with
  `-DBUILD_PYTHON`
* `-DCOVERAGE` was removed as it doesn't work with either modern GCC
  or Clang
2025-01-21 17:13:03 +00:00
Catherine
bb2336ad73 Fix #embed support in bbasm and use it when available.
This removes the atomic rename for bbasm outputs because it embeds
the resulting paths into the `.cc` files in embed mode. In any case
the write should be fast enough to not be a big risk for interrupted
builds.

This was tested with Clang 19 only (gcc hasn't had a release that
supports `#embed` yet).
2025-01-21 17:13:03 +00:00
Catherine
dcfb7d8c33 CMake: align Himbaechel targets with non-Himbaechel ones.
Primarily, this commit makes both of them use the `BBAsm` functions
to build and compile `.bba` files.

In addition, Himbaechel targets are now aligned with the rest in
how they are configured: instead of having all uarches enabled with
all of the devices disabled (the opposite of the rest of nextpnr),
uarches must be enabled explicitly but they come with all devices
enabled (except for Xilinx, which does not have a list of devices).
2025-01-21 17:13:03 +00:00
Catherine
f5776a6d64 CMake: eliminate family.cmake/CMakeLists.txt split.
While it served a purpose (granting the ability to build `.bba` files
separately from the rest of nextpnr), it made things excessively
convoluted, especially around paths.

This commit removes the ability to pre-generate chip databases. As far
as I know, I was the primary user of that feature. It can be added back
if there is demand for it.

In exchange the per-family `CMakeLists.txt` files are now much easier
to understand.
2025-01-21 17:13:03 +00:00
Catherine
a951faa16d CMake: extract bbasm compilation into a function.
This fully preserves existing functionality, although the `embed` mode
is untested and seems broken.
2025-01-21 17:13:03 +00:00
Catherine
43b2f38520 CMake: use imported target for pybind11.
This accounts for the use of either the system or the vendored pybind11.

Fixes #1428.
2025-01-21 15:05:57 +00:00
Miodrag Milanović
284fb3e874
Updating CI to work with ubuntu-latest (#1426)
* Fix build using ubuntu-latest

* Update to latest icestorm
2025-01-20 14:58:51 +00:00
Catherine
155adc3f5d CMake: rationalize and refactor build system.
The two main changes, done together in this commit, are:
* Eliminating most instances of `aux_source_directory()`, replacing
  them with explicit file listings; and
* Moving these file listings into respective subdirectories by
  representing respective nextpnr components as interface libraries.

In addition, the GUI CMake script tree was simplified since it had
a lot of unused/redundant code.

The `aux_source_directory()` command is not recommended for use by
CMake itself because it misses dependency changes when adding/removing
files, and consequently causes build failures requiring a clean rebuild.

This commit does not touch anything related to architectures/families,
which are very complex and redundant all on their own.
2025-01-16 11:36:44 +01:00
Catherine
d214308f5f CMake: reformat for consistency.
Normalize keywords to:

    if (...)
    elseif (...)
    else()
    endif()

    foreach (...)
    endforeach()

    other(...)

Normalize whitespace to 4 spaces.
2025-01-16 11:36:44 +01:00
Catherine
c48157aa4b googletest: fix -Werror=maybe-uninitialized failure. 2025-01-16 11:36:44 +01:00
Catherine
574f504787 Find all components of Python at the same time.
This is explicitly recommended by the FindPython module documentation
and is required to avoid failed builds on some systems. See:
https://cmake.org/cmake/help/latest/module/FindPython.html
2025-01-13 03:29:43 +00:00
Catherine
ab7a372491
himbaechel: allow subsetting uarches. (#1416)
E.g. selecting only Gowin instead of the default shrinks the resulting
binary by ~30%.
2025-01-12 08:13:08 +01:00
YRabbit
92694d7db7
Gowin. BUGFIX. Do not create missing wires. (#1418)
Erroneously created wires for specific IOs on the underside of some
chips.

Fixes https://github.com/YosysHQ/nextpnr/issues/1417

Also cosmetic edits.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2025-01-12 08:12:06 +01:00
Catherine
5fe680390f
Various fixes for clang/libc++ build (#1415)
* Gowin: add header includes required on libstdc++.

* kernel: fix incorrect printf-style format.

* himbaechel: add missing `override` qualifiers.

* Gowin: remove unnecessary `std::move`.

These calls inhibit RVO, a stronger optimization than moving an object.
2025-01-12 08:11:33 +01:00
myrtle
55bd760808
ice40: Don't constrain multiple potentially-incompatible FFs to same tile (#1413)
Signed-off-by: gatecat <gatecat@ds0.me>
2025-01-02 11:08:42 +01:00
YRabbit
c565e364bc
Gowin. Add the ability to place registers in IOB (#1403)
* Gowin. Add the ability to place registers in IOB

IO blocks have registers: for input, for output and for OutputEnable
signal - IREG, OREG and TREG respectively.

Each of the registers has one implicit non-switched wire, which one
depends on the type of register (IREG has a Q wire, OREG has a D wire).
Although the registers can be activated independently of each other they
share the CLK, ClockEnable and LocalSetReset wires and this places
restrictions on the possible combinations of register types in a single
IO.

Register placement in IO blocks is enabled by specifying the command
line keys --vopt ireg_in_iob, --vopt oreg_in_iob, or --vopt ioreg_in_iob.

It should be noted that specifying these keys leads to attempts to place
registers in IO blocks, but no errors are generated in case of failure.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Registers in IO

Check for unconnected ports.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. IO regs. Verbose warnings.

If an attempt to place an FF in an IO block fails, issue a warning
detailing the reason for the failure, whether it is a register type
conflict, a network requirement violation, or a control signal conflict.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. BUGFIX. Fix FFs compatibility.

Flipflops with a fixed ClockEnable input cannot coexist with flipflops
with a variable one.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. FFs in IO.  Changing diagnostic messages.

Placement modes are still specified by the command line keys
ireg_in_iob/oreg_in_iob/ioreg_in_iob, but also introduces more granular
control in the form of attributes at I/O ports:

  (* NOIOBFF *) - registers are never placed in this IO,

  (* IOBFF *) - registers must be placed in this IO, in case of failure
  a warning (not an error) with the reason for nonplacement is issued,

  _attribute_absence_ - no diagnostics will be issued: managed to place - good, failed - not bad either.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Registers in IO.

Change the logic for handling command line keys and attributes -
attributes allow routines to be placed in IO regardless of global mode.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Registers in IO. Fix style.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2025-01-01 13:11:57 +01:00
Aritz Erkiaga
0345b6e803
Fix gowin ALU SUB mode ports (#1407) 2025-01-01 13:05:54 +01:00
YRabbit
3d350c21c5
Gowin. BUGFIX. Global clock routing. (#1410)
Adds additional restrictions on the first PIP after the clock source -
only connections to SPINEs are allowed. This allowed to correct the
behaviour of DQCEs since the latter can only disable/enable SPINEs.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-12-31 10:31:03 +01:00
Tarik Graba
f01465f628
Add attributes to the hierarchical cells (#1405)
* Adds attributes to the hierarchical cells

* python: add binding for hierarchical cells attributes

* frontend/base: import hierarchical cells attributes
2024-12-17 11:30:39 +01:00
Miodrag Milanović
d810aac867
Add GroupId related calls to Himbaechel API (#1399)
* Add GroupId related calls to Himbaechel API

* Example uarch using new API features

* Update drawGroup to propagate only GroupId
2024-12-05 13:59:33 +01:00
Miodrag Milanović
5a807110de
Adding NanoXplore NG-Ultra support (#1397)
* ng-ultra: new architecture

* Implementation as in D2 deliverable

* Support for nxdesignsuite-24.0.0.0-20240429T102300

* Save memory by directly outputing json

* Add support for bidirectional IOs

* cleanup

* Create BFRs properly

* Add IOM insertion

* Cleanup

* Block certain pips depending of DDFR mode

* Add LUT bypass to improve routability

* Add bypass for CSC mode of GCK

* Fix IOM case

* Initial memory support

* Better RF/XRF handling

* fix

* RF placement and legalization

* Disconnect non available ports for NX_RAM

* cleanup

* Add RFB/RAM context support for latest release

* Remove ports that must not be used

* Proper port used only on RFB

* Add structure for clock sinks

* Use cell type where applicable

* Add clock sinks for other cell types

* Validation check fixes

* Commented too restrictive placement

* Added more crossbar wire type

* Hande IO termination input

* Fail early due to NX tools limitation for now

* Validations and fixes for RAM I/Os

* Fix for latest version of tools

* Use ctx->idf where applicable

* warn if RAM ports are not actually used

* Fix IOM packing

* Fix CY packing

* Change how constants are handled on CY

* Post placement optimization for CY

* Address comments for PR

* pack and export  GCK, WFG and PLL

* Cover more global routing cases

* Constraing to location if provided

* Place at LOC

* Pack and export DSP

* wip

* wip

* notes

* wip

* wip

* Validate DSPs

* DSP cascading

* Check mandatory parameters for DSP

* existing gck

* wip

* export all the rest for bitstream

* CDC packing

* add more sinks

* place FIFO

* map rest of FIFO ports

* enable pll by default

* cleanup

* Initial XLUT support

* Fix statistics

* Properly duplicate GCKs

* RRSTO and WRSTO are not used on XFIFO

* Fix for latest version of JSON format

* Implement GCK limitations

* cleanup

* cleanup

* Add more signals and use lowskew name

* cleanup code a bit

* Fix wfb

* detect cascaded GCKs

* Handle DFR

* Route dfr clock properly

* Cleanup

* Cleanup bitstream code

* Review issues addressed

* Move helper routines

* Expose private members for unit tests

* cleanup

* remove scale factor

* make all location helper arrays static

* Addressed review comments

* Support post-routing CSC and SCC

* Support NX_BFF

* Place CSS and SCC only on allowed locations

* Support latest Impulse

* ng_ultra: Expand bounding box further for left-edge IO

Signed-off-by: gatecat <gatecat@ds0.me>

* Export all IO parameters in bitstream

* Handle new CSV order or parameters and additional validation

* Add some more undocumented values for CSV

* Support for old and new CSV formats

* Initial DDFR support

* Display warning message once per file

* Address review issues

* Fix crash on memory access

* Make boundbox fit NG-Ultra internal design

* Update attributes after dff rewrite

* Implement basic NG-Ultra LUT-DFF unit tests

* Always use first seen xbar input

Signed-off-by: gatecat <gatecat@ds0.me>

* Simplified crossbar pip detection

* Change order to prevent issues with some unconnected constants

* Pack LUT and multiple DFF in stripe

* Place DFF chains

* Improve large DFF chains

* Rename to pack_dff_chains

* Better use XLUTs when possible

* pack output DFF together with XLUT

* option to disable XLUT optimiziations

* Make more optimizations optional

* fix to use pre-increment

* GCK for lowskew signals

* Bugfix for nets that are not part of lowskew network

* Fix bitstream export for PLL cell

* Remove separate route lowskew

* Allow WFG mode 2

* Merge inverter into GCK

* Add CSC per TILE when needed

* Improve reusage of existing cell for CSC

* Take preferred CSC

* Cleanup

* When in place CSC size not important

* Cleanup

* Reset and Load restriction

* make csc optimisation optional

* Proper count for IO resources

* Detect when there is no next cell for DSP chain

* Do not incorporate loops in XLUT

* Check if output exists

* Update copyright for delivery

* Make building NG-Ultra chip database optional, follow filename convention

* Ported drawing code to new API

* Update expandBoundingBox for NG-Ultra

* Copyright and license update

* Add README information

* cleanup and constids

* Using ctx->idf where applicable

* remove if_using_basecluster

* refactor extra data usage

* refactor to use create_cell_ptr only

* optimized getCSC

* optimize critical path a bit

* clangformat

* disable clangformat where applicable

---------

Signed-off-by: gatecat <gatecat@ds0.me>
Co-authored-by: Lofty <dan.ravensloft@gmail.com>
Co-authored-by: gatecat <gatecat@ds0.me>
2024-12-04 09:00:05 +01:00
YRabbit
5eaa1b3f1f
Gowin. Add IODELAY. (#1398)
* Gowin. Add IODELAY.

Input/Output delay (IODELAY) is programmable delay uint in IO block.

This delay line is enabled before/after the IO pad and allows the signal
to be delayed statically or dynamically during 0-127 stages each lasting
from 18 to 30 picoseconds depending on the chip family.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Replacing assertions with log_error.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-11-30 09:24:59 +01:00
YRabbit
2b8a235776
Gowin. Add Input Edge Monitor (#1396)
Add sampling part to IO blocks (input only). This edge detector will
allow to dynamically adjust DDR decoding window in the future.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-11-27 09:57:34 +01:00
Miodrag Milanović
0e69425794
Add expandBoundingBox method to API (#1395)
* Add expandBoundingBox method to API

* Update API documentation
2024-11-26 10:13:41 +01:00
Miodrag Milanović
55035465aa
Himbaechel GUI (#1295)
* Extend Himbaechel API with gfx drawing methods

* Add bel drawing in example uarch

* changed API and added tile wire id in db

* extend API so we can distinguish CLK wires

* added bit more wires

* less horrid way of handling gfx ids

* loop wire range

* removed not needed brackets

* bump database version to 5

* Removed not used GfxFlags
2024-11-21 15:13:22 +01:00
YRabbit
9c2d96f86e
Gowin. FFs placement. (#1386)
* Gowin. FFs placement.

* Allow clusters to be created from FFs and LUTs;

* Immediately create pass-through LUTs from free LUTs adjacent to FF - at the same time ensure alternating use of LUT inputs;

* In case of constant networks, such pass-through LUTs are disconnected from networks altogether;

* Allow FF to be placed directly into SSRAM slides - this is useful when using synchronous reading.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Fix aux name creation

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Use I3 for pass-trough LUTs

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-10-22 12:49:44 +02:00
myrtle
f36a6571c1
cmake: Use upstream BoostConfig.cmake instead of cmake's own (#1387)
Signed-off-by: gatecat <gatecat@ds0.me>
2024-10-22 10:35:54 +02:00
Meinhard Kissich
cf42baa43b
Fix RNG seed initialization (#1383) 2024-10-09 18:25:02 +02:00
gatecat
7c459805f6 himbaechel: Bump DB version for package extra_data addition
Signed-off-by: gatecat <gatecat@ds0.me>
2024-10-09 15:21:10 +02:00
Pepijn de Vos
028be1462a
apicula: add support for magic sip pins (#1370)
* apicula: add support for magic sip pins

* fix nullptr check

* DDR fix by xiwang

* WIP support for setting the iostd

* add iostd
2024-10-09 15:16:36 +02:00
Meinhard Kissich
d27993f019
Placer: Fix static legalise radius (#1382) 2024-10-08 15:20:33 +02:00
Rowan Goemans
0e5b1348e6
timing_log: Handle potentially missing net when reporting crit path (#1381) 2024-10-04 08:07:55 +02:00
myrtle
854549a5ab
ice40: Fix missing clock pin types (#1380)
Signed-off-by: gatecat <gatecat@ds0.me>
2024-10-04 08:07:13 +02:00
myrtle
75d2ce6a92
heap: Fix ripup criterea (#1378)
Signed-off-by: gatecat <gatecat@ds0.me>
2024-10-02 22:36:57 +02:00
YRabbit
65cf6d8da7
Gowin. Fix the port check for connectivity. (#1376)
* Gowin. Fix the port check for connectivity.

What happens is that it's not enough to check for a network, we also
need to make sure that the network is functional: has src and sinks.

And the style edits - they get automatically when I make sure to run
clang-format10.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Fix the port check for connectivity.

What happens is that it's not enough to check for a network, we also
need to make sure that the network is functional: has src and sinks

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-10-02 22:36:36 +02:00
Lofty
268b32c341 router2: additional heatmap data 2024-10-02 16:29:55 +02:00
Adrien Prost-Boucle
b3b2392893 clang-format on basectx.h 2024-10-01 15:24:40 +02:00
Adrien Prost-Boucle
7f33329fe1 Himbaechel Xilinx : XDC commands : Also search nets with lowercase for better interoperability with other synthesis tools and RTL languages 2024-10-01 15:24:40 +02:00
Adrien Prost-Boucle
3d00b97e0a Himbaechel Xilinx : Support get_nets with braces around net name in XDC commands 2024-10-01 15:24:40 +02:00
Adrien Prost-Boucle
a9cc7f453d Himbaechel Xilinx : Support multiple nets per command 2024-10-01 15:24:40 +02:00
Adrien Prost-Boucle
ff9ba9e090 Himbaechel Xilinx : More warning messages about unsupported things in XDC file 2024-10-01 15:24:40 +02:00
Adrien Prost-Boucle
cc04882b17 BaseCtx : Fix crash in getNetByAlias() 2024-10-01 15:24:40 +02:00
gatecat
9b51c6e337 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-30 14:51:33 +02:00
gatecat
fcdaf3f86c Remove fpga_interchange
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-30 13:10:30 +02:00
gatecat
1967db170d xilinx: Support for complex IOLOGIC
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-27 17:37:46 +02:00
gatecat
24fc33c014 xilinx: Basic I/ODDR support
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-27 17:09:15 +02:00
gatecat
d3c0f945da xilinx: Fix BRAM placement, clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-27 16:24:47 +02:00
gatecat
38e5faca85 xilinx: Fix workaround for unsupported xdc construct
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-27 16:07:38 +02:00
gatecat
e4dfd4e622 xilinx: Support single-port LUTRAM variants
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-26 18:11:01 +02:00
gatecat
7516b8950a xilinx: Few more stub timings
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-26 17:30:36 +02:00
gatecat
118ecbc6b3 xilinx: Remove unnecessary assert
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-26 15:58:16 +02:00
gatecat
c90d872e35 xilinx: Filter out another missing pip type
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-26 15:56:20 +02:00
Adrien Prost-Boucle
437fb70ed3 Himbaechel xilinx : Fix packing of cascaded DSP 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
cd51a0c2fc Placer : Emit non-fatal error messages before ending the program 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
9da05b6001 Himbaechel xilinx : DSP packing : Emit a non-fatal error message 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
2031a067a0 Himbaechel xilinx : More flexibility about types of DSP parameters 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
81bf92a855 Himbaechel xilinx : DSP packing : Disable clustering 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
8a0e062520 Himbaechel xilinx : DSP packing : Improve code efficiency 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
a08229d6b6 Placer : Clearer messages in warnings and errors 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
9bea22ed1e Himbaechel xilinx : DSP packing : Fix identification of cascaded ports and share identification code 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
ad9a54cc69 Himbaechel xilinx : More cascaded input ports for which routing is skipped 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
04f5f80766 Himbaechel xilinx : Add safety check in DSP packing for 7-series 2024-09-24 12:06:56 +02:00
Adrien Prost-Boucle
db0c99199e Himbaechel xilinx : Add support of DSP packing for 7-series 2024-09-24 12:06:56 +02:00
Rowan Goemans
bbdf7aacb0 timing_log: warn on min time violation when timing fail is allowed 2024-09-24 08:57:21 +02:00
Rowan Goemans
0af42f1218 common: Use NPNR_ASSERT_FALSE for unreachable case 2024-09-24 08:57:21 +02:00
Rowan Goemans
93e233dad9 timing: Fix hold slack not matching reported path delay 2024-09-24 08:57:21 +02:00
Rowan Goemans
098dcaedec timing: remove the articial clock delay inflation 2024-09-24 08:57:21 +02:00
Rowan Goemans
0fce4b8f4e timing: lower clock_delay_fact to 1 to check if CI passes 2024-09-24 08:57:21 +02:00
Rowan Goemans
25d64b2105 timing_log: Fix logging indendation to match master
timing: Disable clock_skew analysis by default
2024-09-24 08:57:21 +02:00
Rowan Goemans
5488cd994b router: Enable clock skew analysis during routing 2024-09-24 08:57:21 +02:00
Rowan Goemans
8ee2c5612c timing: Add safe zero check function for delay_t 2024-09-24 08:57:21 +02:00
Rowan Goemans
a7f79fd681 timing: minor cleanup and stupid mistake fixups 2024-09-24 08:57:21 +02:00
Rowan Goemans
bca6f6394a timing: Fix slack calculations
timing: Fix max_delay_by_domain_pair function
timing: Fix hold time check
2024-09-24 08:57:21 +02:00
Rowan Goemans
eb0bf9ea9c report: Handle new segment types
timing_log: Use common segment type strings
2024-09-24 08:57:21 +02:00
Rowan Goemans
3b7fec8c4f report: Handle new segment types 2024-09-24 08:57:21 +02:00
Rowan Goemans
4488d42368 log: Remove bad usage of [[no_return]] 2024-09-24 08:57:21 +02:00
Rowan Goemans
8e12dfc693 timing: cleanup clock2clock reporting
timing: Add clock2clock delay as seperate
        timing line item.
2024-09-24 08:57:21 +02:00
Rowan Goemans
86106cb49a timing: integrate c2c delays and cleanup code 2024-09-24 08:57:21 +02:00
Rowan Goemans
fc3b2de8da timing: Add clock skew to arrival and required time 2024-09-24 08:57:21 +02:00
Rowan Goemans
60ee682d58 timing: Make hold violations an error 2024-09-24 08:57:21 +02:00
Rowan Goemans
82ea65d984 timing: Report min delay violated in timing logger 2024-09-24 08:57:21 +02:00
Rowan Goemans
7aeed52c06 common: Add some convenience functions for development 2024-09-24 08:57:21 +02:00
Rowan Goemans
c25da06d03 timing: Start identification of min_delay violations 2024-09-24 08:57:21 +02:00
Rowan Goemans
44665a9c4d timing: Allow critical path traversal for shortest paths 2024-09-24 08:57:21 +02:00
Rowan Goemans
2d542eb63a timing: Add hold time to bound of critical path report 2024-09-24 08:57:21 +02:00
gatecat
4b63b1115e Bump tests submodule
Signed-off-by: gatecat <gatecat@ds0.me>
2024-09-20 13:44:45 +02:00
Jonas Thörnblad
6ca64526bb Fix handling of RNG seed
* Fix truncation of output seed value from 64 bits to 32 bits (int
  instead of uint64) when written to json file.

* Fix input seed value conversion when --seed option is used.

* Remove input seed value scrambling (use of rngseed()) when --seed
  or --randomize-seed option is used since the output seed value will
  be the scrambled value and not the seed that was actually supplied
  or generated.
2024-09-18 16:29:32 +02:00
Rowan Goemans
2627d4e0ad
ecp5: Allow disabling of global promotion (#1367) 2024-09-12 20:16:17 +02:00
YRabbit
50bd8d09b0
Gowin. Implement the EMCU primitive. (#1366)
* Gowin. Implement the EMCU primitive.

Add support for the GW1NSR-4C's embedded Cortex-M3 processor. Since it
uses flash in its own way, we disable additional flash processing for
this case.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Fix merge.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-09-12 08:53:39 +01:00
YRabbit
ff7b8535bc
Gowin. Add DHCEN primitive. (#1349)
* Gowin. Add DHCEN primitive.

This primitive allows you to dynamically turn off and turn on the
networks of high-speed clocks.

This is done tracking the routes to the sinks and if the route passes
through a special HCLK MUX (this may be the input MUX or the output MUX,
as well as the interbank MUX), then the control signal of this MUX is
used.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Change the DHCEN binding

Use the entire PIP instead of a wire - avoids normalisation and may also
be useful in the future when calculating clock stuff.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-09-11 10:18:26 +01:00
Rowan Goemans
8d0f52fbf9
timing: Move towards DelayPairs for timing reporting (#1359) 2024-09-11 07:23:46 +01:00
YRabbit
4d1de4532a
Gowin. BUGFIX. Create all Clock Pips. (#1358)
Some Clocks PIPS were not created due to a check for the presence of a
delay class, now all wires are attributed to the class so that there is
no longer any need for this check.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-09-05 21:39:26 +01:00
YRabbit
4cf7afedf7
Gowin. Implement the UserFlash primitive (#1357)
* Gowin. Implement the UserFlash primitive

Some Gowin chips have embedded flash memory accessible from the fabric.
Here we add primitives that allow access to this memory.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* Gowin. Fix cell creation

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-09-04 11:55:35 +01:00
Christian Fibich
2dc712130c
allow LFD2NX devices to be specified with --device (#1353) 2024-08-21 12:36:23 +02:00
YRabbit
32e2d9223c Gowin. BUGFIX. Timing
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-08-21 11:27:59 +01:00
YRabbit
01737a400c Gowin. Add clock wires delays.
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-08-21 10:58:55 +01:00
Lofty
ccdc2f6f13 himbaechel/gowin: add timing information 2024-08-21 10:58:55 +01:00
Adrien Prost-Boucle
fa55e93848 Himbaechel xilinx : Fix regex to parse Zynq device names 2024-08-19 21:06:45 +01:00
Rowan Goemans
0d5d32951c
SDC parsing support (#1348)
* kernel: Add SDC file parser

* kernel: Add sdc as valid option

* kernel/sdc: Add error on EOF when fetching strings

* kernel/sdc: WIP command parsing for set_false_path

* kernel/sdc: Fully parse set_false_path

* kernel/sdc: Handle review comments
2024-08-12 17:45:27 +02:00
Rowan Goemans
f199c3e576
Update shell.nix (#1347) 2024-08-12 16:29:26 +02:00
Saviour Owolabi
e9e7dce23d
Himbaechel Gowin: HCLK Support (#1340)
* Himbaechel Gowin: Add support for CLKDIV and CLKDIV2

* Himbaechel Gowin: Add support for CLKDIV and CLKDIV2

* Gowin Himbaechel: HCLK Bug fixes and corrections
2024-08-03 15:57:22 +02:00
YRabbit
11d335c7ce Gowin. Fix GW2A-18(c) DCS and DQCE
We filter out PIPs from these chips that bypass DCS.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-08-02 14:12:16 +02:00
YRabbit
10a5a44b81 Gowin. Implement clock management primitives.
DQCE and DCS primitives are added.

DQCE allows the internal logic to enable or disable the clock network in
the quadrant. When clock network is disabled, all logic drivern by this
clock is no longer toggled, thus reducing the total power consumtion of
the device.

DCS allows you to select one of four sources for two clock wires (6 and 7).
Wires 6 and 7 have not been used up to this point.

Since "hardware" primitives operate strictly in their own quadrants,
user-specified primitives are converted into one or more "hardware"
primitives as needed.

Also:
  - minor edits to make the most of helper functions like connectPorts()
  - when creating bases, the corresponding constants are assigned to the
    VCC and GND wires, but for now huge nodes are used because, for an
    unknown reason, the constants mechanism makes large examples
    inoperable. So for now we remain on the nodes.

Compatible with older Apicula databases.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-08-02 14:12:16 +02:00
YRabbit
f17caa2379 Gowin. BUGFIX. Fix placement checks
It was not taken into account that there are only 6 ALUs per cell. As a
result, on complex designs where ALUs and LUT-based memory are involved
and there are many LUTs (like in the RISCV emulator), there were
sometimes false positives about placement conflicts.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-07-29 11:31:59 +01:00
YRabbit
eb099a9244 Gowin. Bugfix.
The statement in the Gowin documentation that in the reading mode
"READ_MODE=0" the output register is not used and the OCE signal is
ignored is not confirmed by practice - if the OCE was left unconnected
or connected to the constant network, then a change in output data was
observed even with CE=0, as well as the absence of such at CE=1.

Synchronizing CE and OCE helps and the memory works properly in complex
systems such as RISC-V emulation and i8080 emulation (with 32K RAM and
16K BSRAM based ROM), but there is no theoretical basis for this fix, so
it is a hack.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-07-09 14:18:35 +02:00
YRabbit
1871afe9b9 Gowin. Taking into account the features of ROM
For pROM(X9) primitives in images generated by Gowin IDE, there is an
interesting recommunication of inputs depending on the data bit depth.
For example, in some cases, a high logical level may be applied to the
Write Enable input, which, let’s say, is not entirely usual for Read
Only memory.

Here we will do similar manipulations.

In addition, several minor bug fixes are included:

 - Fixed bit numbering for non-X9 series primitives.
 - Fixed decoder generation for BLKSEL - do not assume unused inputs are
   connected to GND.
 - Use default values for BSRAM parameters - don't assume their
   mandatory presence.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-07-09 14:18:35 +02:00
Miodrag Milanovic
cecd6b3f4d Document context related calls in HimbaechelAPI 2024-07-08 16:45:24 +02:00
Miodrag Milanovic
6b5b21e165 Enable user to override setupArchContext in Himbaechel arch 2024-07-08 16:45:24 +02:00
YRabbit
7dd4a8c1d5 Gowin. Implement power saving primitive
As the board on the GW1N-1 chip becomes a rarity, its replacement is the
Tangnano1k board with the GW1NZ-1 chip. This chip has a unique mechanism
for turning off power to important things such as OSC, PLL, etc.

Here we introduce a primitive that allows energy saving to be controlled
dynamically.

We also bring the names of some functions to uniformity.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-07-08 16:44:49 +02:00
TG
ba293437e0 ice40: Fix Python bindings for pip iterators 2024-07-03 15:09:27 +02:00
YRabbit
0639681b73 Gowin. Fix BSRAM block selection.
In the images generated by Gowin IDE, the signals for dynamic BSRAM
block selection (BLKSEL[2:0]) are not always connected directly to the
ports - some chips add LUT2, LUT3 or LUT4 to turn these signals into
Clock Enable.  Apparently there are chips with an error in the operation
of these ports.

Here we make such a decoder instead of using ports directly.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-07-03 15:09:13 +02:00
YRabbit
2e8280a949 Gowin. Fix pipeline mode in BSRAM.
It seems that the internal registers on the BSRAM output pins in
READ_MODE=1'b1 (pipeline) mode do not function properly because in the
images generated by Gowin IDE an external register is added to each pin,
and the BSRAM itself switches to READ_MODE=1'b0 (bypass) mode .

This is observed on Tangnano9k and Tangnano20k boards.

Here we repeat this fix.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-06-25 11:14:02 +02:00
YRabbit
8f87918230 Gowin. Add fix for Single Port BSRAM
Add description of BSRAM harness

In some cases, Gowin IDE adds a number of LUTs and DFFs to the BSRAM. Here we are trying to add similar elements.

More details with pictures: https://github.com/YosysHQ/apicula/blob/master/doc/bsram-fix.md

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-06-25 11:14:02 +02:00
gatecat
a29a17f8f2 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2024-06-18 13:54:12 +02:00
gatecat
c89037db49 static: Speedup with parallel wirelength gradient computation
Signed-off-by: gatecat <gatecat@ds0.me>
2024-06-18 13:53:42 +02:00
gatecat
945cf48c6c static: Various convergence improvements for ECP5
Signed-off-by: gatecat <gatecat@ds0.me>
2024-06-18 11:05:59 +02:00
gatecat
61cc5259d9 prefine: Add shared lock around bel availability checks
Signed-off-by: gatecat <gatecat@ds0.me>
2024-06-12 16:11:18 +02:00
gatecat
b7f91e57a0 Update cached Yosys in CI
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-17 06:31:43 +02:00
gatecat
59a29e5f42 nexus: Use a toposort when preplacing clock primitives
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-17 06:31:43 +02:00
gatecat
423f1b7159 static: Make bin stamping more consistent
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-08 09:53:28 +02:00
gmanricks
09703c7f35 update import to boost 2024-05-06 11:22:56 +02:00
gmanricks
eb0554319f use boost for windows path 2024-05-06 11:22:56 +02:00
gmanricks
f99346ba61 fix for windows path 2024-05-06 11:22:56 +02:00
Miodrag Milanovic
0dc4bcb203 Format utlilisation for larger FPGA as well 2024-05-06 11:22:33 +02:00
gatecat
3f2451f8d7 static: Guard density CSV dumps behind a flag
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-03 09:50:40 +02:00
gatecat
89e3b7d23d static: Fix float overflow issue
Co-authored-by: Lofty <dan.ravensloft@gmail.com>
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-03 09:39:24 +02:00
gatecat
7a00e76cb1 static: Exclude dark nodes from steplength
Signed-off-by: gatecat <gatecat@ds0.me>
2024-05-03 09:36:09 +02:00
Patrick Dähne
f085950383 Fixed header files for boost 1.85.0 2024-04-30 12:13:11 +02:00
Miodrag Milanovic
edcafcf085 Ignore compile artifacts in rust directory 2024-04-19 11:55:49 +02:00
YRabbit
4d5c48ad83 Gowin. Fix DSP MULT36X36
When multiplying 36 bits by 36 bits using four 18x18 multipliers, the
sign bits of the higher 18-bit parts of the multipliers were correctly
switched, but what was incorrect was leaving the sign bits of the lower
parts of the multipliers uninitialized. They now connect to VSS.

Addresses https://github.com/YosysHQ/apicula/issues/242

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-04-19 11:55:39 +02:00
YRabbit
d3b53d8e1a Gowin. PLL Pads. Fix the condition.
Do not search for pads if the signal source for the PLL is something
other than the IO pin - these are guaranteed to already be placed and
have a bound Bel.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-04-09 10:15:42 +02:00
YRabbit
6b7723e4c1 Gowin. Add PLL pads.
If the CLKIN input of the PLL is connected to a special pin, then it
makes sense to try to place the PLL so that it uses a direct implicit
non-switched connection to this pin.

The transfer of information about pins for various purposes has been
implemented (clock input signal, feedback, etc), but so far only CLKIN
is used.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-04-09 10:15:42 +02:00
Miodrag Milanovic
9bb46b98b4 update ci build script 2024-04-05 12:25:52 +02:00
Miodrag Milanovic
1f25f2067a Make example more like other arch 2024-04-05 12:25:52 +02:00
Miodrag Milanovic
ac725465a9 gui: fix warning on closing application 2024-04-05 12:25:52 +02:00
Miodrag Milanovic
75af8ccfd2 gui: user more reliable locking 2024-04-05 12:25:52 +02:00
Miodrag Milanovic
465cbfaf19 Add share to .gitignore 2024-04-05 12:25:52 +02:00
Jason Thorpe
7f9f75c0d3 Tweak the FreeBSD version of proc_self_dirname() to work on NetBSD and use it.
Resolves issue #1298.
2024-03-27 22:02:16 +00:00
Andrew Bell
b4da57598e One more warning. 2024-03-22 09:50:11 +00:00
Andrew Bell
693058abb7 Eliminate gcc13 warnings. 2024-03-22 09:50:11 +00:00
YRabbit
5ecb669a41 gowin: BUGFIX fix typo
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-03-22 09:49:01 +00:00
YRabbit
210e0fa33b gowin: Add support for DSP primitives.
For the following primitives:
  - PADD9
  - PADD18
  - MULT9X9
  - MULT18X18
  - MULT36X36
  - MULTALU18X18
  - MULTALU36X18
  - MULTADDALU18X18
  - ALU54D
packing and processing of fixed wires between macro and between DSP
blocks is implemented.
Clusters of DSP and macro blocks are processed using custom placement of
cluster elements.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-03-22 09:47:10 +00:00
YRabbit
ff96fc5af1 gowin: Himbaechel. Fix IDES16/OSER16
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-03-13 14:22:43 +01:00
YRabbit
4e8436a1fc gowin: Himbaechel. Allow to combine IOLOGIC.
Corrects the situation when it is impossible to use IOBUF with two
IOLOGIC elements at the same time - input and output.

Addresses https://github.com/YosysHQ/nextpnr/issues/1275

This is done by dividing one IOLOGIC Bel into two - input IOLOGIC and
output IOLOGIC plus checking for compatibility of the cells located
there.

At the moment, this check is simple and allows only the combination of
DDR and DDRC primitives.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-03-13 14:22:43 +01:00
YRabbit
4981ebb698 gowin: Himbaechel. Improve the global router
A small improvement - do not waste time analyzing already processed
networks in the previous step (and possibly steps).

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-03-13 14:22:11 +01:00
gatecat
05ed9308d6 ecp5: Improve router performance on slower speed grades
Signed-off-by: gatecat <gatecat@ds0.me>
2024-02-21 08:14:51 +01:00
gatecat
aa26ba7ea1 static: Improve singleton handling
Signed-off-by: gatecat <gatecat@ds0.me>
2024-02-20 10:25:35 +01:00
gatecat
255633c9f3 static: First pass at timing-driven placement
Signed-off-by: gatecat <gatecat@ds0.me>
2024-02-12 09:09:13 +01:00
YRabbit
cc273c1257 gowin: Himbaechel. Handle SDP OCE
Semi-dual port BSRAM (in Gowin terminology) has the same feature as
Single Port - the CE and OCE signals must be synchronized.

Such a sin has not yet been noticed for Dual Port.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-02-09 08:04:08 +01:00
YRabbit
833cb86b51 gowin: Himbaechel. Edit message text.
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-02-09 08:03:56 +01:00
YRabbit
4eeb56c0e0 gowin: Himbaechel. Improve global router.
* Don't stop at the first bad "arc", but use the global network to the
  maximum.
* Report partial/full use of global wires for the network.
* In case of complete routing failure, releasing the source - this is
  actually a BUGFIX.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-02-09 08:03:56 +01:00
YRabbit
b05cb86291 gowin: Himbaechel. Global router BUGFIX.
Ignore networks without users.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-01-29 13:18:24 +01:00
Miodrag Milanovic
a65ddff8ba Update workflows 2024-01-29 10:37:55 +01:00
YRabbit
325985e055 gowin: Himbaechel. SPX9 BSRAM BUGFIX.
This type setting is not needed here - the packer distinguishes memory
features by the X9 attribute, which will be correct anyway.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-01-27 15:05:58 +01:00
gatecat
e7192cd375 static: Fix ifdefs
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-26 17:57:22 +01:00
gatecat
9dcd0eff16 static: Add a basic threadpool
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-25 08:24:41 +01:00
YRabbit
73b7de74a5 gowin: Himbaechel. Fix the style.
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-01-23 14:00:29 +01:00
YRabbit
91b0c4f90a gowin: Himbaechel. Deal with SP BSRAM ports.
The OCE signal in the SP(X)9B primitive is intended to control the
built-in output register. The documentation states that this port is
invalid when READ_MODE=0 is used. However, it has been experimentally
established that you cannot simply apply VCC or GND to it and forget it
- the discrepancy between the signal on this port and the signal on the
CE port leads to both skipping data reading and unnecessary reading
after CE has switched to 0.
Here we force these ports to be connected to the network, except in the
case where the user controls the OCE signal using non-constant signals.

Also:
  * All PIPs for clock spines are made inaccessible to the common router
    - in general, using these routes for signals that have not been
    processed by a special globals router is fraught with effects that
    are difficult to detect.
  * The INV primitive has been added purely to speed up development -
    this primitive is not generated by Yosys, but is almost always
    present in vendor output files.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2024-01-23 14:00:29 +01:00
Pepijn de Vos
97f5c3ca2c Add documentation for Himbaechel Gowin uarch 2024-01-23 12:17:01 +01:00
gatecat
8968c84ce6 Increase the set of PnR-excluded cells
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-23 12:16:14 +01:00
gatecat
4220ce1007 gui: Remove const on max_elems_
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-18 15:34:40 +01:00
Lofty
a0e360fd9f rust: convert netinfo_driver to Option 2024-01-18 14:07:23 +01:00
Lofty
14a09061a5 rust: rework portref_cell 2024-01-18 14:07:23 +01:00
Lofty
dfd651a132 rust: fix some calls that got wrongly replaced 2024-01-18 14:07:23 +01:00
Lofty
d0e01661a5 rust: fix build error 2024-01-18 14:07:23 +01:00
Lofty
6e4e81429c rust: nets isn't send/sync 2024-01-18 14:07:23 +01:00
Lofty
c8e1cbc5f2 rust: transform pointers to references where possible 2024-01-18 14:07:23 +01:00
Lofty
c5fc34f11a rust: slight cleanup 2024-01-18 14:07:23 +01:00
Lofty
f12e76479c rust: add mutex for arch manipulation 2024-01-18 14:07:23 +01:00
gatecat
2afb1f632e clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-12 10:09:28 +01:00
gatecat
d00fdc8f7a frontend: Ignore $scopeinfo
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-11 15:48:53 +01:00
Lofty
257fbe549d readme: update build prerequisites 2024-01-05 20:12:05 +00:00
Lofty
d557e3e35f hashlib: constify const_iterators 2024-01-04 17:32:56 +01:00
Lofty
2c8ad5fa1d Fix a renamed Qt item 2024-01-04 17:32:56 +01:00
Lofty
d867019dcb upgrade to C++17 2024-01-04 17:32:56 +01:00
gatecat
5013392841 Add trivially copyable invariant for ID types
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-04 17:04:28 +01:00
dragonmux
6a9ad61051 rust: Fixed an unused parameter warning 2024-01-04 10:39:45 +01:00
dragonmux
cb269b46d6 rust: Made the wrap helper inline and fixed an accidental copy error 2024-01-04 10:39:45 +01:00
dragonmux
3e46fbc655 rust: Reworked the unwrap helpers by effectively hiding the crime of memcpy()'ing into a non-POD type from the compiler
There is still the possibility that this can explode horribly, but the result should be the same codegen and fixes the warning

This also makes the helpers `inline` so they'll usually be compiled out for a nice speed boost
2024-01-04 10:39:45 +01:00
dragonmux
cfeb588d32 rust: Reworked npnr_context_get_pips_leak() using std::accumulate() and fixed an accidental copy problem 2024-01-04 10:39:45 +01:00
dragonmux
7d0c4eaf1b rust: Reworked npnr_context_get_wires_leak() using std::accumulate() and fixed an accidental copy problem 2024-01-04 10:39:45 +01:00
dragonmux
e9c69ac00c gui: Fixed unused parameters and spurious ; warnings in one of the headers 2024-01-04 10:39:45 +01:00
dragonmux
cb4db2d368 ice40: Fixed unused parameters and spurious ; warnings in some of the headers 2024-01-04 10:39:45 +01:00
dragonmux
7fd80c5a92 common/kernel: Fixed unused parameters and spurious ; warnings in some of the headers 2024-01-04 10:39:45 +01:00
Lofty
50d43742ce rust: silence warnings 2024-01-03 14:51:33 +01:00
gatecat
e12ab86c75 rust: Fix segfault
Signed-off-by: gatecat <gatecat@ds0.me>
2024-01-03 13:42:18 +01:00
Lofty
1bbcc5f2c4 (broken) third round of review fixes 2024-01-03 13:42:18 +01:00
Lofty
49d505831d second round of review fixes 2024-01-03 13:42:18 +01:00
Lofty
1dbd81067a first round of review fixes 2024-01-03 13:42:18 +01:00
Lofty
d2297b1ba0 Add Rust FFI bindings 2024-01-03 13:42:18 +01:00
gatecat
4a4025192a run clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-12-26 09:54:34 +01:00
Miodrag Milanovic
41914876ef .gitignore for nextpnr-himbaechel 2023-12-23 11:09:26 +01:00
gatecat
56587859d3 nexus: Improve error reporting for illegal carry chains
Signed-off-by: gatecat <gatecat@ds0.me>
2023-12-22 15:40:29 +01:00
gatecat
535709a9a9 placer1: Fix various bitrot
Signed-off-by: gatecat <gatecat@ds0.me>
2023-12-13 11:37:30 +01:00
Lofty
d1083fd348 static/ice40: bug fixes for ultraplus 2023-12-13 11:37:20 +01:00
Miodrag Milanovic
b4ca68c8ef Add ability to override Cluster methods in Himbaechel 2023-12-11 13:53:52 +01:00
gatecat
6d9322457e static: Reduce stddev of initial solution
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-26 16:51:47 +01:00
YRabbit
c13b34f20e gowin: Himbaechel. Add BSRAM for all chips.
The following primitives are implemented for the GW1N-1, GW2A-18,
    GW2AR-18C, GW1NSR-4C, GW1NR-9C, GW1NR-9 and GW1N-4 chips:

    * pROM     - read only memory - (bitwidth: 1, 2, 4, 8, 16, 32).
    * pROMX9   - read only memory - (bitwidth: 9, 18, 36).
    * SDPB     - semidual port    - (bitwidth: 1, 2, 4, 8, 16, 32).
    * SDPX9B   - semidual port    - (bitwidth: 9, 18, 36).
    * DPB      - dual port        - (bitwidth: 16).
    * DPX9B    - dual port        - (bitwidth: 18).
    * SP       - single port      - (bitwidth: 1, 2, 4, 8, 16, 32).
    * SPX9     - single port      - (bitwidth: 9, 18, 36).

    For GW1NSR-4C and GW1NR-9 chips, SP/SPX9 primitives with data widths
    of 32/36 bits are implemented using a pair of 16-bit wide
    primitives.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-11-26 13:08:09 +01:00
YRabbit
90d4863dd4 gowin: Himbaechel. Add GW1NZ-1 BSRAM.
The following primitives are implemented for the GW1NZ-1 chip:

* pROM     - read only memory - (bitwidth: 1, 2, 4, 8, 16, 32).
* pROMX9   - read only memory - (bitwidth: 9, 18, 36).
* SDPB     - semidual port    - (bitwidth: 1, 2, 4, 8, 16, 32).
* SDPX9B   - semidual port    - (bitwidth: 9, 18, 36).
* DPB      - dual port        - (bitwidth: 16).
* DPX9B    - dual port        - (bitwidth: 18).
* SP       - single port      - (bitwidth: 1, 2, 4, 8, 16, 32).
* SPX9     - single port      - (bitwidth: 9, 18, 36).

Also:
 - The creation of databases for GW1NS-2 has been removed - this was not
   planned to be supported in Himbaechel from the very beginning and
   even examples were not created in apicula for this chip due to the
   lack of boards with it on sale.
 - It is temporarily prohibited to connect DFFs and LUTs into clusters
   because for some reason this prevents the creation of images on lower
   chips (placer cannot find the placement), although without these
   clusters the images are quite working. Requires further research.
 - Added creation of ALU with mode 0 - addition. Such an element is not
   generated by Yosys, but it is a favorite vendor element and its
   support here greatly simplifies the compilation of vendor netlists.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-11-26 13:08:09 +01:00
YRabbit
f2c280feda gowin: Himbaechel. Initial BSRAM support
Only pROM/pROMX9 for now

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-11-26 13:08:09 +01:00
Miodrag Milanovic
e3f4578b3b CRLF -> LF eol 2023-11-23 09:22:07 +01:00
Miodrag Milanovic
ec60542ffd create wiremap for himbaechel arch 2023-11-23 09:22:07 +01:00
Miodrag Milanovic
1ec8e411d7 set render bound box, so grid is displayed 2023-11-23 08:21:26 +01:00
Miodrag Milanovic
0b8a93eed5 fix compile warning 2023-11-23 08:21:26 +01:00
gatecat
de3d5be8f0 python: Remove deprecated use of Py_SetProgramName
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-23 06:49:15 +01:00
Balint Cristian
7814f44883 Fix abstract class implementation for fpga_interchange
Signed-off-by: Balint Cristian <cristian.balint@gmail.com>
2023-11-23 06:49:01 +01:00
gatecat
6683fd4ada himbaechel: Fix when more then 32k unique node shapes
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-22 17:11:27 +01:00
gatecat
55635cf2cd Update README
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-17 09:14:19 +01:00
gatecat
e2a887ef0d himbaechel: Switch default back to router1 for now
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-17 09:09:59 +01:00
gatecat
5bfe0dd1b1 himbaechel: Adding a xilinx uarch for xc7 with prjxray
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-14 17:12:09 +01:00
laanwj
a32ad13a86 ecp5: Don't segfault while packing FFs when DI port of TRELLIS_FF unconnected
Currently a segfault happens when the DI port is not specified. Leaving
it unconnected is probably incorrect, but it shouldn't crash the placer.
Fix by adding a check.
2023-11-14 11:55:51 +01:00
uis
a4d2244300 Fix printf formats 2023-11-13 13:59:51 +01:00
YRabbit
0106c3d299 gowin: Himbaechel. Diff io BUGFIX.
Fixed incorrect use of attributes instead of parameters.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-11-13 13:59:28 +01:00
Lofty
506d5f9422 machxo2: less pessimistic delay prediction 2023-11-09 06:48:50 +01:00
gatecat
4c6003ac0b router2: Don't use estimates for constant nets
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-07 15:55:22 +01:00
gatecat
7b0e082000 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-07 09:02:35 +01:00
gatecat
cf647463e4 himbaechel: Add support for new constants API
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-07 09:00:03 +01:00
gatecat
fe52840054 archapi: Add new API for global constant routing
Signed-off-by: gatecat <gatecat@ds0.me>
2023-11-07 09:00:03 +01:00
Lofty
e3c44dd20a ice40: add IO group to static 2023-11-07 08:18:35 +01:00
Lofty
214cc4315e static: density multiplier should be a vector 2023-11-07 08:18:35 +01:00
Lofty
d6f54fd9df ice40: add static placer support 2023-10-29 08:46:33 +01:00
gatecat
d40c6e850d himbaechel: Generation speedup and improvements
Signed-off-by: gatecat <gatecat@ds0.me>
2023-10-29 07:46:45 +01:00
gatecat
74d7ebc71f clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-10-28 17:10:42 +02:00
Justin Rajewski
6bae89b8b7 Undid accidental formatting 2023-10-26 21:21:14 +02:00
Justin Rajewski
7cac0249a1 Fixes for building on windows 2023-10-26 21:21:14 +02:00
Justin Rajewski
95f0a19391 Fixes for building on windows 2023-10-26 21:21:14 +02:00
Justin Rajewski
1238b69d74 Fixes for building on windows 2023-10-26 21:21:14 +02:00
gatecat
4a7e58a938 static/ecp5: zero bel area for RAMW because it's a zero-area cell
Signed-off-by: gatecat <gatecat@ds0.me>
2023-10-14 09:40:41 +02:00
Miodrag Milanovic
5a2eff2120 compile fix 2023-10-09 09:00:27 +02:00
gatecat
0eb9a9ad02 placer_static: Initial prototype
Signed-off-by: gatecat <gatecat@ds0.me>
2023-10-02 14:56:40 +02:00
Miodrag Milanovic
95e7598cc6 Fix timing lookup for DP8KC 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
e4cb7ea337 proper clock calc due after funcion change 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
f0325730a8 made higher estimate and use proper speed 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
c2e7d3d611 remove commented sections 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
1811c71438 update trellis version 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
1a92c83c3a properly assign latest fuzzed data 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
ed7064b210 select proper signal 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
72546a2186 made delay_t int type 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
657d2898cf import proper data where possible 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
c2b75b355f use timing data 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
40313eacf0 fix import 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
1edb449601 optimization/cleanup 2023-10-02 14:49:17 +02:00
Miodrag Milanovic
58cb8a830a Load timing data 2023-10-02 14:49:17 +02:00
rowanG077
e8602fb56d std::numeric_limits<delay_t>::lowest() -> ::min() 2023-09-28 16:27:15 +02:00
Wanda
c07ca64ebe hashlib: Improve pool hash function. 2023-09-27 17:08:29 +02:00
rowanG077
3f2e550f51 tmg: Fix logging of slack histogram 2023-09-25 13:20:40 +02:00
rowanG077
38d2a4b844 tmg: Fix argument order in run method
Router 2 expects "update_route_delays" to be the first argument to `tmg.run`.
2023-09-25 13:20:40 +02:00
YRabbit
8e84006ee7 gowin: Himbaechel. Specify the chip variant.
For GW2A-18 and GW1N-9 you need to specify the family in addition to partno.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-16 10:10:51 +02:00
YRabbit
682c91476f gowin: Himbaechel. Fix install path
Use himbaechel/gowin instead of himbaechel/gowin/gowin.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-16 09:38:10 +02:00
YRabbit
f5996ff4a1 gowin: Himbaechel. Support DragonFlyBSD
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-16 07:38:57 +02:00
YRabbit
8a54e5ec1c gowin: Himbaechel. Support DragonFlyBSD
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-16 07:38:57 +02:00
YRabbit
165e89f49a gowin: Himbaechel. Support DragonFlyBSD
We add support right here so that later I don’t have to make patches to the ports.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-16 07:38:57 +02:00
gatecat
565927dfcc himbaechel: Add discovery of uarch and chipdb
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-15 08:23:43 +02:00
gatecat
3cac90a30a himbaechel: Fix for Python 3.9
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-13 14:35:58 +02:00
gatecat
3e1e783873 himbaechel: Initial timing support
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-08 09:55:49 +02:00
YRabbit
890d7f7617 gowin: Himbaechel. Use a more appropriate function
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-08 09:15:35 +02:00
YRabbit
78ee20b5da gowin: Himbaechel. Extend clock router
Now the clock router can place a buffer into the specified network,
which divides the network into two parts: from the source to the buffer,
routing occurs through any available PIPs, and after the buffer to the
sink, only through a dedicated global clock network.

This is made specifically for the Tangnano20k where the external
oscillator is soldered to a regular non-clock pin. But it can be used
for other purposes, you just need to remember that the recipient must be
a CLK input or output pin.

The port/network to set the buffer to is specified in the .CST file:

CLOCK_LOC "name" BUFG;

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-09-08 09:15:35 +02:00
gatecat
f9825c3130 ice40: only set/clear negclk bit if IO clock actually used
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-08 09:15:27 +02:00
Catherine
eef5243fba himbaechel/gowin: recognize -DAPYCULA_INSTALL_PREFIX=.../virtualenv.
This option can be empty, in which case the virtualenv is left
exactly as it was in the build environment.
2023-09-07 10:47:54 +02:00
Catherine
732b329e7d himbaechel/gowin: recognize -DHIMBAECHEL_GOWIN_DEVICES=all. 2023-09-07 10:47:54 +02:00
gatecat
9994ba1d19 json: Fix handling of offsets in backend
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-07 08:00:05 +02:00
gatecat
79c6840fef ecp5: Improve packer robustness to FF dangling M input
Signed-off-by: gatecat <gatecat@ds0.me>
2023-09-02 11:38:20 +02:00
gatecat
a9a9251e42 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-08-31 10:30:19 +02:00
YRabbit
98b09c369f gowin: Himbaechel. Fix the device selection
Slightly change the Gowin device selection mechanism for database generation.
By default, nothing is generated as before.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 10:09:04 +02:00
YRabbit
3e0b9826b5 gowin: Himbaechel. Fix problems.
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
aca14cc420 gowin: Himbaechel. Install bases
Install the Himbaechel gowin chipdb .bin files to
/usr/local/nextpnr/himbaehel/gowin

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
6513299126 gowin: Himbaechel. Handling of disabled units
Using  extra cell functions to mark disabled units using the PLL example.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
f42805984c gowin: Himbaechel. Improve CMake thing a little
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
fdd45d12fd gowin: Himbaechel. Add rough CMake stuff
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
1b926b2703 gowin: Himbaechel. Fix IO for GW1NZ-1
In these chips, the midline IOs are still simple, but are no longer just
IOBUF - that is, unlike the GW1N-1 IBUF and OBUF are not obtained by
applying a signal to the OEN input.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
84a27c3ebf gowin: Himbaechel. Improve error messages
OSER16/IDES16 placement issue reports now indicate which location is
having trouble.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
09b7cad7f1 gowin: Himbaechel. Refactor.
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
e85bb1c28c gowin: Himbaechel. Fix DESER and PLL
- OSER4 can be located in neighboring IOs;
- PLLVR also needs to rename the inputs.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
4d0afdfd60 gowin: Himbaechel. Add the GW1N-4 simple IOs
And also fix the clock router to allow (with a warning) non-dedicated
routing in case of false detection of clock wires.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
0994e11b73 gowin: Himbaechel. Add OSER16 and IDES16
Information about what function (main or auxiliary) the cell performs in
these primitives is transmitted through the tile's extra data. And this
also allows us to remove the calculation of the coordinates of the
auxiliary cell on the go.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
a823543932 gowin: Himbaechel. Unify the creation of tail types
A single mechanism for creating a new type of tile if special functions
are found in the chip database that depend on the coordinates of the
tile.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
87ae77fbc6 gowin: Himbaechel. Add IDES primitives
As well as the implementation of all OSC primitives.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
3a073540c2 gowin: Himbaechel. Add OSER10 and OVIDEO
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
dfb701b5ab gowin: Himbaechel. Add OSER8
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
5e9a96d358 gowin: Himbaechel. Add SERDES and differential IO
- experiment with notifyBelChange as an auxiliary cells reservation mechanism;
- since HCLK pips depend on the coordinates, and not on the tile type,
  the tile type is copied if necessary;
- information about supported types of differential IO primitives has
  been added to the extra information of the chip;

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
01044cc910 gowin: Himbaechel. Add redundant checks
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
03c413a27a gowin: Himbaechel. Add simplified IO
Add processing IO located on the sides of some chips. These are IOBUF,
which are converted into IBUF and OBUF not by fuses, but by signaling to
OE.

Also added the creation of a Global Set / Reset for all chips, instead
of a list of tile types, information from the apicula database is used,
and minor fixes.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
df13104384 gowin: Himbaechel. Add extra chip data
To implement unusual IOs that have a dynamically changing configuration
 it is convenient to store the switching method in the additional chip
 data.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
49f8620ac9 gowin: Himbaechel. Implement PLLs
- The global router is modified to work out the routing of PLL outputs and inputs;
- Added API function to change wire type after its creation - there was
  a need to unify all wires included in the node at the stage of node
  creation, when all wires have already been created.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
6eeac1cabf gowin: Himbaechel. Use pin functions info
Use information about pin functions in the clock router.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
b2ec06dfe8 gowin: Himbaechel. Implement the GSR primitive
On GW2A chips, the global set/reset is in its own cell.
Also corrections to ALU generation.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
6cac19c055 gowin: Himbaechel. Add constraint file processing.
- minor fixes for pinout saving;
- CST parser taken from generic-based apicula;
- $nextpnr IOB detachment is copied here because it is necessary to copy
  attributes from deleted bels.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
3d3039e25c gowin: Himbaechel. Add bundle data generation.
The pin data is only being populated so far, but not used.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
2930d80627 gowin: Himbaechel. Add a clock router.
Shamelessly adapted gatecat's router.
Very early version, not yet puzzled with recognizing clock sources and
controlling the type of wires involved.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
c4b3268e90 gowin: Himbaechel. Add the LUTRAM
- RAM16SDP1, RAM16SDP2 and RAM16SDP4 support;
    - Reading in these primitives is asynchronous, but we have taken
      measures so that DFF Bels remain unoccupied and they can be used
      to implement synchronous reading.
    - misc fixes.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
c9b23a01db gowin: Himbaechel. Add ALU.
- Added support for ALU running in "2" ADDSUB mode, the mode that yosys generates for gowin;
- Supports specifying an arbitrary input carry as well as passing the output carry to logic;
- A small restructuring of the source files.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
c82654d003 gowin: Himbaechel. Add a wideluts
- MUX2_LUT5, MUX2_LUT6, MUX2_LUT7 and MUX2_LUT8 support;
- storing a common class of files in extra_data;
- misc fixes.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
f7fbe0db04 gowin: Himbaechel, fix style
Run clang-format

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
e4d2e1bd85 gowin: add support for all DFF types
Himbaechel-gowin has learned how to place DFFs of all types by tracking
the compatibility of CLK, CE and LSR inputs, as well as placing mutually
compatible flip-flops in adjacent slices.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
ae89430075 gowin: add global VCC and VSS networks
- VSS and VCC sources in each cell are used;
- constant LUT inputs are disabled;
- putting the class declaration into a header file.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
fb5f764b85 gowin: Add himbaechel arch
- wires, nodes and whites are generated from bases - apicula;
- roting of SN and EW bidirectional wires is supported;
- supports "wrapping" the wires at the edges of the chip;
- LUT1-4 and two types of DFF(R) are supported;
- simple IO is supported.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
24e1734999 generate bba
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-08-31 08:28:09 +02:00
YRabbit
bc7cd4f20e wip start 2023-08-31 08:28:09 +02:00
Miodrag Milanovic
b9592093b5 Update examples to synth_lattice 2023-08-30 16:27:17 +02:00
Miodrag Milanovic
5497a37de1 VLO,VHI support for ECP5 2023-08-29 10:05:30 +02:00
Miodrag Milanovic
688f1ba983 widelut support for xo2/xo3/xo3d 2023-08-29 10:04:58 +02:00
gatecat
e08471dfaf router2: Improve robustness when critical nets conflict
Signed-off-by: gatecat <gatecat@ds0.me>
2023-08-24 09:20:44 +02:00
gatecat
977180524a nexus: More DPHY clock ports that require general routing hop
Signed-off-by: gatecat <gatecat@ds0.me>
2023-08-23 11:42:39 +02:00
gatecat
a01e2c9068 nexus: Be robust to parameters shorter than expected
Signed-off-by: gatecat <gatecat@ds0.me>
2023-08-23 11:42:39 +02:00
rowanG077
053dfc98f0 use std::numeric_limits instead of macros 2023-08-18 09:15:37 +02:00
rowanG077
1fdd683344 Do not use C++20 struct initilisation 2023-08-18 09:15:37 +02:00
rowanG077
240f89081f Add back error/warning for combinational loops 2023-08-18 09:15:37 +02:00
rowanG077
d2a489d5e9 Remove old timing analyser 2023-08-18 09:15:37 +02:00
rowanG077
b0820eeaaa Formatting and display async path in json report 2023-08-18 09:15:37 +02:00
rowanG077
cfd3a52a3c tmg: add timing_report 2023-08-18 09:15:37 +02:00
rowanG077
596873c302 tmg: Add net_timings, crit path and slack hist 2023-08-18 09:15:37 +02:00
rowanG077
8b51674a6b Add critical path report to modern timing engine 2023-08-18 09:15:37 +02:00
rowanG077
d9f009b570 Split timing into old and new code 2023-08-18 09:15:37 +02:00
gatecat
88714c54ec ecp5: Fix TQFP144 package import
Signed-off-by: gatecat <gatecat@ds0.me>
2023-08-17 16:28:35 +02:00
Miodrag Milanovic
053d89570f Use type name directly 2023-08-17 11:18:45 +02:00
Miodrag Milanovic
adacaf65f4 additional new constants 2023-08-17 11:18:45 +02:00
Miodrag Milanovic
83f65169a3 different oscilator for XO3D 2023-08-17 11:18:45 +02:00
Aki Van Ness
679b662a2b Added a code of conduct, which was taken from the YosysHQ/yosys repo 2023-08-08 16:52:08 +02:00
gatecat
54b2045726 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-06-20 10:58:18 +02:00
rowanG077
914999673c Rip out budgets 2023-06-20 10:57:10 +02:00
YRabbit
77afaf23a5 gowin: use the correct version of apicula 2023-06-20 10:48:48 +02:00
YRabbit
1260f2f7d7 gowin: Add support for GW2A series chips
* Limited to Tangprimer 20k or GW2A-LV18PG256C8/I7 chip.
* Clock lines are disabled.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-06-20 10:48:48 +02:00
Lofty
cbd6496d35 router2: fix 8935c186 (again) 2023-06-19 13:47:23 +02:00
Meinhard Kissich
6c0b4443d5 Removes unnecessary argument 2023-06-16 16:46:09 +02:00
Meinhard Kissich
bbe9ea9d65 gowin: fixes default networks 2023-06-16 16:46:09 +02:00
Lofty
787fac7649 router2: fix 8935c186 2023-06-14 03:40:48 +01:00
Lofty
71a6b99633 router2: revisit nodes with lower delay 2023-06-13 08:24:01 +01:00
Lofty
8935c1867f router2: revisit nodes with lower cost 2023-06-13 08:24:01 +01:00
rowanG077
863ad4cee8 Add .cache used by clangd to gitignore 2023-06-12 14:11:36 +02:00
rowanG077
68a2b2710f Add nix shell 2023-06-12 14:11:36 +02:00
rowanG077
8a79a3522c build: Flatten include dirs when building comp db 2023-06-12 14:11:36 +02:00
rowanG077
cb4846a58d build: push INSTALL_PREFIX from env to cmake var 2023-06-12 14:11:36 +02:00
Rowan Goemans
0f947ee693
Timing: Fix combinational paths through all ports (#1175)
Fixes https://github.com/YosysHQ/nextpnr/issues/1174
2023-06-12 10:25:01 +02:00
Rowan Goemans
5b958c4d80
Analyse async paths in TimingAnalyser (#1171) 2023-06-09 08:01:47 +02:00
Lofty
119b47acf3 mistral: add 8x40-bit M10K addressing mode 2023-05-31 00:49:27 +01:00
Lofty
c5666c47fe mistral: fix corner cases related to 13x1-bit M10Ks 2023-05-29 20:02:41 +01:00
Lofty
e5a5de53c1 mistral: fallback to guess if simulator has no waveform 2023-05-25 18:35:52 +01:00
Lofty
5936464967
router2: add alternate weight option (#1162) 2023-05-25 10:47:10 +02:00
Lofty
7912a61ce3
mistral: rework delay estimate (#1161) 2023-05-22 13:01:20 +02:00
gatecat
7dafd4da58 mistral: Fix uninitialised comb_out for plain LUTs
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-22 09:44:38 +02:00
Meinhard Kissich
f03da6568b
Fix segfault when clocking a FF from a ring oscillator (#1160)
* fix segfault when clocking a FF from a ring osc

* Change std::set to pool

Co-authored-by: Lofty <dan.ravensloft@gmail.com>

---------

Co-authored-by: Meinhard Kissich <meinhard.kissich@tugraz.at>
Co-authored-by: Lofty <dan.ravensloft@gmail.com>
2023-05-22 09:39:05 +02:00
gatecat
1d3e5151ba clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-19 09:00:31 +02:00
YRabbit
354b7daf12
gowin: implement differential IO primitives (#1159)
* gowin: implement differential IO primitives

Adds missing TLVDS_IBUF/IOBUF/TBUF primitives, as well as all kinds of
LVDS emulation primitives.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* gowin: fix build

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

* gowin: support as differential not only pins A and B

The GW1N-1 and GW1NZ-1 chips have cells with pins up to I, we provide
support for such pins.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>

---------

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-05-19 08:59:19 +02:00
Lofty
7a827b1c78 mistral: improve timing calculation 2023-05-17 20:38:03 +02:00
gatecat
ea925f39fb archapi: Add getArcDelayOverride
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-17 09:54:14 +01:00
gatecat
98121308e0 mistral: Don't allow route-through LUTs in carry-mode ALMs
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-17 10:31:52 +02:00
gatecat
bc7b8b63ed mistral: Add a basic global clock router
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-16 18:17:06 +02:00
Nathaniel Quillin
ca2e328a5f rename c++20 keyword s/requires/requires_range.
See https://en.cppreference.com/w/cpp/language/requires for more details.
2023-05-16 12:43:40 +02:00
gatecat
57b923a603 himbächel: Initial implementation
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-13 08:26:41 +02:00
gatecat
e3529d3356 machxo2: Global placement and clock routing from nexus
Signed-off-by: gatecat <gatecat@ds0.me>
2023-05-08 10:38:16 +02:00
Miodrag Milanovic
91771895b6 Removed not tested/used code 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
0067bcc615 use latest trellis and add arch tests 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
10595726c1 fix warning 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
e012f7b4f8 update prjtrellis version 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
8fd4735292 handle some SYSCONFIG 2023-05-04 14:23:08 +02:00
gatecat
655aee1f9d Fix invalid accesses during certain IO packing cases 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
582cd526ac display freq with two digits after decimal point 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
909917cb61 Add clock constraints for new primitives 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
90a6578c53 handle VLO and VHI 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
a2d08dc79e Made PDPW8KC to work 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
01c631870e pio and iologic missing constants 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
d7c2eb3fae Fix warning 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
8c19e6f83a clangformat 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
7ac3d0d901 basic support for few small primitives 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
2a35f0292a add constants for new primitives 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
c6f1f124f2 removed commented and not used code 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
3281ca6717 Add missing muxes for BRAM 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
437b57a510 Added getBelGlobalBuf 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
8c38e7ba61 Working BRAM 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
3a7770dca2 Add missing bel pins 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
19176ab597 Made PLL to work 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
1b3283fb7c Add constants for new bels 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
a79c2f3209 Add additional pic tiles 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
4b3ae70ca8 support DCC and use spine data 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
c04c961949 Import spine data 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
9121880c5f added a comment for constraining FF location 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
5e3fe3a4dc do not support FF on slice C when there is dpram 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
55518011e3 ramw and dram changes according to @gatecat 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
6ec3423405 LSRONMUX disable if not used by FF 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
510d92e01b cleanup FF and made DPRAM work in simple case 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
b6bb0cd5b8 Update CMakeList for machxo2 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
80c461bddd add write_bitstream to pybindings 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
05a191a014 Added LPF support 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
a0ba9afcba CCU2D is auto tied low 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
aacb36bf15 Use CCU2D cell 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
a00f810093 fix 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
6f85053b03 more like ecp5 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
3624fe90b2 one step closer 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
6508a0c267 This should not be here 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
153144022f More of making it inline 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
ca3d32e5ac make source more inline with ecp5 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
62ace58204 add missing bind and lutperm 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
7f8518d938 Import lutperm data 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
442142a47a typo fixes 2023-05-04 14:23:08 +02:00
Lofty
398b2ab569 bitstream emission update 2023-05-04 14:23:08 +02:00
Lofty
235a575267 port ecp5 split slice to machxo2 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
b033b915a6 Add bitgen for the rest of XO2 and XO3 2023-05-04 14:23:08 +02:00
Lofty
89c71bc8ac bitstream fixes for xo3 2023-05-04 14:23:08 +02:00
Miodrag Milanovic
80705e9bbb Support enabling XO3 and XO3D 2023-05-04 14:23:08 +02:00
YRabbit
051bdb12b3 gowin: fix style
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-20 08:35:50 +02:00
YRabbit
729757d55e gowin: fix style
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-20 08:35:50 +02:00
YRabbit
fdc2769259 gowin: add a common mechanism for placing ports
If the port is in a different cell than the primitive, then use the alias mechanism.
Considerably compact code for OSC as an example.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-20 08:35:50 +02:00
YRabbit
71192dc7a3 gowin: Remove inherited code for ODDR(c)
Implement ODDR(c) as part of IOLOGIC and remove all old code associated
with those primitives.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-14 09:23:00 +02:00
gatecat
b0a78de78f fabulous: Support for configurable LUT size
Signed-off-by: gatecat <gatecat@ds0.me>
2023-04-13 13:29:52 +02:00
YRabbit
62b8baa959 gowin: Fix style
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-12 14:35:17 +02:00
YRabbit
fddacb3dc1 gowin: implement IDES16 and OSER16 primitives
These are very cumbersome primitives that take up two cells and
consequently 4 IOLOGIC bels.
The primitives are implemented for the chips that contain them and are
supported by apicula GW1NSR-4C, GW1NR-9 and GW1NR-9C.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-12 14:35:17 +02:00
gatecat
7557d33dc6 ecp5: Fix invalid accesses during certain IO packing cases
Signed-off-by: gatecat <gatecat@ds0.me>
2023-04-12 06:56:59 +02:00
gatecat
6455b5dd26 viaduct: Add support for GUIs
Signed-off-by: gatecat <gatecat@ds0.me>
2023-04-11 19:11:54 +02:00
YRabbit
9bcefe46a8 gowin: Add implementation of IDDR and IDDRC primitives
Simple deserialization primitives are implemented for all supported boards.

Compatible with older apicula bases.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-06 08:41:54 +02:00
gatecat
23f2877dde fabulous: Fix bel names for pass bels in v2 format
Signed-off-by: gatecat <gatecat@ds0.me>
2023-04-05 15:45:18 +02:00
YRabbit
20b7f760d9 gowin: Add support for IDES primitives
* placement of IDES4, IVIDEO, IDES8 and IDES10 primitives is supported;
* primitives are implemented for the GW1N-1, GW1NZ-1, GW1NSR-4C,
  GW1NR-9, GW1NR-9C chips;
* tricks required for IOLOGIC to work on one side of the -9 and -9C
  chips are taken into account;

Compatible with old apicula bases.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-04-04 10:00:08 +02:00
YRabbit
b36e8a3013 gowin: bugfix
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-03-23 12:37:53 +01:00
YRabbit
c52906e8bc gowin: Rename questionable ports
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-03-23 12:37:53 +01:00
YRabbit
38eb1f05ff gowin: Change the way errors are processed
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-03-23 12:37:53 +01:00
YRabbit
95ace0fade gowin: Add support for OSER primitives
* placement of OSER4, OVIDEO, OSER8 and SER10 primitives is supported;
* primitives are implemented for the GW1N-1, GW1NZ-1, GW1NSR-4C,
  GW1NR-9, GW1NR-9C chips;
* the initial support for special HCLK clock wires is implemented to the
  extent necessary for OSER primitives to function;
* output to both regular IO and TLVDS_OBUF is supported;
* tricks required for IOLOGIC to work on one side of the -9 and -9C
  chips are taken into account;
* various edits, such as using idf() instead of the local buffer.

Compatible with old apicula bases.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-03-23 12:37:53 +01:00
gatecat
b3c33bd0ab ice40: Fix BRAM NegClk bitstream logic
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-20 18:54:57 +01:00
Miodrag Milanovic
d7e450b7b6 Update prjtrellis 2023-03-20 09:53:35 +01:00
Miodrag Milanovic
35eeaa7cc5 Add ramaining PIO tiles 2023-03-20 09:53:35 +01:00
Miodrag Milanovic
3f4c8d15d9 Use unified io location data 2023-03-20 09:53:35 +01:00
Miodrag Milanovic
0ce72e1a31 Use TRELLIS primitives 2023-03-20 09:53:35 +01:00
Miodrag Milanovic
ad5f6fccaa Use RelSlice, make more in line with ecp5 arch 2023-03-20 09:53:35 +01:00
gatecat
e4fcd3740d cmake: Make HeAP placer always-enabled
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-17 10:38:11 +01:00
gatecat
4111cc25d6 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-17 09:31:38 +01:00
Miodrag Milanovic
656bfdb819 Update prjtrellis to latest 2023-03-16 15:19:48 +01:00
Miodrag Milanovic
d337ab93e6 Update to latest prjtrellis 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
11a90aff83 Fix out of tree builds and place h in generated 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
f008d7c4d8 Let top tiles be on top 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
6eb5f2a77e Enable wires and add dummy wire type for now 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
1f115ddd32 Basic GUI part selection 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
26798038fe Fix examples 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
7ad9914e51 Extend chipdb with metadata 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
d5b5f7e4b2 add new field handling in chip config format 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
4396a646a7 Add simple BEL graphics 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
26f23e3121 Make small GUI changes 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
18ad718e53 Expand list of possible devices 2023-03-16 13:37:23 +01:00
Miodrag Milanovic
12911a7470 Remove anoying warning from cmake 2023-03-16 13:37:23 +01:00
gatecat
ff14547601 ecp5: Update GUI rendering to match arch changes
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-16 13:13:57 +01:00
gatecat
39b6584274 clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-16 11:27:08 +01:00
gatecat
132a98a91d router1: Add error when dest port has no wire
Signed-off-by: gatecat <gatecat@ds0.me>
2023-03-06 14:15:48 +01:00
gatecat
2f509734df fabulous: Misc improvements
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-28 21:39:25 +01:00
gatecat
cdd7bb676f fabulous: Support for complex flops in PnR
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-28 21:39:25 +01:00
gatecat
1dcc2f777d ice40: Add python binding for write_bitstream
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-28 18:46:35 +01:00
gatecat
5d0aa77861 fabulous: Add timing model for carries
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-27 08:42:56 +01:00
gatecat
26fcf349ad fabulous: LUT permutation support
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-27 08:42:56 +01:00
gatecat
14050f991b fabulous: Global constant wires scheme
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-23 10:05:55 +01:00
Catherine
1809e18c7b CMake: detect platform support for threads 2023-02-23 10:05:44 +01:00
Catherine
ebbaf8c08d common: disable parallel refinement only without threads.
Previously it was always disabled on WebAssembly builds.
2023-02-23 09:45:19 +01:00
Catherine
8f0731edc9 common: update deprecated use of boost::filesystem::basename. 2023-02-23 09:44:46 +01:00
Catherine
088c822e28 CMake: check if warning flag is supported before use.
Clang 11 is failing on -Wno-format-truncation.
2023-02-23 09:44:29 +01:00
Catherine
e94479ccd5
Merge pull request #1108 from whitequark/fix-includes
common: add missing includes for libc++
2023-02-23 04:13:10 +00:00
Catherine
4b4f4a7da1 common: add missing includes for libc++. 2023-02-23 02:32:19 +00:00
myrtle
0c4e0d4312
Merge pull request #1106 from YosysHQ/gatecat/fab_carry
fabulous: Add support for packing carry chains
2023-02-21 15:47:17 +01:00
gatecat
0ed964247e fabulous: Add support for packing carry chains
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-21 14:41:48 +01:00
myrtle
03b6fb3ddb
Merge pull request #1103 from rodgert/master
Include <cstdint> in common/kernel/hashlib.h
2023-02-21 11:50:43 +01:00
myrtle
19fec70aa3
Merge pull request #1105 from YosysHQ/gatecat/nexus-io-error
nexus: Check IO-bank compatibility
2023-02-21 11:50:34 +01:00
gatecat
61021a22ee nexus: Check IO-bank compatibility
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-21 11:18:35 +01:00
Thomas W Rodgers
825d646196 Include <cstdint> in common/kernel/hashlib.h
The definitions for uint32_t, uint64_t report as undefined when
compiling under GCC13. They were previously found by transitive
includes, but this is not guaranteed to work, and GCC13 forced
the issue.
2023-02-18 10:26:01 -08:00
gatecat
16bcc51ffb fabulous: Further tweak magic numbers
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-16 15:53:15 +01:00
myrtle
2275ff78e2
Merge pull request #1102 from rowanG077/print-random-seed
common: Print out generated seed value
2023-02-16 12:55:28 +01:00
myrtle
c65762b5c2
Merge pull request #1101 from YosysHQ/gatecat/fab-fake-timings
fabulous: Add fake timings
2023-02-16 12:18:00 +01:00
rowanG077
32e818204e common: Print out generated seed value 2023-02-16 12:02:00 +01:00
gatecat
06b675b345 fabulous: Add fake timings
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-16 11:56:58 +01:00
myrtle
78dabb7b8f
Merge pull request #1092 from rowanG077/werror
common: Implement Werror flag
2023-02-14 15:03:57 +01:00
Catherine
57693bcb7f
Update links to IceStorm in README
Fixes #1099.
2023-02-14 05:47:16 +00:00
myrtle
dd9426606a
Merge pull request #1098 from YosysHQ/lofty/fix-machxo2-pybindings
machxo2: Fix Python bindings for pip iterators
2023-02-13 14:07:51 +01:00
Lofty
52b02f7377 machxo2: Fix Python bindings for pip iterators 2023-02-13 12:49:00 +00:00
rowanG077
3608c3eb02 common: Implement Werror flag 2023-02-13 10:52:05 +01:00
myrtle
b5125aac31
Merge pull request #1090 from rowanG077/ecp5-propagate-dcsc-clk-ct
ecp5: Propagate clock constraints through DCSC
2023-02-13 10:25:07 +01:00
myrtle
ba3801e010
Merge pull request #1097 from YosysHQ/gatecat/fab-bram-fix
fabulous: Improve names for BRAM bels
2023-02-10 13:41:36 +01:00
gatecat
eb70e95079 fabulous: Improve names for BRAM bels
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-10 13:23:31 +01:00
myrtle
1226fad4f6
Merge pull request #1096 from YosysHQ/gatecat/ecp5-ioce-fix
ecp5: Handle the case where both CE are the same constant
2023-02-10 07:29:10 +01:00
gatecat
a8a88d4813 ecp5: Handle the case where both CE are the same constant
Signed-off-by: gatecat <gatecat@ds0.me>
2023-02-09 11:12:15 +01:00
myrtle
a93f49eb04
Merge pull request #1094 from uis246/master
gowin: Add bels for new types of oscillators
2023-02-07 09:20:59 +01:00
uis
69fe654f02 gowin: Add bels for new types of oscillator 2023-02-06 21:45:55 +00:00
rowanG077
9e8f8b7b45 streamline constant_net detection 2023-02-06 17:05:28 +01:00
rowanG077
d2bf44ba45 ecp5: DSCS clock propagation if modesel is 0 constant 2023-02-06 16:27:45 +01:00
myrtle
48b0025732
Merge pull request #1087 from yrabbit/gw1nr-9
gowin: Add PLL support for the GW1NR-9 chip
2023-02-02 08:30:56 +00:00
YRabbit
2edc77836d Merge branch 'master' into gw1nr-9 2023-02-02 07:35:38 +10:00
rowanG077
a38ee0786a ecp5: Propagate clock constraints through DSCS 2023-02-01 19:12:10 +01:00
myrtle
f328130c0a
Merge pull request #1089 from smunaut/icegate
ice40: Add support for PLL ICEGATE function
2023-02-01 14:34:29 +00:00
Sylvain Munaut
582410629b ice40: Don't assert on unknown extra_config bits if they are 0
Bits are 0 by default anyway, so if they are unknown (because icestorm
is too od) but we want them at 0 ... it's not much of an issue.

Signed-off-by: Sylvain Munaut <tnt@246tNt.com>
2023-02-01 15:14:18 +01:00
Sylvain Munaut
49ae495344 ice40: Add support for PLL ICEGATE function
Technically you can enable it independently on CORE and GLOBAL
output, but this is not exposed in the classic primitive, so
we do the same as icecube2 and enable/disable it for both output
path depending on the argument

Signed-off-by: Sylvain Munaut <tnt@246tNt.com>
2023-02-01 11:41:35 +01:00
myrtle
235e0d28e9
Merge pull request #1088 from rowanG077/ecp5-singleton-lpf
ecp5: LOCATE in LPF works on singleton vector
2023-01-31 22:29:11 +00:00
rowanG077
803c57d052 ecp5: LOCATE in LPF works on singleton vector 2023-01-31 21:05:32 +01:00
YRabbit
5d5ea57e21 gowin: Add PLL support for the GW1NS-2C chip
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-31 18:46:38 +10:00
YRabbit
aac36ecf3f gowin: Add PLL support for GW1NR-4 chips
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-31 08:58:33 +10:00
myrtle
bfa3e047ce
Merge pull request #1086 from smunaut/out_z
ice40: Improve `output` handling vs pull-ups and undriven
2023-01-30 22:06:58 +00:00
YRabbit
2829a7d70a gowin: Proper use of the C++ mechanisms
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-30 18:59:09 +10:00
YRabbit
6a1212a1e1 gowin: Add PLL support for the GW1NR-9 chip
And also unified the fixing of PLL to bels: the point is that PLL being
at a certain location has the possibility to use a direct implicit wire
to the clock source, but once we decide to use this direct wire, the PLL
can no longer be moved.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-30 12:49:57 +10:00
Sylvain Munaut
f50d4c1ed1 ice40: Support for undriven / unconnected output ports
If a port specified as output (and thus had a $nextpnr_obuf inserted)
is undriven (const `z` or const `x`), we make sure to not enable
the output driver. Also enable pull-ups if it was requested by the user.

Signed-off-by: Sylvain Munaut <tnt@246tNt.com>
2023-01-29 22:35:19 +01:00
Sylvain Munaut
0cf5006e3b ice40: Rework pull-up attribute copy to SB_IO blocks
We try to copy the attribute only when there is a chance for
the output driver to not be active.

Note that this can _also_ happen when a port is specified as
output but has a TBUF, which the previous code wasn't handling.

We could copy the attribute "all-the-time" but this would
mean if a user specified a `-pullup yes` in the PCF for a
permanently driven output pin, we'd be burning power for
nothing.

Signed-off-by: Sylvain Munaut <tnt@246tNt.com>
2023-01-29 22:35:19 +01:00
myrtle
f80b871dd5
Merge pull request #1084 from YosysHQ/gatecat/ecp5-ioff-fix
ecp5: Improve IOFF CE handling robustness
2023-01-27 11:20:45 +01:00
myrtle
d661d117af
Merge pull request #1085 from yrabbit/gw1nr-9c-pll
gowin: Add PLL support for the GW1NR-9C chip
2023-01-27 11:20:35 +01:00
YRabbit
2d45d57b32 gowin: Add PLL support for the GW1NR-9C chip
This chip is used in the Tangnano9k board.

  * all parameters of the rPLL primitive are supported;
  * all PLL outputs are treated as clock sources and optimized routing
    is applied to them;
  * primitive rPLL on different chips has a completely different
    structure: for example in GW1N-1 it takes two cells, and in GW1NR-9C
    as many as four, despite this unification was carried out and
    different chips are processed by the same functions, but this led to
    the fact that you can not use the PLL chip GW1N-1 with the old
    apicula bases - will issue a warning and refuse to encode primitive.
    In other cases compatibility is supported.
  * Cosmetic change: the usage report shows the rPLL names without any
    service bels.
  * I use ctx->idf() on occasion, it's not a total redesign.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-26 20:26:05 +10:00
gatecat
9b5e5f124c clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-25 10:29:32 +01:00
gatecat
c8cb063656 ecp5: Improve IOFF CE handling robustness
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-25 09:26:12 +01:00
myrtle
b9ed39bc1c
Merge pull request #1081 from danc86/eigen-cmake-imported-target
use eigen as an IMPORTED target in CMake
2023-01-24 16:33:54 +01:00
Dan Callaghan
4c7e805f18 use eigen as an IMPORTED target in CMake
Eigen considers the EIGEN3_INCLUDE_DIRS and EIGEN3_DEFINITIONS variables
to be deprecated and they will no longer be exported in the next release
after 3.4.0:
f2984cd077

Use the IMPORTED target instead, which seems to be the preferred way of
consuming third-party CMake libraries.
2023-01-24 13:11:50 +11:00
myrtle
985c688bf6
Merge pull request #1080 from YosysHQ/gatecat/missing-includes
Add missing <set> includes
2023-01-23 08:51:31 +01:00
gatecat
7845b66512 Add missing <set> includes
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-20 09:04:41 +01:00
myrtle
06eaffc57c
Merge pull request #1077 from yrabbit/gw1nsr-4c_0
gowin: add a PLL primitive for the GW1NS-4 series
2023-01-19 06:44:46 +01:00
YRabbit
cc45f5ec48 gowin: improve error message
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-19 07:12:39 +10:00
YRabbit
ba4d7b1e9a gowin: to use the FB network detection function
The chip used in tangnano4k does not have such pins, but we call the
function anyway in the expectation of other chips.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-19 06:31:55 +10:00
myrtle
dc2dac1f9e
Merge pull request #1078 from YosysHQ/gatecat/route-delay-quad
context: Add getNetinfoRouteDelayQuad
2023-01-18 17:19:20 +01:00
gatecat
6079326633 context: Add getNetinfoRouteDelayQuad
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-18 16:28:33 +01:00
YRabbit
b22eebac30 gowin: add a PLL primitive for the GW1NS-4 series
* both instances of the new PLLVR type are supported;
  * primitive placement is optimized for the use of dedicated PLL clock
    pins;
  * all 4 outputs of each primitive can use the clock nets (only 5 lines
    in total at the same time so far).

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-18 19:18:02 +10:00
myrtle
a46afc6ff8
Merge pull request #1076 from adamgreig/ecp5-dsp-remap
ECP5: Add DSP signal remapping
2023-01-04 20:01:05 +01:00
Adam Greig
8d8c244e00
Add remapping of DSP clk/ce/rst signals in a block.
Each DSP block contains two slices, and each slice contains multiple
MULT18X18D and ALU54B units. Each unit configures each register to use
any of CLK0/1/2/3, CE0/1/2/3, and RST0/1/2/3 ports, and the ports are
connected per unit (so for example, two MULTs in the same block could
connect their CLK0s to different external signals). However, the
hardware only has one actual port per block, so it's required that
all CLK0 signals within a block are the same.

Because the packer is in general allowed to combine two unrelated units
into one block, it may end up combining units that use different signals
for the same port, which would eventually have caused a router failure.

This commit adds validity checks which ensure only unique signals are
used per block, and adds remapping so that conflicting signals are
automatically reassigned when possible and required.
2023-01-04 18:34:30 +00:00
Adam Greig
174848b4b3
Include ALU54B in cell types with wire location overrides 2023-01-04 13:48:39 +00:00
gatecat
f89b959b5f clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-02 09:33:11 +01:00
myrtle
5cea801a2f
Merge pull request #1075 from YosysHQ/gatecat/ecp5-lpf-errors
ecp5: Improve error handling for missing end-"
2023-01-02 09:25:14 +01:00
gatecat
d210a5aded ecp5: Improve error handling for missing end-"
Signed-off-by: gatecat <gatecat@ds0.me>
2023-01-02 08:39:00 +01:00
myrtle
3338227ef7
Merge pull request #1073 from yrabbit/doc
doc: fix the list format
2023-01-01 10:21:40 +01:00
YRabbit
7c4f44c783 doc: fix the list format
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2023-01-01 10:26:28 +10:00
myrtle
eaf2bc8bdd
Merge pull request #1071 from yrabbit/to-float
gowin: bugfix and improved clock router
2022-12-30 11:58:39 +01:00
YRabbit
b8ab3116b2 gowin: improve clock wire routing
The dedicated router for clock wires now understands not only the IO
pins but also the rPLL outputs as clock sources.

This simple router sets an optimal route, so it is now the default
router. It can be disabled with the --disable-globals command line flag
if desired, but this is not recommended due to possible clock skew.

Still for GW1N-4C there is no good router for clock wires as there
external quartz resonator is connected via PLL.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-30 11:55:39 +10:00
YRabbit
8424dc79d2 gowin: correct the delay calculation
And do a full enumeration when searching for a delay because it is not
yet clear whether the orderliness of the vector is guaranteed.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-29 11:17:45 +10:00
myrtle
0004cd54db
Merge pull request #1069 from yrabbit/mistype
doc: fix a mistype
2022-12-27 15:32:09 +01:00
YRabbit
2e5c799566 doc: fix a mistype
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-23 17:14:26 +10:00
myrtle
76fea8268c
Merge pull request #1068 from YosysHQ/cleanup_and_sync
Cleanup and sync
2022-12-22 21:19:21 +01:00
Miodrag Milanovic
64f7306b24 initialize netShareWeight 2022-12-22 20:16:13 +01:00
Miodrag Milanovic
4af8964069 propagate netShareWeight 2022-12-22 16:11:10 +01:00
Miodrag Milanovic
bd628ce591 Remove deprecated functions 2022-12-22 15:26:39 +01:00
myrtle
a80d63b268
Merge pull request #1066 from arjenroodselaar/place_timeout
Timeout when legal placement can't be found for cell
2022-12-21 07:10:09 +00:00
myrtle
b101f0092c
Merge pull request #1067 from yrabbit/wasm
gowin: fix build for wasm
2022-12-21 07:09:27 +00:00
YRabbit
d6cbe4b7f8 gowin: fix build for wasm
A large number of global variables are not suitable for WASM, so
completely disable the graphics part where the main array of them is
used.  For other architectures GUI is still possible.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-21 16:13:08 +10:00
Arjen Roodselaar
be1f700b0b Set divisor instead of absolute value 2022-12-20 13:10:37 -08:00
Arjen Roodselaar
923458a2c9 Allow setting cell placement timeout 2022-12-20 11:15:06 -08:00
Arjen Roodselaar
d5299f144f Add --no-placer-timeout flag to override timeout during refinement 2022-12-19 22:58:52 -08:00
Arjen Roodselaar
2712cbf6e4 Increase timeout 2022-12-19 14:00:19 -08:00
myrtle
3ea3a931ca
Merge pull request #1065 from YosysHQ/gatecat/heap-chains-fix
heap: encourage more spreading of heterogenous chains
2022-12-19 08:44:26 +00:00
Arjen Roodselaar
6e0311efca Timeout when legal placement can't be found for cell 2022-12-17 16:07:57 -08:00
myrtle
78926b31db
Merge pull request #1064 from YosysHQ/gatecat/ecp5-main-fix
ecp5: Only write bitstream if --textcfg passed
2022-12-17 12:22:36 +00:00
gatecat
ccb573298c heap: encourage more spreading of heterogenous chains
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-17 10:50:20 +00:00
gatecat
bc18d18a95 ecp5: Only write bitstream if --textcfg passed
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-17 10:37:15 +00:00
myrtle
16ffd02a9d
Merge pull request #1061 from yrabbit/fix-clock-gui
gowin: not crush on unknown clock tap's sources
2022-12-14 08:00:02 +01:00
YRabbit
bc3d9f3108 gowin: not crush on unknown clock tap's sources
As preparation for possible changes to the clock wiring system.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-14 15:35:55 +10:00
myrtle
b5d30c7387
Merge pull request #1060 from yrabbit/pll-inputs
gowin: BUGFIX: Correctly handle resets
2022-12-09 09:27:58 +01:00
YRabbit
aa8359c73e gowin: BUGFIX: Correctly handle resets
When a single primitive occupies several cells, care must be taken when
manipulating the parameters of that primitive: when creating cells, each
cell must receive a copy of all the parameters and not modify them
unnecessarily.  That is, if possible, it is better to make all parameter
changes before dividing the primitive into cells.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-09 12:55:22 +10:00
myrtle
0eb53d59d8
Merge pull request #1059 from YosysHQ/gatecat/validity-errors
Add new option for verbose validity errors, use for ice40
2022-12-07 16:19:55 +01:00
Sean Anderson
df99b4ff6c ice40: Add debugs to isBelLocationValid for SB_IO
When there is a constraint conflict while placing IOs, the user gets an
error message such as

ERROR: Bel 'X0/Y27/io1' of type 'SB_IO' is not valid for cell 'my_pin' of type 'SB_IO'

While this identifies the problematic cell, it does not explain why
there is a problem. Add some verbose messages to allow users to
determine where the problem is. This can result in something like

Info: Net '$PACKER_VCC_NET' for cell 'my_pin' conflicts with net 'ce' for 'ce_pin'

which provides something actionable.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
2022-12-07 10:32:38 +01:00
gatecat
603b60da8d api: add explain_invalid option to isBelLocationValid
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-07 10:27:58 +01:00
myrtle
519011533a
Merge pull request #1058 from YosysHQ/gatecat/bounds-refactor
refactor: rename ArcBounds -> BoundingBox and use this in HeAP
2022-12-07 10:26:17 +01:00
gatecat
d1afd6c0f1 heap: Remove custom bounding-box type
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-07 10:02:16 +01:00
gatecat
e260ac33ab refactor: ArcBounds -> BoundingBox
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-07 10:00:53 +01:00
myrtle
a342b96bb0
Merge pull request #1055 from yrabbit/pll-pins
gowin: add PLL pins processing
2022-12-06 21:20:59 +01:00
YRabbit
150a482b77 gowin: change the way networks are handled
Until a comprehensive clock router is developed, the order in which
private cases are handled is important.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-06 23:07:01 +10:00
myrtle
cd3b76e3f7
Merge pull request #1056 from YosysHQ/gatecat/generic-fix-consts
viaduct: Fix constant connectivity
2022-12-06 12:27:03 +01:00
gatecat
3a61bb4119 viaduct: Fix constant connectivity
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-06 10:04:59 +01:00
YRabbit
e6a8d0f4fc Merge branch 'master' into pll-pins 2022-12-04 21:33:36 +10:00
myrtle
db25c5c889
Merge pull request #1054 from YosysHQ/gatecat/api-add-const
api: Make NetInfo* of checkPipAvailForNet const
2022-12-04 12:27:53 +01:00
YRabbit
2e68962a02 gowin: add PLL pins processing
Uses the information of the special input pins for the PLL in the
current chip. If such pins are involved, no routing is performed and
information about the use of implicit wires is passed to the packer.

The RESET and RESET_P inputs are now also disabled if they are connected
to VSS/VCC.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-04 15:06:44 +10:00
gatecat
91454515f4 Unbreak CI
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-02 14:26:13 +01:00
gatecat
c62a947a28 api: Make NetInfo* of checkPipAvailForNet const
Signed-off-by: gatecat <gatecat@ds0.me>
2022-12-02 14:20:39 +01:00
myrtle
f07d9a1835
Merge pull request #1048 from yrabbit/chipdb-cfg
gowin: add information about pin configurations
2022-12-02 09:58:46 +01:00
YRabbit
b0791a01c9 gowin: update the apicula version
Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-12-02 08:49:56 +10:00
YRabbit
7638146782 Merge branch 'master' into chipdb-cfg 2022-12-02 08:28:51 +10:00
myrtle
719f89806a
Merge pull request #1053 from YosysHQ/gatecat/pbfix
ecp5: Fix Python bindings for pip iterators
2022-11-28 09:45:11 +01:00
gatecat
6ee3daf06a ecp5: Fix Python bindings for pip iterators
Signed-off-by: gatecat <gatecat@ds0.me>
2022-11-28 09:00:41 +01:00
YRabbit
ec53ae0c3b gowin: add information about pin configurations
Includes information on additional pin functions such as RPLL_C_IN, GCLKC_3, SCLK and others.
This allows a decision to be made about special network routing of such pins

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-11-25 20:49:26 +10:00
myrtle
c61d490bd4
Merge pull request #1045 from yrabbit/unused-ports
gowin: mark the PLL ports that are not in use
2022-11-20 13:55:01 +01:00
YRabbit
378ca60a2f gowin: mark the PLL ports that are not in use
Unused ports are deactivated by special fuse combinations, rather than
being left dangling in the air.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-11-20 22:04:09 +10:00
myrtle
c8406b71fe
Merge pull request #1042 from yrabbit/add-z1
gowin: add support for a more common chip
2022-11-12 11:17:06 +01:00
YRabbit
d4642d918c gowin: add support for a more common chip
The GW1N-1 and GW1NZ-1 have a similar PLL, but the board with the former
chip is already very hard to buy, so let's experiment with a more
affordable chip.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-11-12 10:12:43 +10:00
myrtle
1aa9cda77a
Merge pull request #1040 from yrabbit/pll-stage0
gowin: add initial PLL support
2022-11-11 10:28:19 +01:00
YRabbit
9013b2de50 gowin: use ctx->idf() a bit
Replacing snprintf() with ctx->idf() in PLL commit, but not yet a
complete overhaul.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-11-11 09:19:16 +10:00
myrtle
79cb2f9e20
Merge pull request #1041 from YosysHQ/gatecat/fix-copy-warning
Fix "implicit copy constructor for 'Property' is deprecated"
2022-11-10 15:40:50 +01:00
gatecat
8a69bd0735 Fix "implicit copy constructor for 'Property' is deprecated"
Signed-off-by: gatecat <gatecat@ds0.me>
2022-11-10 10:57:41 +01:00
gatecat
6930ab3acd fabulous: Tweak delay estimate
Signed-off-by: gatecat <gatecat@ds0.me>
2022-11-10 10:55:37 +01:00
YRabbit
a84ded4793 gowin: add initial PLL support
The rPLL primitive for the simplest chip (GW1N-1) in the family is
processed. All parameters of the primitive are passed on to gowin_pack,
and general-purpose wires are used for routing outputs of the primitive.

Compatible with older versions of apicula, but in this case will refuse
to place the new primitive.

Signed-off-by: YRabbit <rabbit@yrabbit.cyou>
2022-11-10 19:14:41 +10:00
Miodrag Milanović
ac17c36bec
Merge pull request #1037 from YosysHQ/fix_python_ver
Fix python version in CI
2022-10-24 09:45:57 +02:00
Miodrag Milanovic
4ffa47d897 Fix python version in CI 2022-10-24 09:42:16 +02:00
Miodrag Milanovic
010b2e5ecf Update CI script 2022-10-24 09:28:34 +02:00
gatecat
445d32497d run clangformat
Signed-off-by: gatecat <gatecat@ds0.me>
2022-10-17 12:35:02 +02:00
myrtle
bd082132ce
Merge pull request #1034 from lushaylabs/support-windows-crlf
Support windows line endings in constraints for nextpnr-gowin
2022-10-17 12:33:33 +02:00
myrtle
c2dbaa2b11
Merge pull request #1035 from tyler274/patch-1
Correct Not Equal operator implementation in ice40
2022-10-17 10:43:34 +02:00
Tyler
613d84fb72
Correct Not Equal operator implementation in ice40
I noticed this during my work reimplementing nextpnr, and it seems to be dead and wrong, or at least dead. Either way I think this is what was intended unless anyone can correct me.
2022-10-17 01:19:51 -07:00
Lushay Labs
a7acda95f0
support windows line endings 2022-10-09 23:47:09 +03:00
myrtle
0d1ea9e6ed
Merge pull request #1032 from davidlattimore/registered-output-xform
nexus: Transform registered output parameters
2022-10-05 11:07:42 +02:00
David Lattimore
1602774d27 nexus: Transform registered output parameters
Dual ported:
OUTREG_A -> OUT_REGMODE_A
OUTREG_B -> OUT_REGMODE_B

Pseudo dual ported:
OUTREG -> OUT_REGMODE_B

Single ported:
OUTREG -> OUT_REGMODE_A
2022-10-05 14:40:49 +11:00
myrtle
41709dac8f
Merge pull request #1031 from YosysHQ/gatecat/fab-next
fabulous: Add support for the CLB muxes
2022-09-30 16:13:32 +02:00
gatecat
3826a31ad3 fabulous: Pack, validity check and FASM support for muxes
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-30 13:27:51 +02:00
gatecat
124c0fc812 fabulous: Add split MUX bels
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-30 12:03:16 +02:00
myrtle
c44b034fc3
Merge pull request #1030 from YosysHQ/gatecat/ice40-dsp25_10-fix
ice40: Fix handling of carry out route-thru via 25,14
2022-09-26 11:48:48 +02:00
gatecat
a16d184956 ice40: Fix handling of carry out route-thru via 25,14
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-26 09:33:38 +02:00
myrtle
f0f9070adb
Merge pull request #1029 from airskywater/airskywater-patch-1
Fix runtime segmentation fault
2022-09-24 10:30:30 +02:00
airskywater
9572f6f032
Modify code to meet the code style preferences 2022-09-24 14:46:35 +08:00
airskywater
c702e15a3f
Add more sanity check for pointers 2022-09-24 12:03:44 +08:00
airskywater
78f67ae5bc
fix runtime segmentation fault
disable null pointer dereference!
2022-09-24 11:35:40 +08:00
myrtle
f4e6bbd383
Merge pull request #1019 from antmicro/support-clock-relations
Support cross-domain clock relations in timing analyser
2022-09-20 15:55:43 +02:00
Maciej Kurc
9000c41c4b Added the --ignore-rel-clk option to control timing checks for cross-domain paths, formatted code
Signed-off-by: Maciej Kurc <mkurc@antmicro.com>
2022-09-20 14:40:40 +02:00
myrtle
136ab81cbd
Merge pull request #1028 from YosysHQ/gatecat/router2-reserve-src
router2: Reserve source wire, too; ice40 fixes
2022-09-20 14:37:55 +02:00
gatecat
a920ffcf70 ice40: implement checkPipAvailForNet
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-20 14:15:10 +02:00
gatecat
415c097df8 router2: Reserve source wire, too
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-20 13:42:51 +02:00
gatecat
376cedd558 fabulous: fix, but disable, IO configuration
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-16 09:32:15 +02:00
myrtle
a3a641f449
Merge pull request #1026 from YosysHQ/gatecat/ecp5-bitstream-refactor
ecp5: Split bitstream generation into more functions
2022-09-16 09:16:35 +02:00
myrtle
d58e85f297
Merge pull request #1023 from YosysHQ/gatecat/ice40up-bram-pol
ice40: Fix UltraPlus BRAM clock polarity
2022-09-16 06:38:04 +02:00
myrtle
e5da8be4f8
Merge pull request #1025 from YosysHQ/gatecat/nexus-dev-fixes
nexus: Add ES2 device names and --list-devices
2022-09-15 18:03:57 +02:00
gatecat
9e272810d8 ecp5: Split bitstream generation into more functions
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-15 13:28:43 +02:00
gatecat
7ca3ba3835 nexus: Add ES2 device names and --list-devices
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-15 12:27:36 +02:00
myrtle
79aad0988a
Merge pull request #1015 from YosysHQ/gatecat/fabulous-viaduct
fabulous: Add a viaduct uarch
2022-09-15 09:07:56 +02:00
myrtle
3983d4fe53
Merge pull request #1024 from YosysHQ/gatecat/pybind11-bump
3rdparty: Bump vendored pybind11 version for py3.11 support
2022-09-15 09:06:35 +02:00
gatecat
a72f898ff4 3rdparty: Bump vendored pybind11 version for py3.11 support
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-14 09:28:47 +02:00
gatecat
0a8c411692 ice40: Fix UltraPlus BRAM clock polarity
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-14 09:24:49 +02:00
gatecat
f423055390 fabulous: Add a viaduct uarch
Signed-off-by: gatecat <gatecat@ds0.me>
2022-09-09 14:48:57 +02:00
Maciej Kurc
1f1bae3e23 Code cleanup
Signed-off-by: Maciej Kurc <mkurc@antmicro.com>
2022-08-31 16:19:15 +02:00
Maciej Kurc
60a6e8b070 Added timing check for cross-domain paths for related clocks
Signed-off-by: Maciej Kurc <mkurc@antmicro.com>
2022-08-31 14:15:33 +02:00
Maciej Kurc
9a61ad9234 Augmented TimingAnalyser class with detection of clock to clock relations
Signed-off-by: Maciej Kurc <mkurc@antmicro.com>
2022-08-30 17:30:58 +02:00
Maciej Kurc
8b6be09809 Fixed port timing classes of DCC ports in the Nexus architecture
Signed-off-by: Maciej Kurc <mkurc@antmicro.com>
2022-08-30 17:30:13 +02:00
750 changed files with 96260 additions and 41466 deletions

23
.github/ci/build_himbaechel.sh vendored Normal file
View File

@ -0,0 +1,23 @@
#!/bin/bash
function get_dependencies {
:
}
function build_nextpnr {
mkdir build
pushd build
cmake .. -DARCH=himbaechel -DHIMBAECHEL_UARCH=example -DHIMBAECHEL_EXAMPLE_DEVICES=example
make nextpnr-himbaechel -j`nproc`
popd
}
function run_tests {
:
}
function run_archcheck {
pushd build
./nextpnr-himbaechel --device EXAMPLE --test
popd
}

View File

@ -1,64 +0,0 @@
#!/bin/bash
# Install capnproto libraries
function build_capnp {
curl -O https://capnproto.org/capnproto-c++-0.8.0.tar.gz
tar zxf capnproto-c++-0.8.0.tar.gz
pushd capnproto-c++-0.8.0
./configure
make -j`nproc` check
sudo make install
popd
git clone https://github.com/capnproto/capnproto-java.git
pushd capnproto-java
make -j`nproc`
sudo make install
popd
}
# Install latest Yosys
function build_yosys {
DESTDIR=`pwd`/.yosys
pushd yosys
make -j`nproc`
sudo make install DESTDIR=$DESTDIR
popd
}
function get_dependencies {
# Install python-fpga-interchange libraries
git clone -b ${PYTHON_INTERCHANGE_TAG} https://github.com/SymbiFlow/python-fpga-interchange.git ${PYTHON_INTERCHANGE_PATH}
pushd ${PYTHON_INTERCHANGE_PATH}
git submodule update --init --recursive
python3 -m pip install -r requirements.txt
popd
if [ ${DEVICE} == "LIFCL-17" ] || [ ${DEVICE} == "LIFCL-40" ]; then
# Install prjoxide
curl --proto '=https' -sSf https://sh.rustup.rs | sh -s -- -y
git clone --recursive https://github.com/gatecat/prjoxide.git
pushd prjoxide/libprjoxide
# TODO: use a tag instead of a commit, like python-fpga-interchange
git reset --hard ${PRJOXIDE_REVISION}
PATH=$PATH:$HOME/.cargo/bin cargo install --path prjoxide --all-features
popd
else
# Install RapidWright
git clone https://github.com/Xilinx/RapidWright.git ${RAPIDWRIGHT_PATH}
pushd ${RAPIDWRIGHT_PATH}
./gradlew updateJars --no-watch-fs
make compile
popd
fi
}
function build_nextpnr {
build_capnp
mkdir build
pushd build
cmake .. -DARCH=fpga_interchange -DRAPIDWRIGHT_PATH=${RAPIDWRIGHT_PATH} -DPYTHON_INTERCHANGE_PATH=${PYTHON_INTERCHANGE_PATH} -DWERROR=on
make nextpnr-fpga_interchange -j`nproc`
popd
}

View File

@ -17,5 +17,8 @@ function run_tests {
} }
function run_archcheck { function run_archcheck {
: pushd build
./nextpnr-machxo2 --device LCMXO2-1200HC-4SG32C --test
./nextpnr-machxo2 --device LCMXO3LF-6900C-6BG256C --test
popd
} }

View File

@ -9,34 +9,35 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
arch: [mistral, ice40, ecp5, generic, nexus, machxo2, gowin] arch: [mistral, ice40, ecp5, generic, nexus, machxo2, gowin, himbaechel]
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
DEPS_PATH: ${{ github.workspace }}/deps DEPS_PATH: ${{ github.workspace }}/deps
YOSYS_REVISION: bd7ee79486d4e8788f36de8c25a3fb2df451d682 YOSYS_REVISION: 7045cf509e1d95cbc973746674cf2d7c73c02e50
ICESTORM_REVISION: 9f66f9ce16941c6417813cb87653c735a78b53ae ICESTORM_REVISION: 68044cc4dac829729ccd0ee88d0780525b515746
TRELLIS_REVISION: 48486ebd1e03e4ac42c96299e881adf9d43bc241 TRELLIS_REVISION: 36c615d1740473cc3574464c7f0bed44da20e5b6
PRJOXIDE_REVISION: c3fb1526cf4a2165e15b74f4a994d153c7695fe4 PRJOXIDE_REVISION: c3fb1526cf4a2165e15b74f4a994d153c7695fe4
MISTRAL_REVISION: ebfc0dd2cc7d6d2159b641a397c88554840e93c9 MISTRAL_REVISION: ebfc0dd2cc7d6d2159b641a397c88554840e93c9
APYCULA_REVISION: 0.2a4 APYCULA_REVISION: 0.8.2a1
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
with: with:
submodules: recursive submodules: recursive
- uses: actions/setup-python@v2 - uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install - name: Install
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install git make cmake libboost-all-dev python3-dev libeigen3-dev tcl-dev lzma-dev libftdi-dev clang bison flex swig qt5-default iverilog sudo apt-get install git make cmake libboost-all-dev python3-dev pypy3 libeigen3-dev tcl-dev lzma-dev libftdi-dev clang bison flex swig qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools iverilog libreadline-dev liblzma-dev
- name: Cache yosys installation - name: Cache yosys installation
uses: actions/cache@v2 uses: actions/cache@v4
id: cache-yosys id: cache-yosys
with: with:
path: .yosys path: .yosys
key: cache-yosys-${{ env.YOSYS_REVISION }}-r2 key: cache-yosys-${{ env.YOSYS_REVISION }}-r3
- name: Build yosys - name: Build yosys
run: | run: |
@ -45,11 +46,11 @@ jobs:
if: steps.cache-yosys.outputs.cache-hit != 'true' if: steps.cache-yosys.outputs.cache-hit != 'true'
- name: Cache icestorm installation - name: Cache icestorm installation
uses: actions/cache@v2 uses: actions/cache@v4
id: cache-icestorm id: cache-icestorm
with: with:
path: .icestorm path: .icestorm
key: cache-icestorm-${{ env.ICESTORM_REVISION }} key: cache-icestorm-${{ env.ICESTORM_REVISION }}-r3
if: matrix.arch == 'ice40' if: matrix.arch == 'ice40'
- name: Build icestorm - name: Build icestorm
@ -59,11 +60,11 @@ jobs:
if: matrix.arch == 'ice40' && steps.cache-icestorm.outputs.cache-hit != 'true' if: matrix.arch == 'ice40' && steps.cache-icestorm.outputs.cache-hit != 'true'
- name: Cache trellis installation - name: Cache trellis installation
uses: actions/cache@v2 uses: actions/cache@v4
id: cache-trellis id: cache-trellis
with: with:
path: .trellis path: .trellis
key: cache-trellis-${{ env.TRELLIS_REVISION }} key: cache-trellis-${{ env.TRELLIS_REVISION }}-r3
if: matrix.arch == 'ecp5' || matrix.arch == 'machxo2' if: matrix.arch == 'ecp5' || matrix.arch == 'machxo2'
- name: Build trellis - name: Build trellis
@ -73,11 +74,11 @@ jobs:
if: (matrix.arch == 'ecp5' || matrix.arch == 'machxo2') && steps.cache-trellis.outputs.cache-hit != 'true' if: (matrix.arch == 'ecp5' || matrix.arch == 'machxo2') && steps.cache-trellis.outputs.cache-hit != 'true'
- name: Cache prjoxide installation - name: Cache prjoxide installation
uses: actions/cache@v2 uses: actions/cache@v4
id: cache-prjoxide id: cache-prjoxide
with: with:
path: .prjoxide path: .prjoxide
key: cache-prjoxide-${{ env.PRJOXIDE_REVISION }} key: cache-prjoxide-${{ env.PRJOXIDE_REVISION }}-r3
if: matrix.arch == 'nexus' if: matrix.arch == 'nexus'
- name: Build prjoxide - name: Build prjoxide

View File

@ -1,132 +0,0 @@
name: FPGA interchange CI tests
on: [push, pull_request]
jobs:
Build-yosys:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v2
- name: Install
run: |
sudo apt-get update
sudo apt-get install git make cmake libboost-all-dev python3-dev libeigen3-dev tcl-dev clang bison flex swig locales libtinfo-dev
- name: ccache
uses: hendrikmuhs/ccache-action@v1
- name: Get yosys
run: |
git clone https://github.com/YosysHQ/yosys.git
cd yosys
echo "YOSYS_SHA=$(git rev-parse HEAD)" >> $GITHUB_ENV
- name: Cache yosys installation
uses: actions/cache@v2
id: cache-yosys
with:
path: .yosys
key: cache-yosys-${{ env.YOSYS_SHA }}
- name: Build yosys
run: |
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
source ./.github/ci/build_interchange.sh
build_yosys
if: steps.cache-yosys.outputs.cache-hit != 'true'
Build-nextpnr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v2
- name: Install
run: |
sudo apt-get update
sudo apt-get install git make cmake libboost-all-dev python3-dev libeigen3-dev tcl-dev clang bison flex swig
- name: ccache
uses: hendrikmuhs/ccache-action@v1
- name: Execute build nextpnr
run: |
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
source ./.github/ci/build_interchange.sh
build_nextpnr
Run-Tests:
runs-on: ubuntu-latest
needs: [Build-yosys, Build-nextpnr]
strategy:
# Don't terminate jobs when one fails. This is important when
# debugging CI failures.
fail-fast: false
matrix:
device: [xc7a35t, xc7a100t, xc7a200t, xc7z010, LIFCL-17, LIFCL-40]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v2
- name: Install
run: |
sudo apt-get update
sudo apt-get install git make cmake libboost-all-dev python3-dev libeigen3-dev tcl-dev clang bison flex swig
- name: ccache
uses: hendrikmuhs/ccache-action@v1
- name: Get yosys
run: |
git clone https://github.com/YosysHQ/yosys.git
cd yosys
echo "YOSYS_SHA=$(git rev-parse HEAD)" >> $GITHUB_ENV
- name: Cache yosys installation
uses: actions/cache@v2
id: cache-yosys
with:
path: .yosys
key: cache-yosys-${{ env.YOSYS_SHA }}
- name: Build yosys
run: |
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
source ./.github/ci/build_interchange.sh
build_yosys
if: steps.cache-yosys.outputs.cache-hit != 'true'
- name: Execute build interchange script
env:
RAPIDWRIGHT_PATH: ${{ github.workspace }}/RapidWright
PYTHON_INTERCHANGE_PATH: ${{ github.workspace }}/python-fpga-interchange
PYTHON_INTERCHANGE_TAG: v0.0.20
PRJOXIDE_REVISION: 318331f8b30c2e2a31cc41d51f104b671e180a8a
DEVICE: ${{ matrix.device }}
run: |
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
source ./.github/ci/build_interchange.sh
build_nextpnr && get_dependencies
- name: Run tests
env:
DEVICE: ${{ matrix.device }}
run: |
export PATH="$GITHUB_WORKSPACE/.yosys/usr/local/bin:$PATH"
which yosys
cd build
make all-$DEVICE-tests -j`nproc`

4
.gitignore vendored
View File

@ -1,4 +1,3 @@
/generated/
/objs/ /objs/
/nextpnr-generic* /nextpnr-generic*
/nextpnr-ice40* /nextpnr-ice40*
@ -7,6 +6,8 @@
/nextpnr-fpga_interchange* /nextpnr-fpga_interchange*
/nextpnr-gowin* /nextpnr-gowin*
/nextpnr-machxo2* /nextpnr-machxo2*
/nextpnr-himbaechel*
.cache
cmake-build-*/ cmake-build-*/
Makefile Makefile
cmake_install.cmake cmake_install.cmake
@ -27,6 +28,7 @@ a.out
*.il *.il
/generic/examples/blinky.png /generic/examples/blinky.png
build/ build/
share/
*.asc *.asc
*.bin *.bin
/Testing/* /Testing/*

9
.gitmodules vendored
View File

@ -1,6 +1,9 @@
[submodule "tests"] [submodule "tests"]
path = tests path = tests
url = https://github.com/YosysHQ/nextpnr-tests url = https://github.com/YosysHQ/nextpnr-tests
[submodule "fpga-interchange-schema"] [submodule "himbaechel/uarch/xilinx/meta"]
path = 3rdparty/fpga-interchange-schema path = himbaechel/uarch/xilinx/meta
url = https://github.com/SymbiFlow/fpga-interchange-schema.git url = https://github.com/gatecat/nextpnr-xilinx-meta
[submodule "3rdparty/corrosion"]
path = 3rdparty/corrosion
url = https://github.com/corrosion-rs/corrosion

View File

@ -1,4 +1,5 @@
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.11) CMAKE_MINIMUM_REQUIRED(VERSION 3.13)
PROJECT(QtPropertyBrowser) PROJECT(QtPropertyBrowser)
##################### Look for required libraries ###################### ##################### Look for required libraries ######################

1
3rdparty/corrosion vendored Submodule

@ -0,0 +1 @@
Subproject commit be76480232216a64f65e3b1d9794d68cbac6c690

@ -1 +0,0 @@
Subproject commit 6b2973788692be86c4a8b2cff1353e603e5857a3

View File

@ -1004,7 +1004,7 @@ void StackLowerThanAddress(const void* ptr, bool* result) {
// Make sure AddressSanitizer does not tamper with the stack here. // Make sure AddressSanitizer does not tamper with the stack here.
GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_
bool StackGrowsDown() { bool StackGrowsDown() {
int dummy; int dummy = 0;
bool result; bool result;
StackLowerThanAddress(&dummy, &result); StackLowerThanAddress(&dummy, &result);
return result; return result;

View File

@ -13,6 +13,7 @@
#ifndef WIN32_LEAN_AND_MEAN #ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN
#endif #endif
#define NOMINMAX
#include <windows.h> #include <windows.h>
#include <tchar.h> #include <tchar.h>

View File

@ -7,6 +7,7 @@
#ifdef _WIN32 #ifdef _WIN32
#define WIN32_LEAN_AND_MEAN 1 #define WIN32_LEAN_AND_MEAN 1
#define NOMINMAX
#include <windows.h> #include <windows.h>
static HMODULE libgl; static HMODULE libgl;

View File

@ -57,6 +57,7 @@ extern "C" {
#if defined(_WIN32) && !defined(APIENTRY) && !defined(__CYGWIN__) && !defined(__SCITECH_SNAP__) #if defined(_WIN32) && !defined(APIENTRY) && !defined(__CYGWIN__) && !defined(__SCITECH_SNAP__)
#define WIN32_LEAN_AND_MEAN 1 #define WIN32_LEAN_AND_MEAN 1
#define NOMINMAX
#include <windows.h> #include <windows.h>
#endif #endif

View File

@ -85,7 +85,8 @@ extern "C" {
// example to allow applications to correctly declare a GL_ARB_debug_output // example to allow applications to correctly declare a GL_ARB_debug_output
// callback) but windows.h assumes no one will define APIENTRY before it does // callback) but windows.h assumes no one will define APIENTRY before it does
#undef APIENTRY #undef APIENTRY
#include <windows.h> #define NOMINMAX
#include <windows.h>
#elif defined(GLFW_EXPOSE_NATIVE_COCOA) #elif defined(GLFW_EXPOSE_NATIVE_COCOA)
#include <ApplicationServices/ApplicationServices.h> #include <ApplicationServices/ApplicationServices.h>
#if defined(__OBJC__) #if defined(__OBJC__)

View File

@ -8723,8 +8723,10 @@ static void SettingsHandlerWindow_WriteAll(ImGuiContext* imgui_ctx, ImGuiSetting
#define WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN
#endif #endif
#ifndef __MINGW32__ #ifndef __MINGW32__
#include <Windows.h> #define NOMINMAX
#include <windows.h>
#else #else
#define NOMINMAX
#include <windows.h> #include <windows.h>
#endif #endif
#endif #endif

5
3rdparty/json11/CMakeLists.txt vendored Normal file
View File

@ -0,0 +1,5 @@
add_library(json11 STATIC
json11.cpp
json11.hpp
)
target_include_directories(json11 PUBLIC .)

View File

@ -23,6 +23,7 @@
#include <cassert> #include <cassert>
#include <cmath> #include <cmath>
#include <cstdlib> #include <cstdlib>
#include <cstdint>
#include <cstdio> #include <cstdio>
#include <climits> #include <climits>
#include <cerrno> #include <cerrno>
@ -152,7 +153,7 @@ protected:
// Constructors // Constructors
explicit Value(const T &value) : m_value(value) {} explicit Value(const T &value) : m_value(value) {}
explicit Value(T &&value) : m_value(move(value)) {} explicit Value(T &&value) : m_value(std::move(value)) {}
// Get type tag // Get type tag
Json::Type type() const override { Json::Type type() const override {
@ -199,7 +200,7 @@ class JsonString final : public Value<Json::STRING, string> {
const string &string_value() const override { return m_value; } const string &string_value() const override { return m_value; }
public: public:
explicit JsonString(const string &value) : Value(value) {} explicit JsonString(const string &value) : Value(value) {}
explicit JsonString(string &&value) : Value(move(value)) {} explicit JsonString(string &&value) : Value(std::move(value)) {}
}; };
class JsonArray final : public Value<Json::ARRAY, Json::array> { class JsonArray final : public Value<Json::ARRAY, Json::array> {
@ -207,7 +208,7 @@ class JsonArray final : public Value<Json::ARRAY, Json::array> {
const Json & operator[](size_t i) const override; const Json & operator[](size_t i) const override;
public: public:
explicit JsonArray(const Json::array &value) : Value(value) {} explicit JsonArray(const Json::array &value) : Value(value) {}
explicit JsonArray(Json::array &&value) : Value(move(value)) {} explicit JsonArray(Json::array &&value) : Value(std::move(value)) {}
}; };
class JsonObject final : public Value<Json::OBJECT, Json::object> { class JsonObject final : public Value<Json::OBJECT, Json::object> {
@ -215,7 +216,7 @@ class JsonObject final : public Value<Json::OBJECT, Json::object> {
const Json & operator[](const string &key) const override; const Json & operator[](const string &key) const override;
public: public:
explicit JsonObject(const Json::object &value) : Value(value) {} explicit JsonObject(const Json::object &value) : Value(value) {}
explicit JsonObject(Json::object &&value) : Value(move(value)) {} explicit JsonObject(Json::object &&value) : Value(std::move(value)) {}
}; };
class JsonNull final : public Value<Json::NUL, NullStruct> { class JsonNull final : public Value<Json::NUL, NullStruct> {
@ -257,12 +258,12 @@ Json::Json(double value) : m_ptr(make_shared<JsonDouble>(value)) {
Json::Json(int value) : m_ptr(make_shared<JsonInt>(value)) {} Json::Json(int value) : m_ptr(make_shared<JsonInt>(value)) {}
Json::Json(bool value) : m_ptr(value ? statics().t : statics().f) {} Json::Json(bool value) : m_ptr(value ? statics().t : statics().f) {}
Json::Json(const string &value) : m_ptr(make_shared<JsonString>(value)) {} Json::Json(const string &value) : m_ptr(make_shared<JsonString>(value)) {}
Json::Json(string &&value) : m_ptr(make_shared<JsonString>(move(value))) {} Json::Json(string &&value) : m_ptr(make_shared<JsonString>(std::move(value))) {}
Json::Json(const char * value) : m_ptr(make_shared<JsonString>(value)) {} Json::Json(const char * value) : m_ptr(make_shared<JsonString>(value)) {}
Json::Json(const Json::array &values) : m_ptr(make_shared<JsonArray>(values)) {} Json::Json(const Json::array &values) : m_ptr(make_shared<JsonArray>(values)) {}
Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(move(values))) {} Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(std::move(values))) {}
Json::Json(const Json::object &values) : m_ptr(make_shared<JsonObject>(values)) {} Json::Json(const Json::object &values) : m_ptr(make_shared<JsonObject>(values)) {}
Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(move(values))) {} Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(std::move(values))) {}
/* * * * * * * * * * * * * * * * * * * * /* * * * * * * * * * * * * * * * * * * *
* Accessors * Accessors
@ -360,7 +361,7 @@ struct JsonParser final {
* Mark this parse as failed. * Mark this parse as failed.
*/ */
Json fail(string &&msg) { Json fail(string &&msg) {
return fail(move(msg), Json()); return fail(std::move(msg), Json());
} }
template <typename T> template <typename T>

9
3rdparty/oourafft/CMakeLists.txt vendored Normal file
View File

@ -0,0 +1,9 @@
add_library(oourafft INTERFACE)
target_include_directories(oourafft INTERFACE .)
target_sources(oourafft PUBLIC
fftsg.cc
fftsg.h
fftsg2d.cc
)

3259
3rdparty/oourafft/fftsg.cc vendored Normal file

File diff suppressed because it is too large Load Diff

24
3rdparty/oourafft/fftsg.h vendored Normal file
View File

@ -0,0 +1,24 @@
#ifndef FFTSG_H
#define FFTSG_H
#include "nextpnr_namespaces.h"
NEXTPNR_NAMESPACE_BEGIN
//
// The following FFT library came from
// http://www.kurims.kyoto-u.ac.jp/~ooura/fft.html
//
//
/// 1D FFT ////////////////////////////////////////////////////////////////
void ddct(int n, int isgn, float *a, int *ip, float *w);
void ddst(int n, int isgn, float *a, int *ip, float *w);
/// 2D FFT ////////////////////////////////////////////////////////////////
void ddct2d(int n1, int n2, int isgn, float **a, float *t, int *ip, float *w);
void ddsct2d(int n1, int n2, int isgn, float **a, float *t, int *ip, float *w);
void ddcst2d(int n1, int n2, int isgn, float **a, float *t, int *ip, float *w);
NEXTPNR_NAMESPACE_END
#endif

1310
3rdparty/oourafft/fftsg2d.cc vendored Normal file

File diff suppressed because it is too large Load Diff

167
3rdparty/oourafft/readme.txt vendored Normal file
View File

@ -0,0 +1,167 @@
General Purpose FFT (Fast Fourier/Cosine/Sine Transform) Package
Description:
A package to calculate Discrete Fourier/Cosine/Sine Transforms of
1-dimensional sequences of length 2^N.
Files:
fft4g.c : FFT Package in C - Fast Version I (radix 4,2)
fft4g.f : FFT Package in Fortran - Fast Version I (radix 4,2)
fft4g_h.c : FFT Package in C - Simple Version I (radix 4,2)
fft8g.c : FFT Package in C - Fast Version II (radix 8,4,2)
fft8g.f : FFT Package in Fortran - Fast Version II (radix 8,4,2)
fft8g_h.c : FFT Package in C - Simple Version II (radix 8,4,2)
fftsg.c : FFT Package in C - Fast Version III (Split-Radix)
fftsg.f : FFT Package in Fortran - Fast Version III (Split-Radix)
fftsg_h.c : FFT Package in C - Simple Version III (Split-Radix)
readme.txt : Readme File
sample1/ : Test Directory
Makefile : for gcc, cc
Makefile.f77: for Fortran
testxg.c : Test Program for "fft*g.c"
testxg.f : Test Program for "fft*g.f"
testxg_h.c : Test Program for "fft*g_h.c"
sample2/ : Benchmark Directory
Makefile : for gcc, cc
Makefile.pth: POSIX Thread version
pi_fft.c : PI(= 3.1415926535897932384626...) Calculation Program
for a Benchmark Test for "fft*g.c"
Difference of the Files:
C and Fortran versions are equal and
the same routines are in each version.
"fft4g*.*" are optimized for most machines.
"fft8g*.*" are fast on the UltraSPARC.
"fftsg*.*" are optimized for the machines that
have the multi-level (L1,L2,etc) cache.
The simple versions "fft*g_h.c" use no work area, but
the fast versions "fft*g.*" use work areas.
The fast versions "fft*g.*" have the same specification.
Routines in the Package:
cdft: Complex Discrete Fourier Transform
rdft: Real Discrete Fourier Transform
ddct: Discrete Cosine Transform
ddst: Discrete Sine Transform
dfct: Cosine Transform of RDFT (Real Symmetric DFT)
dfst: Sine Transform of RDFT (Real Anti-symmetric DFT)
Usage:
Please refer to the comments in the "fft**.*" file which
you want to use. Brief explanations are in the block
comments of each package. The examples are also given in
the test programs.
Method:
-------- cdft --------
fft4g*.*, fft8g*.*:
A method of in-place, radix 2^M, Sande-Tukey (decimation in
frequency). Index of the butterfly loop is in bit
reverse order to keep continuous memory access.
fftsg*.*:
A method of in-place, Split-Radix, recursive fast
algorithm.
-------- rdft --------
A method with a following butterfly operation appended to "cdft".
In forward transform :
A[k] = sum_j=0^n-1 a[j]*W(n)^(j*k), 0<=k<=n/2,
W(n) = exp(2*pi*i/n),
this routine makes an array x[] :
x[j] = a[2*j] + i*a[2*j+1], 0<=j<n/2
and calls "cdft" of length n/2 :
X[k] = sum_j=0^n/2-1 x[j] * W(n/2)^(j*k), 0<=k<n.
The result A[k] are :
A[k] = X[k] - (1+i*W(n)^k)/2 * (X[k]-conjg(X[n/2-k])),
A[n/2-k] = X[n/2-k] +
conjg((1+i*W(n)^k)/2 * (X[k]-conjg(X[n/2-k]))),
0<=k<=n/2
(notes: conjg() is a complex conjugate, X[n/2]=X[0]).
-------- ddct --------
A method with a following butterfly operation appended to "rdft".
In backward transform :
C[k] = sum_j=0^n-1 a[j]*cos(pi*j*(k+1/2)/n), 0<=k<n,
this routine makes an array r[] :
r[0] = a[0],
r[j] = Re((a[j] - i*a[n-j]) * W(4*n)^j*(1+i)/2),
r[n-j] = Im((a[j] - i*a[n-j]) * W(4*n)^j*(1+i)/2),
0<j<=n/2
and calls "rdft" of length n :
A[k] = sum_j=0^n-1 r[j]*W(n)^(j*k), 0<=k<=n/2,
W(n) = exp(2*pi*i/n).
The result C[k] are :
C[2*k] = Re(A[k] * (1-i)),
C[2*k-1] = -Im(A[k] * (1-i)).
-------- ddst --------
A method with a following butterfly operation appended to "rdft".
In backward transform :
S[k] = sum_j=1^n A[j]*sin(pi*j*(k+1/2)/n), 0<=k<n,
this routine makes an array r[] :
r[0] = a[0],
r[j] = Im((a[n-j] - i*a[j]) * W(4*n)^j*(1+i)/2),
r[n-j] = Re((a[n-j] - i*a[j]) * W(4*n)^j*(1+i)/2),
0<j<=n/2
and calls "rdft" of length n :
A[k] = sum_j=0^n-1 r[j]*W(n)^(j*k), 0<=k<=n/2,
W(n) = exp(2*pi*i/n).
The result S[k] are :
S[2*k] = Re(A[k] * (1+i)),
S[2*k-1] = -Im(A[k] * (1+i)).
-------- dfct --------
A method to split into "dfct" and "ddct" of half length.
The transform :
C[k] = sum_j=0^n a[j]*cos(pi*j*k/n), 0<=k<=n
is divided into :
C[2*k] = sum'_j=0^n/2 (a[j]+a[n-j])*cos(pi*j*k/(n/2)),
C[2*k+1] = sum_j=0^n/2-1 (a[j]-a[n-j])*cos(pi*j*(k+1/2)/(n/2))
(sum' is a summation whose last term multiplies 1/2).
This routine uses "ddct" recursively.
To keep the in-place operation, the data in fft*g_h.*
are sorted in bit reversal order.
-------- dfst --------
A method to split into "dfst" and "ddst" of half length.
The transform :
S[k] = sum_j=1^n-1 a[j]*sin(pi*j*k/n), 0<k<n
is divided into :
S[2*k] = sum_j=1^n/2-1 (a[j]-a[n-j])*sin(pi*j*k/(n/2)),
S[2*k+1] = sum'_j=1^n/2 (a[j]+a[n-j])*sin(pi*j*(k+1/2)/(n/2))
(sum' is a summation whose last term multiplies 1/2).
This routine uses "ddst" recursively.
To keep the in-place operation, the data in fft*g_h.*
are sorted in bit reversal order.
Reference:
* Masatake MORI, Makoto NATORI, Tatuo TORII: Suchikeisan,
Iwanamikouzajyouhoukagaku18, Iwanami, 1982 (Japanese)
* Henri J. Nussbaumer: Fast Fourier Transform and Convolution
Algorithms, Springer Verlag, 1982
* C. S. Burrus, Notes on the FFT (with large FFT paper list)
http://www-dsp.rice.edu/research/fft/fftnote.asc
Copyright:
Copyright(C) 1996-2001 Takuya OOURA
email: ooura@mmm.t.u-tokyo.ac.jp
download: http://momonga.t.u-tokyo.ac.jp/~ooura/fft.html
You may use, copy, modify this code for any purpose and
without fee. You may distribute this ORIGINAL package.
History:
...
Dec. 1995 : Edit the General Purpose FFT
Mar. 1996 : Change the specification
Jun. 1996 : Change the method of trigonometric function table
Sep. 1996 : Modify the documents
Feb. 1997 : Change the butterfly loops
Dec. 1997 : Modify the documents
Dec. 1997 : Add "fft4g.*"
Jul. 1998 : Fix some bugs in the documents
Jul. 1998 : Add "fft8g.*" and delete "fft4f.*"
Jul. 1998 : Add a benchmark program "pi_fft.c"
Jul. 1999 : Add a simple version "fft*g_h.c"
Jul. 1999 : Add a Split-Radix FFT package "fftsg*.c"
Sep. 1999 : Reduce the memory operation (minor optimization)
Oct. 1999 : Change the butterfly structure of "fftsg*.c"
Oct. 1999 : Save the code size
Sep. 2001 : Add "fftsg.f"
Sep. 2001 : Add Pthread & Win32thread routines to "fftsg*.c"
Dec. 2006 : Fix a minor bug in "fftsg.f"

77
3rdparty/oourafft/readme2d.txt vendored Normal file
View File

@ -0,0 +1,77 @@
General Purpose 2D,3D FFT (Fast Fourier Transform) Package
Files
alloc.c : 2D-array Allocation
alloc.h : 2D-array Allocation
fft4f2d.c : 2D FFT Package in C - Version I (radix 4, 2)
fft4f2d.f : 2D FFT Package in Fortran - Version I (radix 4, 2)
fftsg.c : 1D FFT Package in C - Fast Version (Split-Radix)
fftsg.f : 1D FFT Package in Fortran - Fast Version (Split-Radix)
fftsg2d.c : 2D FFT Package in C - Version II (Split-Radix)
fftsg2d.f : 2D FFT Package in Fortran - Version II (Split-Radix)
fftsg3d.c : 3D FFT Package in C - Version II (Split-Radix)
fftsg3d.f : 3D FFT Package in Fortran - Version II (Split-Radix)
shrtdct.c : 8x8, 16x16 DCT Package
sample2d/
Makefile : for gcc, cc
Makefile.f77: for Fortran
Makefile.pth: Pthread version
fft4f2dt.c : Test Program for "fft4f2d.c"
fft4f2dt.f : Test Program for "fft4f2d.f"
fftsg2dt.c : Test Program for "fftsg2d.c"
fftsg2dt.f : Test Program for "fftsg2d.f"
fftsg3dt.c : Test Program for "fftsg3d.c"
fftsg3dt.f : Test Program for "fftsg3d.f"
shrtdctt.c : Test Program for "shrtdct.c"
Difference of Files
C and Fortran versions are equal and
the same routines are in each version.
---- Difference between "fft4f2d.*" and "fftsg2d.*" ----
"fft4f2d.*" are optimized for the old machines that
don't have the large size CPU cache.
"fftsg2d.*", "fftsg3d.*" use 1D FFT routines in "fftsg.*".
"fftsg2d.*", "fftsg3d.*" are optimized for the machines that
have the multi-level (L1,L2,etc) cache.
Routines in the Package
in fft4f2d.*, fftsg2d.*
cdft2d: 2-dim Complex Discrete Fourier Transform
rdft2d: 2-dim Real Discrete Fourier Transform
ddct2d: 2-dim Discrete Cosine Transform
ddst2d: 2-dim Discrete Sine Transform
rdft2dsort: rdft2d input/output ordering (fftsg2d.*)
in fftsg3d.*
cdft3d: 3-dim Complex Discrete Fourier Transform
rdft3d: 3-dim Real Discrete Fourier Transform
ddct3d: 3-dim Discrete Cosine Transform
ddst3d: 3-dim Discrete Sine Transform
rdft3dsort: rdft3d input/output ordering
in fftsg.*
cdft: 1-dim Complex Discrete Fourier Transform
rdft: 1-dim Real Discrete Fourier Transform
ddct: 1-dim Discrete Cosine Transform
ddst: 1-dim Discrete Sine Transform
dfct: 1-dim Real Symmetric DFT
dfst: 1-dim Real Anti-symmetric DFT
(these routines are called by fftsg2d.*, fftsg3d.*)
in shrtdct.c
ddct8x8s : Normalized 8x8 DCT
ddct16x16s: Normalized 16x16 DCT
(faster than ddct2d())
Usage
Brief explanations are in block comments of each packages.
The examples are given in the test programs.
Copyright
Copyright(C) 1997,2001 Takuya OOURA (email: ooura@kurims.kyoto-u.ac.jp).
You may use, copy, modify this code for any purpose and
without fee. You may distribute this ORIGINAL package.
History
...
Nov. 2001 : Add 3D-FFT routines
Dec. 2006 : Fix a documentation bug in "fftsg3d.*"
Dec. 2006 : Fix a minor bug in "fftsg.f"

View File

@ -1,6 +1,6 @@
version: 1.0.{build} version: 1.0.{build}
image: image:
- Visual Studio 2015 - Visual Studio 2017
test: off test: off
skip_branch_with_pr: true skip_branch_with_pr: true
build: build:
@ -11,15 +11,13 @@ environment:
matrix: matrix:
- PYTHON: 36 - PYTHON: 36
CONFIG: Debug CONFIG: Debug
- PYTHON: 27
CONFIG: Debug
install: install:
- ps: | - ps: |
$env:CMAKE_GENERATOR = "Visual Studio 14 2015" $env:CMAKE_GENERATOR = "Visual Studio 15 2017"
if ($env:PLATFORM -eq "x64") { $env:PYTHON = "$env:PYTHON-x64" } if ($env:PLATFORM -eq "x64") { $env:PYTHON = "$env:PYTHON-x64" }
$env:PATH = "C:\Python$env:PYTHON\;C:\Python$env:PYTHON\Scripts\;$env:PATH" $env:PATH = "C:\Python$env:PYTHON\;C:\Python$env:PYTHON\Scripts\;$env:PATH"
python -W ignore -m pip install --upgrade pip wheel python -W ignore -m pip install --upgrade pip wheel
python -W ignore -m pip install pytest numpy --no-warn-script-location python -W ignore -m pip install pytest numpy --no-warn-script-location pytest-timeout
- ps: | - ps: |
Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip' Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip'
7z x eigen-3.3.7.zip -y > $null 7z x eigen-3.3.7.zip -y > $null

38
3rdparty/pybind11/.clang-format vendored Normal file
View File

@ -0,0 +1,38 @@
---
# See all possible options and defaults with:
# clang-format --style=llvm --dump-config
BasedOnStyle: LLVM
AccessModifierOffset: -4
AllowShortLambdasOnASingleLine: true
AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: false
BinPackParameters: false
BreakBeforeBinaryOperators: All
BreakConstructorInitializers: BeforeColon
ColumnLimit: 99
CommentPragmas: 'NOLINT:.*|^ IWYU pragma:'
IncludeBlocks: Regroup
IndentCaseLabels: true
IndentPPDirectives: AfterHash
IndentWidth: 4
Language: Cpp
SpaceAfterCStyleCast: true
Standard: Cpp11
StatementMacros: ['PyObject_HEAD']
TabWidth: 4
IncludeCategories:
- Regex: '<pybind11/.*'
Priority: -1
- Regex: 'pybind11.h"$'
Priority: 1
- Regex: '^".*/?detail/'
Priority: 1
SortPriority: 2
- Regex: '^"'
Priority: 1
SortPriority: 3
- Regex: '<[[:alnum:]._]+>'
Priority: 4
- Regex: '.*'
Priority: 5
...

View File

@ -1,13 +1,77 @@
FormatStyle: file FormatStyle: file
Checks: ' Checks: |
llvm-namespace-comment, *bugprone*,
modernize-use-override, *performance*,
readability-container-size-empty, clang-analyzer-optin.cplusplus.VirtualCall,
modernize-use-using, clang-analyzer-optin.performance.Padding,
modernize-use-equals-default, cppcoreguidelines-init-variables,
modernize-use-auto, cppcoreguidelines-prefer-member-initializer,
modernize-use-emplace, cppcoreguidelines-pro-type-static-cast-downcast,
' cppcoreguidelines-slicing,
google-explicit-constructor,
llvm-namespace-comment,
misc-definitions-in-headers,
misc-misplaced-const,
misc-non-copyable-objects,
misc-static-assert,
misc-throw-by-value-catch-by-reference,
misc-uniqueptr-reset-release,
misc-unused-parameters,
modernize-avoid-bind,
modernize-loop-convert,
modernize-make-shared,
modernize-redundant-void-arg,
modernize-replace-auto-ptr,
modernize-replace-disallow-copy-and-assign-macro,
modernize-replace-random-shuffle,
modernize-shrink-to-fit,
modernize-use-auto,
modernize-use-bool-literals,
modernize-use-default-member-init,
modernize-use-emplace,
modernize-use-equals-default,
modernize-use-equals-delete,
modernize-use-noexcept,
modernize-use-nullptr,
modernize-use-override,
modernize-use-using,
readability-avoid-const-params-in-decls,
readability-braces-around-statements,
readability-const-return-type,
readability-container-size-empty,
readability-delete-null-pointer,
readability-else-after-return,
readability-implicit-bool-conversion,
readability-inconsistent-declaration-parameter-name,
readability-make-member-function-const,
readability-misplaced-array-index,
readability-non-const-parameter,
readability-qualified-auto,
readability-redundant-function-ptr-dereference,
readability-redundant-smartptr-get,
readability-redundant-string-cstr,
readability-simplify-subscript-expr,
readability-static-accessed-through-instance,
readability-static-definition-in-anonymous-namespace,
readability-string-compare,
readability-suspicious-call-argument,
readability-uniqueptr-delete-release,
-bugprone-easily-swappable-parameters,
-bugprone-exception-escape,
-bugprone-reserved-identifier,
-bugprone-unused-raii,
CheckOptions:
- key: modernize-use-equals-default.IgnoreMacros
value: false
- key: performance-for-range-copy.WarnOnAllAutoCopies
value: true
- key: performance-inefficient-string-concatenation.StrictMode
value: true
- key: performance-unnecessary-value-param.AllowedTypes
value: 'exception_ptr$;'
- key: readability-implicit-bool-conversion.AllowPointerConditions
value: true
HeaderFilterRegex: 'pybind11/.*h' HeaderFilterRegex: 'pybind11/.*h'

View File

@ -0,0 +1,24 @@
template <op_id id, op_type ot, typename L = undefined_t, typename R = undefined_t>
template <typename ThisT>
auto &this_ = static_cast<ThisT &>(*this);
if (load_impl<ThisT>(temp, false)) {
ssize_t nd = 0;
auto trivial = broadcast(buffers, nd, shape);
auto ndim = (size_t) nd;
int nd;
ssize_t ndim() const { return detail::array_proxy(m_ptr)->nd; }
using op = op_impl<id, ot, Base, L_type, R_type>;
template <op_id id, op_type ot, typename L, typename R>
template <detail::op_id id, detail::op_type ot, typename L, typename R, typename... Extra>
class_ &def(const detail::op_<id, ot, L, R> &op, const Extra &...extra) {
class_ &def_cast(const detail::op_<id, ot, L, R> &op, const Extra &...extra) {
@pytest.mark.parametrize("access", ["ro", "rw", "static_ro", "static_rw"])
struct IntStruct {
explicit IntStruct(int v) : value(v){};
~IntStruct() { value = -value; }
IntStruct(const IntStruct &) = default;
IntStruct &operator=(const IntStruct &) = default;
py::class_<IntStruct>(m, "IntStruct").def(py::init([](const int i) { return IntStruct(i); }));
py::implicitly_convertible<int, IntStruct>();
m.def("test", [](int expected, const IntStruct &in) {
[](int expected, const IntStruct &in) {

1
3rdparty/pybind11/.gitattributes vendored Normal file
View File

@ -0,0 +1 @@
docs/*.svg binary

9
3rdparty/pybind11/.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1,9 @@
*.cmake @henryiii
CMakeLists.txt @henryiii
*.yml @henryiii
*.yaml @henryiii
/tools/ @henryiii
/pybind11/ @henryiii
noxfile.py @henryiii
.clang-format @henryiii
.clang-tidy @henryiii

View File

@ -53,6 +53,33 @@ derivative works thereof, in binary and source code form.
## Development of pybind11 ## Development of pybind11
### Quick setup
To setup a quick development environment, use [`nox`](https://nox.thea.codes).
This will allow you to do some common tasks with minimal setup effort, but will
take more time to run and be less flexible than a full development environment.
If you use [`pipx run nox`](https://pipx.pypa.io), you don't even need to
install `nox`. Examples:
```bash
# List all available sessions
nox -l
# Run linters
nox -s lint
# Run tests on Python 3.9
nox -s tests-3.9
# Build and preview docs
nox -s docs -- serve
# Build SDists and wheels
nox -s build
```
### Full setup
To setup an ideal development environment, run the following commands on a To setup an ideal development environment, run the following commands on a
system with CMake 3.14+: system with CMake 3.14+:
@ -66,11 +93,10 @@ cmake --build build -j4
Tips: Tips:
* You can use `virtualenv` (from PyPI) instead of `venv` (which is Python 3 * You can use `virtualenv` (faster, from PyPI) instead of `venv`.
only).
* You can select any name for your environment folder; if it contains "env" it * You can select any name for your environment folder; if it contains "env" it
will be ignored by git. will be ignored by git.
* If you dont have CMake 3.14+, just add “cmake” to the pip install command. * If you don't have CMake 3.14+, just add "cmake" to the pip install command.
* You can use `-DPYBIND11_FINDPYTHON=ON` to use FindPython on CMake 3.12+ * You can use `-DPYBIND11_FINDPYTHON=ON` to use FindPython on CMake 3.12+
* In classic mode, you may need to set `-DPYTHON_EXECUTABLE=/path/to/python`. * In classic mode, you may need to set `-DPYTHON_EXECUTABLE=/path/to/python`.
FindPython uses `-DPython_ROOT_DIR=/path/to` or FindPython uses `-DPython_ROOT_DIR=/path/to` or
@ -78,7 +104,7 @@ Tips:
### Configuration options ### Configuration options
In CMake, configuration options are given with “-D”. Options are stored in the In CMake, configuration options are given with "-D". Options are stored in the
build directory, in the `CMakeCache.txt` file, so they are remembered for each build directory, in the `CMakeCache.txt` file, so they are remembered for each
build directory. Two selections are special - the generator, given with `-G`, build directory. Two selections are special - the generator, given with `-G`,
and the compiler, which is selected based on environment variables `CXX` and and the compiler, which is selected based on environment variables `CXX` and
@ -88,12 +114,12 @@ after the initial run.
The valid options are: The valid options are:
* `-DCMAKE_BUILD_TYPE`: Release, Debug, MinSizeRel, RelWithDebInfo * `-DCMAKE_BUILD_TYPE`: Release, Debug, MinSizeRel, RelWithDebInfo
* `-DPYBIND11_FINDPYTHON=ON`: Use CMake 3.12+s FindPython instead of the * `-DPYBIND11_FINDPYTHON=ON`: Use CMake 3.12+'s FindPython instead of the
classic, deprecated, custom FindPythonLibs classic, deprecated, custom FindPythonLibs
* `-DPYBIND11_NOPYTHON=ON`: Disable all Python searching (disables tests) * `-DPYBIND11_NOPYTHON=ON`: Disable all Python searching (disables tests)
* `-DBUILD_TESTING=ON`: Enable the tests * `-DBUILD_TESTING=ON`: Enable the tests
* `-DDOWNLOAD_CATCH=ON`: Download catch to build the C++ tests * `-DDOWNLOAD_CATCH=ON`: Download catch to build the C++ tests
* `-DOWNLOAD_EIGEN=ON`: Download Eigen for the NumPy tests * `-DDOWNLOAD_EIGEN=ON`: Download Eigen for the NumPy tests
* `-DPYBIND11_INSTALL=ON/OFF`: Enable the install target (on by default for the * `-DPYBIND11_INSTALL=ON/OFF`: Enable the install target (on by default for the
master project) master project)
* `-DUSE_PYTHON_INSTALL_DIR=ON`: Try to install into the python dir * `-DUSE_PYTHON_INSTALL_DIR=ON`: Try to install into the python dir
@ -109,7 +135,7 @@ The valid options are:
* Use `-G` and the name of a generator to use something different. `cmake * Use `-G` and the name of a generator to use something different. `cmake
--help` lists the generators available. --help` lists the generators available.
- On Unix, setting `CMAKE_GENERATER=Ninja` in your environment will give - On Unix, setting `CMAKE_GENERATER=Ninja` in your environment will give
you automatic mulithreading on all your CMake projects! you automatic multithreading on all your CMake projects!
* Open the `CMakeLists.txt` with QtCreator to generate for that IDE. * Open the `CMakeLists.txt` with QtCreator to generate for that IDE.
* You can use `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` to generate the `.json` file * You can use `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` to generate the `.json` file
that some tools expect. that some tools expect.
@ -126,13 +152,26 @@ cmake --build build --target check
`--target` can be spelled `-t` in CMake 3.15+. You can also run individual `--target` can be spelled `-t` in CMake 3.15+. You can also run individual
tests with these targets: tests with these targets:
* `pytest`: Python tests only * `pytest`: Python tests only, using the
[pytest](https://docs.pytest.org/en/stable/) framework
* `cpptest`: C++ tests only * `cpptest`: C++ tests only
* `test_cmake_build`: Install / subdirectory tests * `test_cmake_build`: Install / subdirectory tests
If you want to build just a subset of tests, use If you want to build just a subset of tests, use
`-DPYBIND11_TEST_OVERRIDE="test_callbacks.cpp;test_pickling.cpp"`. If this is `-DPYBIND11_TEST_OVERRIDE="test_callbacks;test_pickling"`. If this is
empty, all tests will be built. empty, all tests will be built. Tests are specified without an extension if they need both a .py and
.cpp file.
You may also pass flags to the `pytest` target by editing `tests/pytest.ini` or
by using the `PYTEST_ADDOPTS` environment variable
(see [`pytest` docs](https://docs.pytest.org/en/2.7.3/customize.html#adding-default-options)). As an example:
```bash
env PYTEST_ADDOPTS="--capture=no --exitfirst" \
cmake --build build --target pytest
# Or using abbreviated flags
env PYTEST_ADDOPTS="-s -x" cmake --build build --target pytest
```
### Formatting ### Formatting
@ -164,18 +203,46 @@ name, pre-commit):
pre-commit install pre-commit install
``` ```
### Clang-Tidy ### Clang-Format
To run Clang tidy, the following recipe should work. Files will be modified in As of v2.6.2, pybind11 ships with a [`clang-format`][clang-format]
place, so you can use git to monitor the changes. configuration file at the top level of the repo (the filename is
`.clang-format`). Currently, formatting is NOT applied automatically, but
manually using `clang-format` for newly developed files is highly encouraged.
To check if a file needs formatting:
```bash ```bash
docker run --rm -v $PWD:/pybind11 -it silkeh/clang:10 clang-format -style=file --dry-run some.cpp
apt-get update && apt-get install python3-dev python3-pytest
cmake -S pybind11/ -B build -DCMAKE_CXX_CLANG_TIDY="$(which clang-tidy);-fix"
cmake --build build
``` ```
The output will show things to be fixed, if any. To actually format the file:
```bash
clang-format -style=file -i some.cpp
```
Note that the `-style-file` option searches the parent directories for the
`.clang-format` file, i.e. the commands above can be run in any subdirectory
of the pybind11 repo.
### Clang-Tidy
[`clang-tidy`][clang-tidy] performs deeper static code analyses and is
more complex to run, compared to `clang-format`, but support for `clang-tidy`
is built into the pybind11 CMake configuration. To run `clang-tidy`, the
following recipe should work. Run the `docker` command from the top-level
directory inside your pybind11 git clone. Files will be modified in place,
so you can use git to monitor the changes.
```bash
docker run --rm -v $PWD:/mounted_pybind11 -it silkeh/clang:15-bullseye
apt-get update && apt-get install -y git python3-dev python3-pytest
cmake -S /mounted_pybind11/ -B build -DCMAKE_CXX_CLANG_TIDY="$(which clang-tidy);--use-color" -DDOWNLOAD_EIGEN=ON -DDOWNLOAD_CATCH=ON -DCMAKE_CXX_STANDARD=17
cmake --build build -j 2
```
You can add `--fix` to the options list if you want.
### Include what you use ### Include what you use
To run include what you use, install (`brew install include-what-you-use` on To run include what you use, install (`brew install include-what-you-use` on
@ -186,12 +253,12 @@ cmake -S . -B build-iwyu -DCMAKE_CXX_INCLUDE_WHAT_YOU_USE=$(which include-what-y
cmake --build build cmake --build build
``` ```
The report is sent to stderr; you can pip it into a file if you wish. The report is sent to stderr; you can pipe it into a file if you wish.
### Build recipes ### Build recipes
This builds with the Intel compiler (assuming it is in your path, along with a This builds with the Intel compiler (assuming it is in your path, along with a
recent CMake and Python 3): recent CMake and Python):
```bash ```bash
python3 -m venv venv python3 -m venv venv
@ -313,6 +380,8 @@ if you really want to.
[pre-commit]: https://pre-commit.com [pre-commit]: https://pre-commit.com
[clang-format]: https://clang.llvm.org/docs/ClangFormat.html
[clang-tidy]: https://clang.llvm.org/extra/clang-tidy/
[pybind11.readthedocs.org]: http://pybind11.readthedocs.org/en/latest [pybind11.readthedocs.org]: http://pybind11.readthedocs.org/en/latest
[issue tracker]: https://github.com/pybind/pybind11/issues [issue tracker]: https://github.com/pybind/pybind11/issues
[gitter]: https://gitter.im/pybind/Lobby [gitter]: https://gitter.im/pybind/Lobby

View File

@ -1,28 +0,0 @@
---
name: Bug Report
about: File an issue about a bug
title: "[BUG] "
---
Make sure you've completed the following steps before submitting your issue -- thank you!
1. Make sure you've read the [documentation][]. Your issue may be addressed there.
2. Search the [issue tracker][] to verify that this hasn't already been reported. +1 or comment there if it has.
3. Consider asking first in the [Gitter chat room][].
4. Include a self-contained and minimal piece of code that reproduces the problem. If that's not possible, try to make the description as clear as possible.
a. If possible, make a PR with a new, failing test to give us a starting point to work on!
[documentation]: https://pybind11.readthedocs.io
[issue tracker]: https://github.com/pybind/pybind11/issues
[Gitter chat room]: https://gitter.im/pybind/Lobby
*After reading, remove this checklist and the template text in parentheses below.*
## Issue description
(Provide a short description, state the expected behavior and what actually happens.)
## Reproducible example code
(The code should be minimal, have no external dependencies, isolate the function(s) that cause breakage. Submit matched and complete C++ and Python snippets that can be easily compiled and run to diagnose the issue.)

View File

@ -0,0 +1,61 @@
name: Bug Report
description: File an issue about a bug
title: "[BUG]: "
labels: [triage]
body:
- type: markdown
attributes:
value: |
Please do your best to make the issue as easy to act on as possible, and only submit here if there is clearly a problem with pybind11 (ask first if unsure). **Note that a reproducer in a PR is much more likely to get immediate attention.**
- type: checkboxes
id: steps
attributes:
label: Required prerequisites
description: Make sure you've completed the following steps before submitting your issue -- thank you!
options:
- label: Make sure you've read the [documentation](https://pybind11.readthedocs.io). Your issue may be addressed there.
required: true
- label: Search the [issue tracker](https://github.com/pybind/pybind11/issues) and [Discussions](https:/pybind/pybind11/discussions) to verify that this hasn't already been reported. +1 or comment there if it has.
required: true
- label: Consider asking first in the [Gitter chat room](https://gitter.im/pybind/Lobby) or in a [Discussion](https:/pybind/pybind11/discussions/new).
required: false
- type: input
id: version
attributes:
label: What version (or hash if on master) of pybind11 are you using?
validations:
required: true
- type: textarea
id: description
attributes:
label: Problem description
placeholder: >-
Provide a short description, state the expected behavior and what
actually happens. Include relevant information like what version of
pybind11 you are using, what system you are on, and any useful commands
/ output.
validations:
required: true
- type: textarea
id: code
attributes:
label: Reproducible example code
placeholder: >-
The code should be minimal, have no external dependencies, isolate the
function(s) that cause breakage. Submit matched and complete C++ and
Python snippets that can be easily compiled and run to diagnose the
issue. — Note that a reproducer in a PR is much more likely to get
immediate attention: failing tests in the pybind11 CI are the best
starting point for working out fixes.
render: text
- type: input
id: regression
attributes:
label: Is this a regression? Put the last known working version here if it is.
description: Put the last known working version here if this is a regression.
value: Not a regression

View File

@ -1,5 +1,8 @@
blank_issues_enabled: false blank_issues_enabled: false
contact_links: contact_links:
- name: Ask a question
url: https://github.com/pybind/pybind11/discussions/new
about: Please ask and answer questions here, or propose new ideas.
- name: Gitter room - name: Gitter room
url: https://gitter.im/pybind/Lobby url: https://gitter.im/pybind/Lobby
about: A room for discussing pybind11 with an active community about: A room for discussing pybind11 with an active community

View File

@ -1,16 +0,0 @@
---
name: Feature Request
about: File an issue about adding a feature
title: "[FEAT] "
---
Make sure you've completed the following steps before submitting your issue -- thank you!
1. Check if your feature has already been mentioned / rejected / planned in other issues.
2. If those resources didn't help, consider asking in the [Gitter chat room][] to see if this is interesting / useful to a larger audience and possible to implement reasonably,
4. If you have a useful feature that passes the previous items (or not suitable for chat), please fill in the details below.
[Gitter chat room]: https://gitter.im/pybind/Lobby
*After reading, remove this checklist.*

View File

@ -1,21 +0,0 @@
---
name: Question
about: File an issue about unexplained behavior
title: "[QUESTION] "
---
If you have a question, please check the following first:
1. Check if your question has already been answered in the [FAQ][] section.
2. Make sure you've read the [documentation][]. Your issue may be addressed there.
3. If those resources didn't help and you only have a short question (not a bug report), consider asking in the [Gitter chat room][]
4. Search the [issue tracker][], including the closed issues, to see if your question has already been asked/answered. +1 or comment if it has been asked but has no answer.
5. If you have a more complex question which is not answered in the previous items (or not suitable for chat), please fill in the details below.
6. Include a self-contained and minimal piece of code that illustrates your question. If that's not possible, try to make the description as clear as possible.
[FAQ]: http://pybind11.readthedocs.io/en/latest/faq.html
[documentation]: https://pybind11.readthedocs.io
[issue tracker]: https://github.com/pybind/pybind11/issues
[Gitter chat room]: https://gitter.im/pybind/Lobby
*After reading, remove this checklist.*

View File

@ -4,8 +4,12 @@ updates:
- package-ecosystem: "github-actions" - package-ecosystem: "github-actions"
directory: "/" directory: "/"
schedule: schedule:
interval: "daily" interval: "weekly"
groups:
actions:
patterns:
- "*"
ignore: ignore:
# Offical actions have moving tags like v1 - dependency-name: actions/checkout
# that are used, so they don't need updates here versions:
- dependency-name: "actions/*" - "<5"

View File

@ -1,3 +1,7 @@
<!--
Title (above): please place [branch_name] at the beginning if you are targeting a branch other than master. *Do not target stable*.
It is recommended to use conventional commit format, see conventionalcommits.org, but not required.
-->
## Description ## Description
<!-- Include relevant issues or PRs here, describe what changed and why --> <!-- Include relevant issues or PRs here, describe what changed and why -->
@ -5,7 +9,8 @@
## Suggested changelog entry: ## Suggested changelog entry:
<!-- fill in the below block with the expected RestructuredText entry (delete if no entry needed) --> <!-- Fill in the below block with the expected RestructuredText entry. Delete if no entry needed;
but do not delete header or rst block if an entry is needed! Will be collected via a script. -->
```rst ```rst

File diff suppressed because it is too large Load Diff

View File

@ -9,6 +9,14 @@ on:
- stable - stable
- v* - v*
permissions:
contents: read
env:
PIP_BREAK_SYSTEM_PACKAGES: 1
# For cmake:
VERBOSE: 1
jobs: jobs:
# This tests various versions of CMake in various combinations, to make sure # This tests various versions of CMake in various combinations, to make sure
# the configure step passes. # the configure step passes.
@ -16,35 +24,35 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
runs-on: [ubuntu-latest, macos-latest, windows-latest] runs-on: [ubuntu-20.04, macos-latest, windows-latest]
arch: [x64] arch: [x64]
cmake: [3.18] cmake: ["3.26"]
include: include:
- runs-on: ubuntu-latest - runs-on: ubuntu-20.04
arch: x64 arch: x64
cmake: 3.4 cmake: "3.5"
- runs-on: ubuntu-20.04
arch: x64
cmake: "3.27"
- runs-on: macos-latest - runs-on: macos-latest
arch: x64 arch: x64
cmake: 3.7 cmake: "3.7"
- runs-on: windows-2016 - runs-on: windows-2019
arch: x86 arch: x64 # x86 compilers seem to be missing on 2019 image
cmake: 3.8 cmake: "3.18"
- runs-on: windows-2016
arch: x86
cmake: 3.18
name: 🐍 3.7 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }} name: 🐍 3.7 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }}
runs-on: ${{ matrix.runs-on }} runs-on: ${{ matrix.runs-on }}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- name: Setup Python 3.7 - name: Setup Python 3.7
uses: actions/setup-python@v2 uses: actions/setup-python@v5
with: with:
python-version: 3.7 python-version: 3.7
architecture: ${{ matrix.arch }} architecture: ${{ matrix.arch }}
@ -55,7 +63,7 @@ jobs:
# An action for adding a specific version of CMake: # An action for adding a specific version of CMake:
# https://github.com/jwlawson/actions-setup-cmake # https://github.com/jwlawson/actions-setup-cmake
- name: Setup CMake ${{ matrix.cmake }} - name: Setup CMake ${{ matrix.cmake }}
uses: jwlawson/actions-setup-cmake@v1.3 uses: jwlawson/actions-setup-cmake@v2.0
with: with:
cmake-version: ${{ matrix.cmake }} cmake-version: ${{ matrix.cmake }}

View File

@ -12,35 +12,49 @@ on:
- stable - stable
- "v*" - "v*"
permissions:
contents: read
env:
FORCE_COLOR: 3
# For cmake:
VERBOSE: 1
jobs: jobs:
pre-commit: pre-commit:
name: Format name: Format
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- uses: actions/setup-python@v2 - uses: actions/setup-python@v5
- uses: pre-commit/action@v2.0.0 with:
python-version: "3.x"
- name: Add matchers
run: echo "::add-matcher::$GITHUB_WORKSPACE/.github/matchers/pylint.json"
- uses: pre-commit/action@v3.0.1
with: with:
# Slow hooks are marked with manual - slow is okay here, run them too # Slow hooks are marked with manual - slow is okay here, run them too
extra_args: --hook-stage manual --all-files extra_args: --hook-stage manual --all-files
clang-tidy: clang-tidy:
# When making changes here, please also review the "Clang-Tidy" section
# in .github/CONTRIBUTING.md and update as needed.
name: Clang-Tidy name: Clang-Tidy
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: silkeh/clang:10 container: silkeh/clang:15-bullseye
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- name: Install requirements - name: Install requirements
run: apt-get update && apt-get install -y python3-dev python3-pytest run: apt-get update && apt-get install -y git python3-dev python3-pytest
- name: Configure - name: Configure
run: > run: >
cmake -S . -B build cmake -S . -B build
-DCMAKE_CXX_CLANG_TIDY="$(which clang-tidy);--warnings-as-errors=*" -DCMAKE_CXX_CLANG_TIDY="$(which clang-tidy);--use-color;--warnings-as-errors=*"
-DDOWNLOAD_EIGEN=ON -DDOWNLOAD_EIGEN=ON
-DDOWNLOAD_CATCH=ON -DDOWNLOAD_CATCH=ON
-DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD=17
- name: Build - name: Build
run: cmake --build build -j 2 run: cmake --build build -j 2 -- --keep-going

View File

@ -3,14 +3,23 @@ on:
pull_request_target: pull_request_target:
types: [closed] types: [closed]
permissions: {}
jobs: jobs:
label: label:
name: Labeler name: Labeler
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps: steps:
- uses: actions/labeler@main - uses: actions/labeler@v4
if: github.event.pull_request.merged == true if: >
github.event.pull_request.merged == true &&
!startsWith(github.event.pull_request.title, 'chore(deps):') &&
!startsWith(github.event.pull_request.title, 'ci(fix):') &&
!startsWith(github.event.pull_request.title, 'docs(changelog):')
with: with:
repo-token: ${{ secrets.GITHUB_TOKEN }} repo-token: ${{ secrets.GITHUB_TOKEN }}
configuration-path: .github/labeler_merged.yml configuration-path: .github/labeler_merged.yml

View File

@ -12,24 +12,32 @@ on:
types: types:
- published - published
permissions:
contents: read
env:
PIP_BREAK_SYSTEM_PACKAGES: 1
PIP_ONLY_BINARY: numpy
jobs: jobs:
# This builds the sdists and wheels and makes sure the files are exactly as # This builds the sdists and wheels and makes sure the files are exactly as
# expected. Using Windows and Python 2.7, since that is often the most # expected. Using Windows and Python 3.6, since that is often the most
# challenging matrix element. # challenging matrix element.
test-packaging: test-packaging:
name: 🐍 2.7 • 📦 tests • windows-latest name: 🐍 3.6 • 📦 tests • windows-latest
runs-on: windows-latest runs-on: windows-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- name: Setup 🐍 2.7 - name: Setup 🐍 3.6
uses: actions/setup-python@v2 uses: actions/setup-python@v5
with: with:
python-version: 2.7 python-version: 3.6
- name: Prepare env - name: Prepare env
run: python -m pip install -r tests/requirements.txt --prefer-binary run: |
python -m pip install -r tests/requirements.txt
- name: Python Packaging tests - name: Python Packaging tests
run: pytest tests/extra_python_package/ run: pytest tests/extra_python_package/
@ -42,15 +50,16 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- name: Setup 🐍 3.8 - name: Setup 🐍 3.8
uses: actions/setup-python@v2 uses: actions/setup-python@v5
with: with:
python-version: 3.8 python-version: 3.8
- name: Prepare env - name: Prepare env
run: python -m pip install -r tests/requirements.txt build twine --prefer-binary run: |
python -m pip install -r tests/requirements.txt build twine
- name: Python Packaging tests - name: Python Packaging tests
run: pytest tests/extra_python_package/ run: pytest tests/extra_python_package/
@ -64,13 +73,13 @@ jobs:
run: twine check dist/* run: twine check dist/*
- name: Save standard package - name: Save standard package
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v4
with: with:
name: standard name: standard
path: dist/pybind11-* path: dist/pybind11-*
- name: Save global package - name: Save global package
uses: actions/upload-artifact@v2 uses: actions/upload-artifact@v4
with: with:
name: global name: global
path: dist/pybind11_global-* path: dist/pybind11_global-*
@ -85,19 +94,21 @@ jobs:
needs: [packaging] needs: [packaging]
steps: steps:
- uses: actions/setup-python@v2 - uses: actions/setup-python@v5
with:
python-version: "3.x"
# Downloads all to directories matching the artifact names # Downloads all to directories matching the artifact names
- uses: actions/download-artifact@v2 - uses: actions/download-artifact@v4
- name: Publish standard package - name: Publish standard package
uses: pypa/gh-action-pypi-publish@v1.4.1 uses: pypa/gh-action-pypi-publish@release/v1
with: with:
password: ${{ secrets.pypi_password }} password: ${{ secrets.pypi_password }}
packages_dir: standard/ packages-dir: standard/
- name: Publish global package - name: Publish global package
uses: pypa/gh-action-pypi-publish@v1.4.1 uses: pypa/gh-action-pypi-publish@release/v1
with: with:
password: ${{ secrets.pypi_password_global }} password: ${{ secrets.pypi_password_global }}
packages_dir: global/ packages-dir: global/

View File

@ -0,0 +1,116 @@
name: Upstream
on:
workflow_dispatch:
pull_request:
permissions:
contents: read
concurrency:
group: upstream-${{ github.ref }}
cancel-in-progress: true
env:
PIP_BREAK_SYSTEM_PACKAGES: 1
# For cmake:
VERBOSE: 1
jobs:
standard:
name: "🐍 3.13 latest • ubuntu-latest • x64"
runs-on: ubuntu-latest
# Only runs when the 'python dev' label is selected
if: "contains(github.event.pull_request.labels.*.name, 'python dev')"
steps:
- uses: actions/checkout@v4
- name: Setup Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
allow-prereleases: true
- name: Setup Boost
run: sudo apt-get install libboost-dev
- name: Update CMake
uses: jwlawson/actions-setup-cmake@v2.0
- name: Run pip installs
run: |
python -m pip install --upgrade pip
python -m pip install -r tests/requirements.txt
- name: Show platform info
run: |
python -m platform
cmake --version
pip list
# First build - C++11 mode and inplace
- name: Configure C++11
run: >
cmake -S . -B build11
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=11
-DCMAKE_BUILD_TYPE=Debug
- name: Build C++11
run: cmake --build build11 -j 2
- name: Python tests C++11
run: cmake --build build11 --target pytest -j 2
- name: C++11 tests
run: cmake --build build11 --target cpptest -j 2
- name: Interface test C++11
run: cmake --build build11 --target test_cmake_build
# Second build - C++17 mode and in a build directory
- name: Configure C++17
run: >
cmake -S . -B build17
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=17
- name: Build C++17
run: cmake --build build17 -j 2
- name: Python tests C++17
run: cmake --build build17 --target pytest
- name: C++17 tests
run: cmake --build build17 --target cpptest
# Third build - C++17 mode with unstable ABI
- name: Configure (unstable ABI)
run: >
cmake -S . -B build17max
-DPYBIND11_WERROR=ON
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-DCMAKE_CXX_STANDARD=17
-DPYBIND11_INTERNALS_VERSION=10000000
- name: Build (unstable ABI)
run: cmake --build build17max -j 2
- name: Python tests (unstable ABI)
run: cmake --build build17max --target pytest
- name: Interface test (unstable ABI)
run: cmake --build build17max --target test_cmake_build
# This makes sure the setup_helpers module can build packages using
# setuptools
- name: Setuptools helpers test
run: |
pip install setuptools
pytest tests/extra_setuptools

View File

@ -41,3 +41,6 @@ pybind11Targets.cmake
/.vscode /.vscode
/pybind11/include/* /pybind11/include/*
/pybind11/share/* /pybind11/share/*
/docs/_build/*
.ipynb_checkpoints/
tests/main.cpp

View File

@ -12,89 +12,144 @@
# #
# See https://github.com/pre-commit/pre-commit # See https://github.com/pre-commit/pre-commit
ci:
autoupdate_commit_msg: "chore(deps): update pre-commit hooks"
autofix_commit_msg: "style: pre-commit fixes"
autoupdate_schedule: monthly
# third-party content
exclude: ^tools/JoinPaths.cmake$
repos: repos:
# Standard hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-merge-conflict
- id: check-symlinks
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
- id: mixed-line-ending
- id: requirements-txt-fixer
- id: trailing-whitespace
- id: fix-encoding-pragma
# Black, the code formatter, natively supports pre-commit # Clang format the codebase automatically
- repo: https://github.com/psf/black - repo: https://github.com/pre-commit/mirrors-clang-format
rev: 20.8b1 rev: "v17.0.6"
hooks: hooks:
- id: black - id: clang-format
# By default, this ignores pyi files, though black supports them types_or: [c++, c, cuda]
types: [text]
files: \.pyi?$
# Changes tabs to spaces # Ruff, the Python auto-correcting linter/formatter written in Rust
- repo: https://github.com/Lucas-C/pre-commit-hooks - repo: https://github.com/astral-sh/ruff-pre-commit
rev: v1.1.9 rev: v0.2.0
hooks: hooks:
- id: remove-tabs - id: ruff
args: ["--fix", "--show-fixes"]
- id: ruff-format
# Flake8 also supports pre-commit natively (same author) # Check static types with mypy
- repo: https://gitlab.com/pycqa/flake8 - repo: https://github.com/pre-commit/mirrors-mypy
rev: 3.8.3 rev: "v1.8.0"
hooks: hooks:
- id: flake8 - id: mypy
additional_dependencies: [flake8-bugbear, pep8-naming] args: []
exclude: ^(docs/.*|tools/.*)$ exclude: ^(tests|docs)/
additional_dependencies:
- markdown-it-py<3 # Drop this together with dropping Python 3.7 support.
- nox
- rich
- types-setuptools
# CMake formatting # CMake formatting
- repo: https://github.com/cheshirekow/cmake-format-precommit - repo: https://github.com/cheshirekow/cmake-format-precommit
rev: v0.6.13 rev: "v0.6.13"
hooks: hooks:
- id: cmake-format - id: cmake-format
additional_dependencies: [pyyaml] additional_dependencies: [pyyaml]
types: [file] types: [file]
files: (\.cmake|CMakeLists.txt)(.in)?$ files: (\.cmake|CMakeLists.txt)(.in)?$
# Check static types with mypy # Standard hooks
- repo: https://github.com/pre-commit/mirrors-mypy - repo: https://github.com/pre-commit/pre-commit-hooks
rev: v0.790 rev: "v4.5.0"
hooks: hooks:
- id: mypy - id: check-added-large-files
# The default Python type ignores .pyi files, so let's rerun if detected - id: check-case-conflict
types: [text] - id: check-docstring-first
files: ^pybind11.*\.pyi?$ - id: check-merge-conflict
# Running per-file misbehaves a bit, so just run on all files, it's fast - id: check-symlinks
pass_filenames: false - id: check-toml
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
- id: mixed-line-ending
- id: requirements-txt-fixer
- id: trailing-whitespace
# Also code format the docs
- repo: https://github.com/asottile/blacken-docs
rev: "1.16.0"
hooks:
- id: blacken-docs
additional_dependencies:
- black==23.*
# Changes tabs to spaces
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: "v1.5.4"
hooks:
- id: remove-tabs
# Avoid directional quotes
- repo: https://github.com/sirosen/texthooks
rev: "0.6.4"
hooks:
- id: fix-ligatures
- id: fix-smartquotes
# Checking for common mistakes
- repo: https://github.com/pre-commit/pygrep-hooks
rev: "v1.10.0"
hooks:
- id: rst-backticks
- id: rst-directive-colons
- id: rst-inline-touching-normal
# Checks the manifest for missing files (native support) # Checks the manifest for missing files (native support)
- repo: https://github.com/mgedmin/check-manifest - repo: https://github.com/mgedmin/check-manifest
rev: "0.43" rev: "0.49"
hooks: hooks:
- id: check-manifest - id: check-manifest
# This is a slow hook, so only run this if --hook-stage manual is passed # This is a slow hook, so only run this if --hook-stage manual is passed
stages: [manual] stages: [manual]
additional_dependencies: [cmake, ninja] additional_dependencies: [cmake, ninja]
# The original pybind11 checks for a few C++ style items # Check for spelling
# Use tools/codespell_ignore_lines_from_errors.py
# to rebuild .codespell-ignore-lines
- repo: https://github.com/codespell-project/codespell
rev: "v2.2.6"
hooks:
- id: codespell
exclude: ".supp$"
args: ["-x.codespell-ignore-lines", "-Lccompiler"]
# Check for common shell mistakes
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: "v0.9.0.6"
hooks:
- id: shellcheck
# Disallow some common capitalization mistakes
- repo: local - repo: local
hooks: hooks:
- id: disallow-caps - id: disallow-caps
name: Disallow improper capitalization name: Disallow improper capitalization
language: pygrep language: pygrep
entry: PyBind|Numpy|Cmake|CCache entry: PyBind|\bNumpy\b|Cmake|CCache|PyTest
exclude: .pre-commit-config.yaml exclude: ^\.pre-commit-config.yaml$
- repo: local # PyLint has native support - not always usable, but works for us
- repo: https://github.com/PyCQA/pylint
rev: "v3.0.3"
hooks: hooks:
- id: check-style - id: pylint
name: Classic check-style files: ^pybind11
language: system
types: - repo: https://github.com/python-jsonschema/check-jsonschema
- c++ rev: 0.28.0
entry: ./tools/check-style.sh hooks:
- id: check-readthedocs
- id: check-github-workflows
- id: check-dependabot

View File

@ -1,3 +1,20 @@
# https://blog.readthedocs.com/migrate-configuration-v2/
version: 2
build:
os: ubuntu-22.04
apt_packages:
- librsvg2-bin
tools:
python: "3.11"
sphinx:
configuration: docs/conf.py
python: python:
version: 3 install:
requirements_file: docs/requirements.txt - requirements: docs/requirements.txt
formats:
- pdf

View File

@ -5,15 +5,30 @@
# All rights reserved. Use of this source code is governed by a # All rights reserved. Use of this source code is governed by a
# BSD-style license that can be found in the LICENSE file. # BSD-style license that can be found in the LICENSE file.
cmake_minimum_required(VERSION 3.4) # Propagate this policy (FindPythonInterp removal) so it can be detected later
if(NOT CMAKE_VERSION VERSION_LESS "3.27")
cmake_policy(GET CMP0148 _pybind11_cmp0148)
endif()
# The `cmake_minimum_required(VERSION 3.4...3.18)` syntax does not work with cmake_minimum_required(VERSION 3.5)
# The `cmake_minimum_required(VERSION 3.5...3.27)` syntax does not work with
# some versions of VS that have a patched CMake 3.11. This forces us to emulate # some versions of VS that have a patched CMake 3.11. This forces us to emulate
# the behavior using the following workaround: # the behavior using the following workaround:
if(${CMAKE_VERSION} VERSION_LESS 3.18) if(${CMAKE_VERSION} VERSION_LESS 3.27)
cmake_policy(VERSION ${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION}) cmake_policy(VERSION ${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION})
else() else()
cmake_policy(VERSION 3.18) cmake_policy(VERSION 3.27)
endif()
if(_pybind11_cmp0148)
cmake_policy(SET CMP0148 ${_pybind11_cmp0148})
unset(_pybind11_cmp0148)
endif()
# Avoid infinite recursion if tests include this as a subdirectory
if(DEFINED PYBIND11_MASTER_PROJECT)
return()
endif() endif()
# Extract project version from source # Extract project version from source
@ -73,31 +88,71 @@ if(CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR)
set(CMAKE_CXX_EXTENSIONS OFF) set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_STANDARD_REQUIRED ON)
endif() endif()
set(pybind11_system "")
set_property(GLOBAL PROPERTY USE_FOLDERS ON)
if(CMAKE_VERSION VERSION_LESS "3.18")
set(_pybind11_findpython_default OFF)
else()
set(_pybind11_findpython_default ON)
endif()
else() else()
set(PYBIND11_MASTER_PROJECT OFF) set(PYBIND11_MASTER_PROJECT OFF)
set(pybind11_system SYSTEM) set(pybind11_system SYSTEM)
set(_pybind11_findpython_default OFF)
endif() endif()
# Options # Options
option(PYBIND11_INSTALL "Install pybind11 header files?" ${PYBIND11_MASTER_PROJECT}) option(PYBIND11_INSTALL "Install pybind11 header files?" ${PYBIND11_MASTER_PROJECT})
option(PYBIND11_TEST "Build pybind11 test suite?" ${PYBIND11_MASTER_PROJECT}) option(PYBIND11_TEST "Build pybind11 test suite?" ${PYBIND11_MASTER_PROJECT})
option(PYBIND11_NOPYTHON "Disable search for Python" OFF) option(PYBIND11_NOPYTHON "Disable search for Python" OFF)
option(PYBIND11_DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION
"To enforce that a handle_type_name<> specialization exists" OFF)
option(PYBIND11_SIMPLE_GIL_MANAGEMENT
"Use simpler GIL management logic that does not support disassociation" OFF)
option(PYBIND11_NUMPY_1_ONLY
"Disable NumPy 2 support to avoid changes to previous pybind11 versions." OFF)
set(PYBIND11_INTERNALS_VERSION
""
CACHE STRING "Override the ABI version, may be used to enable the unstable ABI.")
if(PYBIND11_DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION)
add_compile_definitions(PYBIND11_DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION)
endif()
if(PYBIND11_SIMPLE_GIL_MANAGEMENT)
add_compile_definitions(PYBIND11_SIMPLE_GIL_MANAGEMENT)
endif()
if(PYBIND11_NUMPY_1_ONLY)
add_compile_definitions(PYBIND11_NUMPY_1_ONLY)
endif()
cmake_dependent_option( cmake_dependent_option(
USE_PYTHON_INCLUDE_DIR USE_PYTHON_INCLUDE_DIR
"Install pybind11 headers in Python include directory instead of default installation prefix" "Install pybind11 headers in Python include directory instead of default installation prefix"
OFF "PYBIND11_INSTALL" OFF) OFF "PYBIND11_INSTALL" OFF)
cmake_dependent_option(PYBIND11_FINDPYTHON "Force new FindPython" OFF cmake_dependent_option(PYBIND11_FINDPYTHON "Force new FindPython" ${_pybind11_findpython_default}
"NOT CMAKE_VERSION VERSION_LESS 3.12" OFF) "NOT CMAKE_VERSION VERSION_LESS 3.12" OFF)
# Allow PYTHON_EXECUTABLE if in FINDPYTHON mode and building pybind11's tests
# (makes transition easier while we support both modes).
if(PYBIND11_MASTER_PROJECT
AND PYBIND11_FINDPYTHON
AND DEFINED PYTHON_EXECUTABLE
AND NOT DEFINED Python_EXECUTABLE)
set(Python_EXECUTABLE "${PYTHON_EXECUTABLE}")
endif()
# NB: when adding a header don't forget to also add it to setup.py # NB: when adding a header don't forget to also add it to setup.py
set(PYBIND11_HEADERS set(PYBIND11_HEADERS
include/pybind11/detail/class.h include/pybind11/detail/class.h
include/pybind11/detail/common.h include/pybind11/detail/common.h
include/pybind11/detail/cpp_conduit.h
include/pybind11/detail/descr.h include/pybind11/detail/descr.h
include/pybind11/detail/init.h include/pybind11/detail/init.h
include/pybind11/detail/internals.h include/pybind11/detail/internals.h
include/pybind11/detail/type_caster_base.h
include/pybind11/detail/typeid.h include/pybind11/detail/typeid.h
include/pybind11/attr.h include/pybind11/attr.h
include/pybind11/buffer_info.h include/pybind11/buffer_info.h
@ -107,8 +162,13 @@ set(PYBIND11_HEADERS
include/pybind11/complex.h include/pybind11/complex.h
include/pybind11/options.h include/pybind11/options.h
include/pybind11/eigen.h include/pybind11/eigen.h
include/pybind11/eigen/common.h
include/pybind11/eigen/matrix.h
include/pybind11/eigen/tensor.h
include/pybind11/embed.h include/pybind11/embed.h
include/pybind11/eval.h include/pybind11/eval.h
include/pybind11/gil.h
include/pybind11/gil_safe_call_once.h
include/pybind11/iostream.h include/pybind11/iostream.h
include/pybind11/functional.h include/pybind11/functional.h
include/pybind11/numpy.h include/pybind11/numpy.h
@ -116,7 +176,10 @@ set(PYBIND11_HEADERS
include/pybind11/pybind11.h include/pybind11/pybind11.h
include/pybind11/pytypes.h include/pybind11/pytypes.h
include/pybind11/stl.h include/pybind11/stl.h
include/pybind11/stl_bind.h) include/pybind11/stl_bind.h
include/pybind11/stl/filesystem.h
include/pybind11/type_caster_pyobject_ptr.h
include/pybind11/typing.h)
# Compare with grep and warn if mismatched # Compare with grep and warn if mismatched
if(PYBIND11_MASTER_PROJECT AND NOT CMAKE_VERSION VERSION_LESS 3.12) if(PYBIND11_MASTER_PROJECT AND NOT CMAKE_VERSION VERSION_LESS 3.12)
@ -159,14 +222,33 @@ endif()
# You can also place ifs *in* the Config.in, but not here. # You can also place ifs *in* the Config.in, but not here.
# This section builds targets, but does *not* touch Python # This section builds targets, but does *not* touch Python
# Non-IMPORT targets cannot be defined twice
if(NOT TARGET pybind11_headers)
# Build the headers-only target (no Python included):
# (long name used here to keep this from clashing in subdirectory mode)
add_library(pybind11_headers INTERFACE)
add_library(pybind11::pybind11_headers ALIAS pybind11_headers) # to match exported target
add_library(pybind11::headers ALIAS pybind11_headers) # easier to use/remember
# Build the headers-only target (no Python included): target_include_directories(
# (long name used here to keep this from clashing in subdirectory mode) pybind11_headers ${pybind11_system} INTERFACE $<BUILD_INTERFACE:${pybind11_INCLUDE_DIR}>
add_library(pybind11_headers INTERFACE) $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
add_library(pybind11::pybind11_headers ALIAS pybind11_headers) # to match exported target
add_library(pybind11::headers ALIAS pybind11_headers) # easier to use/remember target_compile_features(pybind11_headers INTERFACE cxx_inheriting_constructors cxx_user_literals
cxx_right_angle_brackets)
if(NOT "${PYBIND11_INTERNALS_VERSION}" STREQUAL "")
target_compile_definitions(
pybind11_headers INTERFACE "PYBIND11_INTERNALS_VERSION=${PYBIND11_INTERNALS_VERSION}")
endif()
else()
# It is invalid to install a target twice, too.
set(PYBIND11_INSTALL OFF)
endif()
include("${CMAKE_CURRENT_SOURCE_DIR}/tools/pybind11Common.cmake") include("${CMAKE_CURRENT_SOURCE_DIR}/tools/pybind11Common.cmake")
# https://github.com/jtojnar/cmake-snips/#concatenating-paths-when-building-pkg-config-files
# TODO: cmake 3.20 adds the cmake_path() function, which obsoletes this snippet
include("${CMAKE_CURRENT_SOURCE_DIR}/tools/JoinPaths.cmake")
# Relative directory setting # Relative directory setting
if(USE_PYTHON_INCLUDE_DIR AND DEFINED Python_INCLUDE_DIRS) if(USE_PYTHON_INCLUDE_DIR AND DEFINED Python_INCLUDE_DIRS)
@ -175,20 +257,18 @@ elseif(USE_PYTHON_INCLUDE_DIR AND DEFINED PYTHON_INCLUDE_DIR)
file(RELATIVE_PATH CMAKE_INSTALL_INCLUDEDIR ${CMAKE_INSTALL_PREFIX} ${PYTHON_INCLUDE_DIRS}) file(RELATIVE_PATH CMAKE_INSTALL_INCLUDEDIR ${CMAKE_INSTALL_PREFIX} ${PYTHON_INCLUDE_DIRS})
endif() endif()
# Fill in headers target
target_include_directories(
pybind11_headers ${pybind11_system} INTERFACE $<BUILD_INTERFACE:${pybind11_INCLUDE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
target_compile_features(pybind11_headers INTERFACE cxx_inheriting_constructors cxx_user_literals
cxx_right_angle_brackets)
if(PYBIND11_INSTALL) if(PYBIND11_INSTALL)
install(DIRECTORY ${pybind11_INCLUDE_DIR}/pybind11 DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) install(DIRECTORY ${pybind11_INCLUDE_DIR}/pybind11 DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
set(PYBIND11_CMAKECONFIG_INSTALL_DIR set(PYBIND11_CMAKECONFIG_INSTALL_DIR
"${CMAKE_INSTALL_DATAROOTDIR}/cmake/${PROJECT_NAME}" "${CMAKE_INSTALL_DATAROOTDIR}/cmake/${PROJECT_NAME}"
CACHE STRING "install path for pybind11Config.cmake") CACHE STRING "install path for pybind11Config.cmake")
if(IS_ABSOLUTE "${CMAKE_INSTALL_INCLUDEDIR}")
set(pybind11_INCLUDEDIR "${CMAKE_INSTALL_FULL_INCLUDEDIR}")
else()
set(pybind11_INCLUDEDIR "\$\{PACKAGE_PREFIX_DIR\}/${CMAKE_INSTALL_INCLUDEDIR}")
endif()
configure_package_config_file( configure_package_config_file(
tools/${PROJECT_NAME}Config.cmake.in "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake" tools/${PROJECT_NAME}Config.cmake.in "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
INSTALL_DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR}) INSTALL_DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})
@ -233,6 +313,30 @@ if(PYBIND11_INSTALL)
NAMESPACE "pybind11::" NAMESPACE "pybind11::"
DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR}) DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})
# pkg-config support
if(NOT prefix_for_pc_file)
if(IS_ABSOLUTE "${CMAKE_INSTALL_DATAROOTDIR}")
set(prefix_for_pc_file "${CMAKE_INSTALL_PREFIX}")
else()
set(pc_datarootdir "${CMAKE_INSTALL_DATAROOTDIR}")
if(CMAKE_VERSION VERSION_LESS 3.20)
set(prefix_for_pc_file "\${pcfiledir}/..")
while(pc_datarootdir)
get_filename_component(pc_datarootdir "${pc_datarootdir}" DIRECTORY)
string(APPEND prefix_for_pc_file "/..")
endwhile()
else()
cmake_path(RELATIVE_PATH CMAKE_INSTALL_PREFIX BASE_DIRECTORY CMAKE_INSTALL_DATAROOTDIR
OUTPUT_VARIABLE prefix_for_pc_file)
endif()
endif()
endif()
join_paths(includedir_for_pc_file "\${prefix}" "${CMAKE_INSTALL_INCLUDEDIR}")
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/tools/pybind11.pc.in"
"${CMAKE_CURRENT_BINARY_DIR}/pybind11.pc" @ONLY)
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/pybind11.pc"
DESTINATION "${CMAKE_INSTALL_DATAROOTDIR}/pkgconfig/")
# Uninstall target # Uninstall target
if(PYBIND11_MASTER_PROJECT) if(PYBIND11_MASTER_PROJECT)
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/tools/cmake_uninstall.cmake.in" configure_file("${CMAKE_CURRENT_SOURCE_DIR}/tools/cmake_uninstall.cmake.in"

View File

@ -1,6 +1,6 @@
prune tests
recursive-include pybind11/include/pybind11 *.h recursive-include pybind11/include/pybind11 *.h
recursive-include pybind11 *.py recursive-include pybind11 *.py
recursive-include pybind11 py.typed recursive-include pybind11 py.typed
recursive-include pybind11 *.pyi
include pybind11/share/cmake/pybind11/*.cmake include pybind11/share/cmake/pybind11/*.cmake
include LICENSE README.rst pyproject.toml setup.py setup.cfg include LICENSE README.rst SECURITY.md pyproject.toml setup.py setup.cfg

View File

@ -3,20 +3,16 @@
**pybind11 — Seamless operability between C++11 and Python** **pybind11 — Seamless operability between C++11 and Python**
|Latest Documentation Status| |Stable Documentation Status| |Gitter chat| |CI| |Build status| |Latest Documentation Status| |Stable Documentation Status| |Gitter chat| |GitHub Discussions| |CI| |Build status|
.. warning:: |Repology| |PyPI package| |Conda-forge| |Python Versions|
Combining older versions of pybind11 (< 2.6.0) with the brand-new Python `Setuptools example <https://github.com/pybind/python_example>`_
3.9.0 will trigger undefined behavior that typically manifests as crashes `Scikit-build example <https://github.com/pybind/scikit_build_example>`_
during interpreter shutdown (but could also destroy your data. **You have been `CMake example <https://github.com/pybind/cmake_example>`_
warned.**)
.. start
We recommend that you wait for Python 3.9.1 slated for release in December,
which will include a `fix <https://github.com/python/cpython/pull/22670>`_
that resolves this problem. In the meantime, please update to the latest
version of pybind11 (2.6.0 or newer), which includes a temporary workaround
specifically when Python 3.9.0 is detected at runtime.
**pybind11** is a lightweight header-only library that exposes C++ types **pybind11** is a lightweight header-only library that exposes C++ types
in Python and vice versa, mainly to create Python bindings of existing in Python and vice versa, mainly to create Python bindings of existing
@ -36,14 +32,14 @@ this heavy machinery has become an excessively large and unnecessary
dependency. dependency.
Think of this library as a tiny self-contained version of Boost.Python Think of this library as a tiny self-contained version of Boost.Python
with everything stripped away that isnt relevant for binding with everything stripped away that isn't relevant for binding
generation. Without comments, the core header files only require ~4K generation. Without comments, the core header files only require ~4K
lines of code and depend on Python (2.7 or 3.5+, or PyPy) and the C++ lines of code and depend on Python (3.6+, or PyPy) and the C++
standard library. This compact implementation was possible thanks to standard library. This compact implementation was possible thanks to
some of the new C++11 language features (specifically: tuples, lambda some C++11 language features (specifically: tuples, lambda functions and
functions and variadic templates). Since its creation, this library has variadic templates). Since its creation, this library has grown beyond
grown beyond Boost.Python in many ways, leading to dramatically simpler Boost.Python in many ways, leading to dramatically simpler binding code in many
binding code in many common situations. common situations.
Tutorial and reference documentation is provided at Tutorial and reference documentation is provided at
`pybind11.readthedocs.io <https://pybind11.readthedocs.io/en/latest>`_. `pybind11.readthedocs.io <https://pybind11.readthedocs.io/en/latest>`_.
@ -75,6 +71,7 @@ pybind11 can map the following core C++ features to Python:
- Internal references with correct reference counting - Internal references with correct reference counting
- C++ classes with virtual (and pure virtual) methods can be extended - C++ classes with virtual (and pure virtual) methods can be extended
in Python in Python
- Integrated NumPy support (NumPy 2 requires pybind11 2.12+)
Goodies Goodies
------- -------
@ -82,8 +79,8 @@ Goodies
In addition to the core functionality, pybind11 provides some extra In addition to the core functionality, pybind11 provides some extra
goodies: goodies:
- Python 2.7, 3.5+, and PyPy/PyPy3 7.3 are supported with an - Python 3.6+, and PyPy3 7.3 are supported with an implementation-agnostic
implementation-agnostic interface. interface (pybind11 2.9 was the last version to support Python 2 and 3.5).
- It is possible to bind C++11 lambda functions with captured - It is possible to bind C++11 lambda functions with captured
variables. The lambda capture data is stored inside the resulting variables. The lambda capture data is stored inside the resulting
@ -92,8 +89,8 @@ goodies:
- pybind11 uses C++11 move constructors and move assignment operators - pybind11 uses C++11 move constructors and move assignment operators
whenever possible to efficiently transfer custom data types. whenever possible to efficiently transfer custom data types.
- Its easy to expose the internal storage of custom data types through - It's easy to expose the internal storage of custom data types through
Pythons buffer protocols. This is handy e.g. for fast conversion Pythons' buffer protocols. This is handy e.g. for fast conversion
between C++ matrix classes like Eigen and NumPy without expensive between C++ matrix classes like Eigen and NumPy without expensive
copy operations. copy operations.
@ -101,7 +98,7 @@ goodies:
transparently applied to all entries of one or more NumPy array transparently applied to all entries of one or more NumPy array
arguments. arguments.
- Pythons slice-based access and assignment operations can be - Python's slice-based access and assignment operations can be
supported with just a few lines of code. supported with just a few lines of code.
- Everything is contained in just a few header files; there is no need - Everything is contained in just a few header files; there is no need
@ -110,7 +107,7 @@ goodies:
- Binaries are generally smaller by a factor of at least 2 compared to - Binaries are generally smaller by a factor of at least 2 compared to
equivalent bindings generated by Boost.Python. A recent pybind11 equivalent bindings generated by Boost.Python. A recent pybind11
conversion of PyRosetta, an enormous Boost.Python binding project, conversion of PyRosetta, an enormous Boost.Python binding project,
`reported <http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf>`_ `reported <https://graylab.jhu.edu/Sergey/2016.RosettaCon/PyRosetta-4.pdf>`_
a binary size reduction of **5.4x** and compile time reduction by a binary size reduction of **5.4x** and compile time reduction by
**5.8x**. **5.8x**.
@ -123,15 +120,14 @@ goodies:
Supported compilers Supported compilers
------------------- -------------------
1. Clang/LLVM 3.3 or newer (for Apple Xcodes clang, this is 5.0.0 or 1. Clang/LLVM 3.3 or newer (for Apple Xcode's clang, this is 5.0.0 or
newer) newer)
2. GCC 4.8 or newer 2. GCC 4.8 or newer
3. Microsoft Visual Studio 2015 Update 3 or newer 3. Microsoft Visual Studio 2017 or newer
4. Intel C++ compiler 18 or newer 4. Intel classic C++ compiler 18 or newer (ICC 20.2 tested in CI)
(`possible issue <https://github.com/pybind/pybind11/pull/2573>`_ on 20.2) 5. Cygwin/GCC (previously tested on 2.5.1)
5. Cygwin/GCC (tested on 2.5.1) 6. NVCC (CUDA 11.0 tested in CI)
6. NVCC (CUDA 11.0 tested) 7. NVIDIA PGI (20.9 tested in CI)
7. NVIDIA PGI (20.7 and 20.9 tested)
About About
----- -----
@ -139,9 +135,9 @@ About
This project was created by `Wenzel This project was created by `Wenzel
Jakob <http://rgl.epfl.ch/people/wjakob>`_. Significant features and/or Jakob <http://rgl.epfl.ch/people/wjakob>`_. Significant features and/or
improvements to the code were contributed by Jonas Adler, Lori A. Burns, improvements to the code were contributed by Jonas Adler, Lori A. Burns,
Sylvain Corlay, Eric Cousineau, Ralf Grosse-Kunstleve, Trent Houliston, Axel Sylvain Corlay, Eric Cousineau, Aaron Gokaslan, Ralf Grosse-Kunstleve, Trent Houliston, Axel
Huebl, @hulucc, Yannick Jadoul, Sergey Lyskov Johan Mabille, Tomasz Miąsko, Huebl, @hulucc, Yannick Jadoul, Sergey Lyskov, Johan Mabille, Tomasz Miąsko,
Dean Moldovan, Ben Pritchard, Jason Rhinelander, Boris Schäling, Pim Dean Moldovan, Ben Pritchard, Jason Rhinelander, Boris Schäling, Pim
Schellart, Henry Schreiner, Ivan Smirnov, Boris Staletic, and Patrick Stewart. Schellart, Henry Schreiner, Ivan Smirnov, Boris Staletic, and Patrick Stewart.
We thank Google for a generous financial contribution to the continuous We thank Google for a generous financial contribution to the continuous
@ -165,7 +161,7 @@ to the terms and conditions of this license.
.. |Latest Documentation Status| image:: https://readthedocs.org/projects/pybind11/badge?version=latest .. |Latest Documentation Status| image:: https://readthedocs.org/projects/pybind11/badge?version=latest
:target: http://pybind11.readthedocs.org/en/latest :target: http://pybind11.readthedocs.org/en/latest
.. |Stable Documentation Status| image:: https://img.shields.io/badge/docs-stable-blue .. |Stable Documentation Status| image:: https://img.shields.io/badge/docs-stable-blue.svg
:target: http://pybind11.readthedocs.org/en/stable :target: http://pybind11.readthedocs.org/en/stable
.. |Gitter chat| image:: https://img.shields.io/gitter/room/gitterHQ/gitter.svg .. |Gitter chat| image:: https://img.shields.io/gitter/room/gitterHQ/gitter.svg
:target: https://gitter.im/pybind/Lobby :target: https://gitter.im/pybind/Lobby
@ -173,3 +169,13 @@ to the terms and conditions of this license.
:target: https://github.com/pybind/pybind11/actions :target: https://github.com/pybind/pybind11/actions
.. |Build status| image:: https://ci.appveyor.com/api/projects/status/riaj54pn4h08xy40?svg=true .. |Build status| image:: https://ci.appveyor.com/api/projects/status/riaj54pn4h08xy40?svg=true
:target: https://ci.appveyor.com/project/wjakob/pybind11 :target: https://ci.appveyor.com/project/wjakob/pybind11
.. |PyPI package| image:: https://img.shields.io/pypi/v/pybind11.svg
:target: https://pypi.org/project/pybind11/
.. |Conda-forge| image:: https://img.shields.io/conda/vn/conda-forge/pybind11.svg
:target: https://github.com/conda-forge/pybind11-feedstock
.. |Repology| image:: https://repology.org/badge/latest-versions/python:pybind11.svg
:target: https://repology.org/project/python:pybind11/versions
.. |Python Versions| image:: https://img.shields.io/pypi/pyversions/pybind11.svg
:target: https://pypi.org/project/pybind11/
.. |GitHub Discussions| image:: https://img.shields.io/static/v1?label=Discussions&message=Ask&color=blue&logo=github
:target: https://github.com/pybind/pybind11/discussions

13
3rdparty/pybind11/SECURITY.md vendored Normal file
View File

@ -0,0 +1,13 @@
# Security Policy
## Supported Versions
Security updates are applied only to the latest release.
## Reporting a Vulnerability
If you have discovered a security vulnerability in this project, please report it privately. **Do not disclose it as a public issue.** This gives us time to work with you to fix the issue before public exposure, reducing the chance that the exploit will be used before a patch is released.
Please disclose it at [security advisory](https://github.com/pybind/pybind11/security/advisories/new).
This project is maintained by a team of volunteers on a reasonable-effort basis. As such, please give us at least 90 days to work on a fix before public exposure.

View File

@ -18,6 +18,4 @@ ALIASES += "endrst=\endverbatim"
QUIET = YES QUIET = YES
WARNINGS = YES WARNINGS = YES
WARN_IF_UNDOCUMENTED = NO WARN_IF_UNDOCUMENTED = NO
PREDEFINED = DOXYGEN_SHOULD_SKIP_THIS \ PREDEFINED = PYBIND11_NOINLINE
PY_MAJOR_VERSION=3 \
PYBIND11_NOINLINE

View File

@ -0,0 +1,3 @@
.highlight .go {
color: #707070;
}

View File

@ -1,11 +0,0 @@
.wy-table-responsive table td,
.wy-table-responsive table th {
white-space: initial !important;
}
.rst-content table.docutils td {
vertical-align: top !important;
}
div[class^='highlight'] pre {
white-space: pre;
white-space: pre-wrap;
}

View File

@ -26,7 +26,9 @@ The following Python snippet demonstrates the intended usage from the Python sid
def __int__(self): def __int__(self):
return 123 return 123
from example import print from example import print
print(A()) print(A())
To register the necessary conversion routines, it is necessary to add an To register the necessary conversion routines, it is necessary to add an
@ -36,7 +38,7 @@ type is explicitly allowed.
.. code-block:: cpp .. code-block:: cpp
namespace pybind11 { namespace detail { namespace PYBIND11_NAMESPACE { namespace detail {
template <> struct type_caster<inty> { template <> struct type_caster<inty> {
public: public:
/** /**
@ -44,7 +46,7 @@ type is explicitly allowed.
* function signatures and declares a local variable * function signatures and declares a local variable
* 'value' of type inty * 'value' of type inty
*/ */
PYBIND11_TYPE_CASTER(inty, _("inty")); PYBIND11_TYPE_CASTER(inty, const_name("inty"));
/** /**
* Conversion part 1 (Python->C++): convert a PyObject into a inty * Conversion part 1 (Python->C++): convert a PyObject into a inty
@ -76,7 +78,7 @@ type is explicitly allowed.
return PyLong_FromLong(src.long_value); return PyLong_FromLong(src.long_value);
} }
}; };
}} // namespace pybind11::detail }} // namespace PYBIND11_NAMESPACE::detail
.. note:: .. note::

View File

@ -52,7 +52,7 @@ can be mapped *and* if the numpy array is writeable (that is
the passed variable will be transparently carried out directly on the the passed variable will be transparently carried out directly on the
``numpy.ndarray``. ``numpy.ndarray``.
This means you can can write code such as the following and have it work as This means you can write code such as the following and have it work as
expected: expected:
.. code-block:: cpp .. code-block:: cpp
@ -112,7 +112,7 @@ example:
.. code-block:: python .. code-block:: python
a = MyClass() a = MyClass()
m = a.get_matrix() # flags.writeable = True, flags.owndata = False m = a.get_matrix() # flags.writeable = True, flags.owndata = False
v = a.view_matrix() # flags.writeable = False, flags.owndata = False v = a.view_matrix() # flags.writeable = False, flags.owndata = False
c = a.copy_matrix() # flags.writeable = True, flags.owndata = True c = a.copy_matrix() # flags.writeable = True, flags.owndata = True
# m[5,6] and v[5,6] refer to the same element, c[5,6] does not. # m[5,6] and v[5,6] refer to the same element, c[5,6] does not.
@ -203,7 +203,7 @@ adding the ``order='F'`` option when creating an array:
.. code-block:: python .. code-block:: python
myarray = np.array(source, order='F') myarray = np.array(source, order="F")
Such an object will be passable to a bound function accepting an Such an object will be passable to a bound function accepting an
``Eigen::Ref<MatrixXd>`` (or similar column-major Eigen type). ``Eigen::Ref<MatrixXd>`` (or similar column-major Eigen type).

View File

@ -75,91 +75,96 @@ The following basic data types are supported out of the box (some may require
an additional extension header to be included). To pass other data structures an additional extension header to be included). To pass other data structures
as arguments and return values, refer to the section on binding :ref:`classes`. as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| Data type | Description | Header file | | Data type | Description | Header file |
+====================================+===========================+===============================+ +====================================+===========================+===================================+
| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` | | ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` | | ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` | | ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` | | ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` | | ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` | | ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` | | ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``char`` | Character literal | :file:`pybind11/pybind11.h` | | ``char`` | Character literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``char16_t`` | UTF-16 character literal | :file:`pybind11/pybind11.h` | | ``char16_t`` | UTF-16 character literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``char32_t`` | UTF-32 character literal | :file:`pybind11/pybind11.h` | | ``char32_t`` | UTF-32 character literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` | | ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` | | ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``const char16_t *`` | UTF-16 string literal | :file:`pybind11/pybind11.h` | | ``const char16_t *`` | UTF-16 string literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``const char32_t *`` | UTF-32 string literal | :file:`pybind11/pybind11.h` | | ``const char32_t *`` | UTF-32 string literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` | | ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` | | ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::u16string`` | STL dynamic UTF-16 string | :file:`pybind11/pybind11.h` | | ``std::u16string`` | STL dynamic UTF-16 string | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::u32string`` | STL dynamic UTF-32 string | :file:`pybind11/pybind11.h` | | ``std::u32string`` | STL dynamic UTF-32 string | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` | | ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::string_view``, | STL C++17 string views | :file:`pybind11/pybind11.h` | | ``std::string_view``, | STL C++17 string views | :file:`pybind11/pybind11.h` |
| ``std::u16string_view``, etc. | | | | ``std::u16string_view``, etc. | | |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` | | ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` | | ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` | | ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` | | ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` | | ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` | | ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::deque<T>`` | STL double-ended queue | :file:`pybind11/stl.h` | | ``std::deque<T>`` | STL double-ended queue | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::valarray<T>`` | STL value array | :file:`pybind11/stl.h` | | ``std::valarray<T>`` | STL value array | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` | | ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` | | ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` | | ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` | | ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` | | ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::optional<T>`` | STL optional type (C++17) | :file:`pybind11/stl.h` | | ``std::optional<T>`` | STL optional type (C++17) | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::experimental::optional<T>`` | STL optional type (exp.) | :file:`pybind11/stl.h` | | ``std::experimental::optional<T>`` | STL optional type (exp.) | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` | | ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` | | ``std::filesystem::path<T>`` | STL path (C++17) [#]_ | :file:`pybind11/stl/filesystem.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` | | ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``std::chrono::time_point<...>`` | STL date/time | :file:`pybind11/chrono.h` | | ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` | | ``std::chrono::time_point<...>`` | STL date/time | :file:`pybind11/chrono.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` | | ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` | | ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
+------------------------------------+---------------------------+-------------------------------+ +------------------------------------+---------------------------+-----------------------------------+
| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
+------------------------------------+---------------------------+-----------------------------------+
.. [#] ``std::filesystem::path`` is converted to ``pathlib.Path`` and
``os.PathLike`` is converted to ``std::filesystem::path``.

View File

@ -42,7 +42,7 @@ types:
.. code-block:: cpp .. code-block:: cpp
// `boost::optional` as an example -- can be any `std::optional`-like container // `boost::optional` as an example -- can be any `std::optional`-like container
namespace pybind11 { namespace detail { namespace PYBIND11_NAMESPACE { namespace detail {
template <typename T> template <typename T>
struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {}; struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {};
}} }}
@ -54,7 +54,7 @@ for custom variant types:
.. code-block:: cpp .. code-block:: cpp
// `boost::variant` as an example -- can be any `std::variant`-like container // `boost::variant` as an example -- can be any `std::variant`-like container
namespace pybind11 { namespace detail { namespace PYBIND11_NAMESPACE { namespace detail {
template <typename... Ts> template <typename... Ts>
struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {}; struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {};
@ -66,18 +66,27 @@ for custom variant types:
return boost::apply_visitor(args...); return boost::apply_visitor(args...);
} }
}; };
}} // namespace pybind11::detail }} // namespace PYBIND11_NAMESPACE::detail
The ``visit_helper`` specialization is not required if your ``name::variant`` provides The ``visit_helper`` specialization is not required if your ``name::variant`` provides
a ``name::visit()`` function. For any other function name, the specialization must be a ``name::visit()`` function. For any other function name, the specialization must be
included to tell pybind11 how to visit the variant. included to tell pybind11 how to visit the variant.
.. warning::
When converting a ``variant`` type, pybind11 follows the same rules as when
determining which function overload to call (:ref:`overload_resolution`), and
so the same caveats hold. In particular, the order in which the ``variant``'s
alternatives are listed is important, since pybind11 will try conversions in
this order. This means that, for example, when converting ``variant<int, bool>``,
the ``bool`` variant will never be selected, as any Python ``bool`` is already
an ``int`` and is convertible to a C++ ``int``. Changing the order of alternatives
(and using ``variant<bool, int>``, in this example) provides a solution.
.. note:: .. note::
pybind11 only supports the modern implementation of ``boost::variant`` pybind11 only supports the modern implementation of ``boost::variant``
which makes use of variadic templates. This requires Boost 1.56 or newer. which makes use of variadic templates. This requires Boost 1.56 or newer.
Additionally, on Windows, MSVC 2017 is required because ``boost::variant``
falls back to the old non-variadic implementation on MSVC 2015.
.. _opaque: .. _opaque:

View File

@ -1,14 +1,6 @@
Strings, bytes and Unicode conversions Strings, bytes and Unicode conversions
###################################### ######################################
.. note::
This section discusses string handling in terms of Python 3 strings. For
Python 2.7, replace all occurrences of ``str`` with ``unicode`` and
``bytes`` with ``str``. Python 2.7 users may find it best to use ``from
__future__ import unicode_literals`` to avoid unintentionally using ``str``
instead of ``unicode``.
Passing Python strings to C++ Passing Python strings to C++
============================= =============================
@ -36,13 +28,13 @@ everywhere <http://utf8everywhere.org/>`_.
} }
); );
.. code-block:: python .. code-block:: pycon
>>> utf8_test('🎂') >>> utf8_test("🎂")
utf-8 is icing on the cake. utf-8 is icing on the cake.
🎂 🎂
>>> utf8_charptr('🍕') >>> utf8_charptr("🍕")
My favorite food is My favorite food is
🍕 🍕
@ -58,9 +50,9 @@ Passing bytes to C++
-------------------- --------------------
A Python ``bytes`` object will be passed to C++ functions that accept A Python ``bytes`` object will be passed to C++ functions that accept
``std::string`` or ``char*`` *without* conversion. On Python 3, in order to ``std::string`` or ``char*`` *without* conversion. In order to make a function
make a function *only* accept ``bytes`` (and not ``str``), declare it as taking *only* accept ``bytes`` (and not ``str``), declare it as taking a ``py::bytes``
a ``py::bytes`` argument. argument.
Returning C++ strings to Python Returning C++ strings to Python
@ -80,7 +72,7 @@ raise a ``UnicodeDecodeError``.
} }
); );
.. code-block:: python .. code-block:: pycon
>>> isinstance(example.std_string_return(), str) >>> isinstance(example.std_string_return(), str)
True True
@ -109,19 +101,23 @@ conversion has the same overhead as implicit conversion.
m.def("str_output", m.def("str_output",
[]() { []() {
std::string s = "Send your r\xe9sum\xe9 to Alice in HR"; // Latin-1 std::string s = "Send your r\xe9sum\xe9 to Alice in HR"; // Latin-1
py::str py_s = PyUnicode_DecodeLatin1(s.data(), s.length()); py::handle py_s = PyUnicode_DecodeLatin1(s.data(), s.length(), nullptr);
return py_s; if (!py_s) {
throw py::error_already_set();
}
return py::reinterpret_steal<py::str>(py_s);
} }
); );
.. code-block:: python .. code-block:: pycon
>>> str_output() >>> str_output()
'Send your résumé to Alice in HR' 'Send your résumé to Alice in HR'
The `Python C API The `Python C API
<https://docs.python.org/3/c-api/unicode.html#built-in-codecs>`_ provides <https://docs.python.org/3/c-api/unicode.html#built-in-codecs>`_ provides
several built-in codecs. several built-in codecs. Note that these all return *new* references, so
use :cpp:func:`reinterpret_steal` when converting them to a :cpp:class:`str`.
One could also use a third party encoding library such as libiconv to transcode One could also use a third party encoding library such as libiconv to transcode
@ -143,7 +139,7 @@ returned to Python as ``bytes``, then one can return the data as a
} }
); );
.. code-block:: python .. code-block:: pycon
>>> example.return_bytes() >>> example.return_bytes()
b'\xba\xd0\xba\xd0' b'\xba\xd0\xba\xd0'
@ -160,7 +156,7 @@ encoding, but cannot convert ``std::string`` back to ``bytes`` implicitly.
} }
); );
.. code-block:: python .. code-block:: pycon
>>> isinstance(example.asymmetry(b"have some bytes"), str) >>> isinstance(example.asymmetry(b"have some bytes"), str)
True True
@ -204,11 +200,6 @@ decoded to Python ``str``.
} }
); );
.. warning::
Wide character strings may not work as described on Python 2.7 or Python
3.3 compiled with ``--enable-unicode=ucs2``.
Strings in multibyte encodings such as Shift-JIS must transcoded to a Strings in multibyte encodings such as Shift-JIS must transcoded to a
UTF-8/16/32 before being returned to Python. UTF-8/16/32 before being returned to Python.
@ -229,16 +220,16 @@ character.
m.def("pass_char", [](char c) { return c; }); m.def("pass_char", [](char c) { return c; });
m.def("pass_wchar", [](wchar_t w) { return w; }); m.def("pass_wchar", [](wchar_t w) { return w; });
.. code-block:: python .. code-block:: pycon
>>> example.pass_char('A') >>> example.pass_char("A")
'A' 'A'
While C++ will cast integers to character types (``char c = 0x65;``), pybind11 While C++ will cast integers to character types (``char c = 0x65;``), pybind11
does not convert Python integers to characters implicitly. The Python function does not convert Python integers to characters implicitly. The Python function
``chr()`` can be used to convert integers to characters. ``chr()`` can be used to convert integers to characters.
.. code-block:: python .. code-block:: pycon
>>> example.pass_char(0x65) >>> example.pass_char(0x65)
TypeError TypeError
@ -259,17 +250,17 @@ a combining acute accent). The combining character will be lost if the
two-character sequence is passed as an argument, even though it renders as a two-character sequence is passed as an argument, even though it renders as a
single grapheme. single grapheme.
.. code-block:: python .. code-block:: pycon
>>> example.pass_wchar('é') >>> example.pass_wchar("é")
'é' 'é'
>>> combining_e_acute = 'e' + '\u0301' >>> combining_e_acute = "e" + "\u0301"
>>> combining_e_acute >>> combining_e_acute
'é' 'é'
>>> combining_e_acute == 'é' >>> combining_e_acute == "é"
False False
>>> example.pass_wchar(combining_e_acute) >>> example.pass_wchar(combining_e_acute)
@ -278,9 +269,9 @@ single grapheme.
Normalizing combining characters before passing the character literal to C++ Normalizing combining characters before passing the character literal to C++
may resolve *some* of these issues: may resolve *some* of these issues:
.. code-block:: python .. code-block:: pycon
>>> example.pass_wchar(unicodedata.normalize('NFC', combining_e_acute)) >>> example.pass_wchar(unicodedata.normalize("NFC", combining_e_acute))
'é' 'é'
In some languages (Thai for example), there are `graphemes that cannot be In some languages (Thai for example), there are `graphemes that cannot be

View File

@ -9,7 +9,7 @@ that you are already familiar with the basics from :doc:`/classes`.
Overriding virtual functions in Python Overriding virtual functions in Python
====================================== ======================================
Suppose that a C++ class or interface has a virtual function that we'd like to Suppose that a C++ class or interface has a virtual function that we'd like
to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is
given as a specific example of how one would do this with traditional C++ given as a specific example of how one would do this with traditional C++
code). code).
@ -133,14 +133,14 @@ a virtual method call.
>>> from example import * >>> from example import *
>>> d = Dog() >>> d = Dog()
>>> call_go(d) >>> call_go(d)
u'woof! woof! woof! ' 'woof! woof! woof! '
>>> class Cat(Animal): >>> class Cat(Animal):
... def go(self, n_times): ... def go(self, n_times):
... return "meow! " * n_times ... return "meow! " * n_times
... ...
>>> c = Cat() >>> c = Cat()
>>> call_go(c) >>> call_go(c)
u'meow! meow! meow! ' 'meow! meow! meow! '
If you are defining a custom constructor in a derived Python class, you *must* If you are defining a custom constructor in a derived Python class, you *must*
ensure that you explicitly call the bound C++ constructor using ``__init__``, ensure that you explicitly call the bound C++ constructor using ``__init__``,
@ -159,8 +159,9 @@ Here is an example:
class Dachshund(Dog): class Dachshund(Dog):
def __init__(self, name): def __init__(self, name):
Dog.__init__(self) # Without this, a TypeError is raised. Dog.__init__(self) # Without this, a TypeError is raised.
self.name = name self.name = name
def bark(self): def bark(self):
return "yap!" return "yap!"
@ -259,7 +260,7 @@ override the ``name()`` method):
.. note:: .. note::
Note the trailing commas in the ``PYBIND11_OVERIDE`` calls to ``name()`` Note the trailing commas in the ``PYBIND11_OVERRIDE`` calls to ``name()``
and ``bark()``. These are needed to portably implement a trampoline for a and ``bark()``. These are needed to portably implement a trampoline for a
function that does not take any arguments. For functions that take function that does not take any arguments. For functions that take
a nonzero number of arguments, the trailing comma must be omitted. a nonzero number of arguments, the trailing comma must be omitted.
@ -804,7 +805,7 @@ to bind these two functions:
} }
)); ));
The ``__setstate__`` part of the ``py::picke()`` definition follows the same The ``__setstate__`` part of the ``py::pickle()`` definition follows the same
rules as the single-argument version of ``py::init()``. The return type can be rules as the single-argument version of ``py::init()``. The return type can be
a value, pointer or holder type. See :ref:`custom_constructors` for details. a value, pointer or holder type. See :ref:`custom_constructors` for details.
@ -812,26 +813,21 @@ An instance can now be pickled as follows:
.. code-block:: python .. code-block:: python
try: import pickle
import cPickle as pickle # Use cPickle on Python 2.7
except ImportError:
import pickle
p = Pickleable("test_value") p = Pickleable("test_value")
p.setExtra(15) p.setExtra(15)
data = pickle.dumps(p, 2) data = pickle.dumps(p)
.. note:: .. note::
Note that only the cPickle module is supported on Python 2.7. If given, the second argument to ``dumps`` must be 2 or larger - 0 and 1 are
not supported. Newer versions are also fine; for instance, specify ``-1`` to
The second argument to ``dumps`` is also crucial: it selects the pickle always use the latest available version. Beware: failure to follow these
protocol version 2, since the older version 1 is not supported. Newer instructions will cause important pybind11 memory allocation routines to be
versions are also fine—for instance, specify ``-1`` to always use the skipped during unpickling, which will likely lead to memory corruption
latest available version. Beware: failure to follow these instructions and/or segmentation faults. Python defaults to version 3 (Python 3-3.7) and
will cause important pybind11 memory allocation routines to be skipped version 4 for Python 3.8+.
during unpickling, which will likely lead to memory corruption and/or
segmentation faults.
.. seealso:: .. seealso::
@ -848,11 +844,9 @@ Python normally uses references in assignments. Sometimes a real copy is needed
to prevent changing all copies. The ``copy`` module [#f5]_ provides these to prevent changing all copies. The ``copy`` module [#f5]_ provides these
capabilities. capabilities.
On Python 3, a class with pickle support is automatically also (deep)copy A class with pickle support is automatically also (deep)copy
compatible. However, performance can be improved by adding custom compatible. However, performance can be improved by adding custom
``__copy__`` and ``__deepcopy__`` methods. With Python 2.7, these custom methods ``__copy__`` and ``__deepcopy__`` methods.
are mandatory for (deep)copy compatibility, because pybind11 only supports
cPickle.
For simple classes (deep)copy can be enabled by using the copy constructor, For simple classes (deep)copy can be enabled by using the copy constructor,
which should look as follows: which should look as follows:
@ -1124,13 +1118,6 @@ described trampoline:
py::class_<A, Trampoline>(m, "A") // <-- `Trampoline` here py::class_<A, Trampoline>(m, "A") // <-- `Trampoline` here
.def("foo", &Publicist::foo); // <-- `Publicist` here, not `Trampoline`! .def("foo", &Publicist::foo); // <-- `Publicist` here, not `Trampoline`!
.. note::
MSVC 2015 has a compiler bug (fixed in version 2017) which
requires a more explicit function binding in the form of
``.def("foo", static_cast<int (A::*)() const>(&Publicist::foo));``
where ``int (A::*)() const`` is the type of ``A::foo``.
Binding final classes Binding final classes
===================== =====================
@ -1153,12 +1140,65 @@ error:
>>> class PyFinalChild(IsFinal): >>> class PyFinalChild(IsFinal):
... pass ... pass
...
TypeError: type 'IsFinal' is not an acceptable base type TypeError: type 'IsFinal' is not an acceptable base type
.. note:: This attribute is currently ignored on PyPy .. note:: This attribute is currently ignored on PyPy
.. versionadded:: 2.6 .. versionadded:: 2.6
Binding classes with template parameters
========================================
pybind11 can also wrap classes that have template parameters. Consider these classes:
.. code-block:: cpp
struct Cat {};
struct Dog {};
template <typename PetType>
struct Cage {
Cage(PetType& pet);
PetType& get();
};
C++ templates may only be instantiated at compile time, so pybind11 can only
wrap instantiated templated classes. You cannot wrap a non-instantiated template:
.. code-block:: cpp
// BROKEN (this will not compile)
py::class_<Cage>(m, "Cage");
.def("get", &Cage::get);
You must explicitly specify each template/type combination that you want to
wrap separately.
.. code-block:: cpp
// ok
py::class_<Cage<Cat>>(m, "CatCage")
.def("get", &Cage<Cat>::get);
// ok
py::class_<Cage<Dog>>(m, "DogCage")
.def("get", &Cage<Dog>::get);
If your class methods have template parameters you can wrap those as well,
but once again each instantiation must be explicitly specified:
.. code-block:: cpp
typename <typename T>
struct MyClass {
template <typename V>
T fn(V v);
};
py::class<MyClass<int>>(m, "MyClassT")
.def("fn", &MyClass<int>::fn<std::string>);
Custom automatic downcasters Custom automatic downcasters
============================ ============================
@ -1188,7 +1228,7 @@ whether a downcast is safe, you can proceed by specializing the
std::string bark() const { return sound; } std::string bark() const { return sound; }
}; };
namespace pybind11 { namespace PYBIND11_NAMESPACE {
template<> struct polymorphic_type_hook<Pet> { template<> struct polymorphic_type_hook<Pet> {
static const void *get(const Pet *src, const std::type_info*& type) { static const void *get(const Pet *src, const std::type_info*& type) {
// note that src may be nullptr // note that src may be nullptr
@ -1199,7 +1239,7 @@ whether a downcast is safe, you can proceed by specializing the
return src; return src;
} }
}; };
} // namespace pybind11 } // namespace PYBIND11_NAMESPACE
When pybind11 wants to convert a C++ pointer of type ``Base*`` to a When pybind11 wants to convert a C++ pointer of type ``Base*`` to a
Python object, it calls ``polymorphic_type_hook<Base>::get()`` to Python object, it calls ``polymorphic_type_hook<Base>::get()`` to
@ -1247,7 +1287,7 @@ Accessing the type object
You can get the type object from a C++ class that has already been registered using: You can get the type object from a C++ class that has already been registered using:
.. code-block:: python .. code-block:: cpp
py::type T_py = py::type::of<T>(); py::type T_py = py::type::of<T>();
@ -1259,3 +1299,37 @@ object, just like ``type(ob)`` in Python.
Other types, like ``py::type::of<int>()``, do not work, see :ref:`type-conversions`. Other types, like ``py::type::of<int>()``, do not work, see :ref:`type-conversions`.
.. versionadded:: 2.6 .. versionadded:: 2.6
Custom type setup
=================
For advanced use cases, such as enabling garbage collection support, you may
wish to directly manipulate the ``PyHeapTypeObject`` corresponding to a
``py::class_`` definition.
You can do that using ``py::custom_type_setup``:
.. code-block:: cpp
struct OwnsPythonObjects {
py::object value = py::none();
};
py::class_<OwnsPythonObjects> cls(
m, "OwnsPythonObjects", py::custom_type_setup([](PyHeapTypeObject *heap_type) {
auto *type = &heap_type->ht_type;
type->tp_flags |= Py_TPFLAGS_HAVE_GC;
type->tp_traverse = [](PyObject *self_base, visitproc visit, void *arg) {
auto &self = py::cast<OwnsPythonObjects&>(py::handle(self_base));
Py_VISIT(self.value.ptr());
return 0;
};
type->tp_clear = [](PyObject *self_base) {
auto &self = py::cast<OwnsPythonObjects&>(py::handle(self_base));
self.value = py::none();
return 0;
};
}));
cls.def(py::init<>());
cls.def_readwrite("value", &OwnsPythonObjects::value);
.. versionadded:: 2.8

View File

@ -18,7 +18,7 @@ information, see :doc:`/compiling`.
.. code-block:: cmake .. code-block:: cmake
cmake_minimum_required(VERSION 3.4) cmake_minimum_required(VERSION 3.5...3.27)
project(example) project(example)
find_package(pybind11 REQUIRED) # or `add_subdirectory(pybind11)` find_package(pybind11 REQUIRED) # or `add_subdirectory(pybind11)`
@ -40,15 +40,15 @@ The essential structure of the ``main.cpp`` file looks like this:
} }
The interpreter must be initialized before using any Python API, which includes The interpreter must be initialized before using any Python API, which includes
all the functions and classes in pybind11. The RAII guard class `scoped_interpreter` all the functions and classes in pybind11. The RAII guard class ``scoped_interpreter``
takes care of the interpreter lifetime. After the guard is destroyed, the interpreter takes care of the interpreter lifetime. After the guard is destroyed, the interpreter
shuts down and clears its memory. No Python functions can be called after this. shuts down and clears its memory. No Python functions can be called after this.
Executing Python code Executing Python code
===================== =====================
There are a few different ways to run Python code. One option is to use `eval`, There are a few different ways to run Python code. One option is to use ``eval``,
`exec` or `eval_file`, as explained in :ref:`eval`. Here is a quick example in ``exec`` or ``eval_file``, as explained in :ref:`eval`. Here is a quick example in
the context of an executable with an embedded interpreter: the context of an executable with an embedded interpreter:
.. code-block:: cpp .. code-block:: cpp
@ -108,7 +108,7 @@ The two approaches can also be combined:
Importing modules Importing modules
================= =================
Python modules can be imported using `module_::import()`: Python modules can be imported using ``module_::import()``:
.. code-block:: cpp .. code-block:: cpp
@ -122,6 +122,7 @@ embedding the interpreter. This makes it easy to import local Python files:
"""calc.py located in the working directory""" """calc.py located in the working directory"""
def add(i, j): def add(i, j):
return i + j return i + j
@ -133,7 +134,7 @@ embedding the interpreter. This makes it easy to import local Python files:
int n = result.cast<int>(); int n = result.cast<int>();
assert(n == 3); assert(n == 3);
Modules can be reloaded using `module_::reload()` if the source is modified e.g. Modules can be reloaded using ``module_::reload()`` if the source is modified e.g.
by an external process. This can be useful in scenarios where the application by an external process. This can be useful in scenarios where the application
imports a user defined data processing script which needs to be updated after imports a user defined data processing script which needs to be updated after
changes by the user. Note that this function does not reload modules recursively. changes by the user. Note that this function does not reload modules recursively.
@ -143,7 +144,7 @@ changes by the user. Note that this function does not reload modules recursively
Adding embedded modules Adding embedded modules
======================= =======================
Embedded binary modules can be added using the `PYBIND11_EMBEDDED_MODULE` macro. Embedded binary modules can be added using the ``PYBIND11_EMBEDDED_MODULE`` macro.
Note that the definition must be placed at global scope. They can be imported Note that the definition must be placed at global scope. They can be imported
like any other module. like any other module.
@ -169,7 +170,7 @@ like any other module.
Unlike extension modules where only a single binary module can be created, on Unlike extension modules where only a single binary module can be created, on
the embedded side an unlimited number of modules can be added using multiple the embedded side an unlimited number of modules can be added using multiple
`PYBIND11_EMBEDDED_MODULE` definitions (as long as they have unique names). ``PYBIND11_EMBEDDED_MODULE`` definitions (as long as they have unique names).
These modules are added to Python's list of builtins, so they can also be These modules are added to Python's list of builtins, so they can also be
imported in pure Python files loaded by the interpreter. Everything interacts imported in pure Python files loaded by the interpreter. Everything interacts
@ -215,9 +216,9 @@ naturally:
Interpreter lifetime Interpreter lifetime
==================== ====================
The Python interpreter shuts down when `scoped_interpreter` is destroyed. After The Python interpreter shuts down when ``scoped_interpreter`` is destroyed. After
this, creating a new instance will restart the interpreter. Alternatively, the this, creating a new instance will restart the interpreter. Alternatively, the
`initialize_interpreter` / `finalize_interpreter` pair of functions can be used ``initialize_interpreter`` / ``finalize_interpreter`` pair of functions can be used
to directly set the state at any time. to directly set the state at any time.
Modules created with pybind11 can be safely re-initialized after the interpreter Modules created with pybind11 can be safely re-initialized after the interpreter
@ -229,8 +230,8 @@ global data. All the details can be found in the CPython documentation.
.. warning:: .. warning::
Creating two concurrent `scoped_interpreter` guards is a fatal error. So is Creating two concurrent ``scoped_interpreter`` guards is a fatal error. So is
calling `initialize_interpreter` for a second time after the interpreter calling ``initialize_interpreter`` for a second time after the interpreter
has already been initialized. has already been initialized.
Do not use the raw CPython API functions ``Py_Initialize`` and Do not use the raw CPython API functions ``Py_Initialize`` and
@ -241,7 +242,7 @@ global data. All the details can be found in the CPython documentation.
Sub-interpreter support Sub-interpreter support
======================= =======================
Creating multiple copies of `scoped_interpreter` is not possible because it Creating multiple copies of ``scoped_interpreter`` is not possible because it
represents the main Python interpreter. Sub-interpreters are something different represents the main Python interpreter. Sub-interpreters are something different
and they do permit the existence of multiple interpreters. This is an advanced and they do permit the existence of multiple interpreters. This is an advanced
feature of the CPython API and should be handled with care. pybind11 does not feature of the CPython API and should be handled with care. pybind11 does not
@ -257,5 +258,5 @@ We'll just mention a couple of caveats the sub-interpreters support in pybind11:
2. Managing multiple threads, multiple interpreters and the GIL can be 2. Managing multiple threads, multiple interpreters and the GIL can be
challenging and there are several caveats here, even within the pure challenging and there are several caveats here, even within the pure
CPython API (please refer to the Python docs for details). As for CPython API (please refer to the Python docs for details). As for
pybind11, keep in mind that `gil_scoped_release` and `gil_scoped_acquire` pybind11, keep in mind that ``gil_scoped_release`` and ``gil_scoped_acquire``
do not take sub-interpreters into account. do not take sub-interpreters into account.

View File

@ -43,18 +43,28 @@ at its exception handler.
| | of bounds access in ``__getitem__``, | | | of bounds access in ``__getitem__``, |
| | ``__setitem__``, etc.) | | | ``__setitem__``, etc.) |
+--------------------------------------+--------------------------------------+ +--------------------------------------+--------------------------------------+
| :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
| | wrong value passed in |
| | ``container.remove(...)``) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::key_error` | ``KeyError`` (used to indicate out | | :class:`pybind11::key_error` | ``KeyError`` (used to indicate out |
| | of bounds access in ``__getitem__``, | | | of bounds access in ``__getitem__``, |
| | ``__setitem__`` in dict-like | | | ``__setitem__`` in dict-like |
| | objects, etc.) | | | objects, etc.) |
+--------------------------------------+--------------------------------------+ +--------------------------------------+--------------------------------------+
| :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
| | wrong value passed in |
| | ``container.remove(...)``) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::type_error` | ``TypeError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::buffer_error` | ``BufferError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::import_error` | ``ImportError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::attribute_error` | ``AttributeError`` |
+--------------------------------------+--------------------------------------+
| Any other exception | ``RuntimeError`` |
+--------------------------------------+--------------------------------------+
Exception translation is not bidirectional. That is, *catching* the C++ Exception translation is not bidirectional. That is, *catching* the C++
exceptions defined above above will not trap exceptions that originate from exceptions defined above will not trap exceptions that originate from
Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
<handling_python_exceptions_cpp>` for further details. <handling_python_exceptions_cpp>` for further details.
@ -67,9 +77,10 @@ Registering custom translators
If the default exception conversion policy described above is insufficient, If the default exception conversion policy described above is insufficient,
pybind11 also provides support for registering custom exception translators. pybind11 also provides support for registering custom exception translators.
To register a simple exception conversion that translates a C++ exception into Similar to pybind11 classes, exception translators can be local to the module
a new Python exception using the C++ exception's ``what()`` method, a helper they are defined in or global to the entire python session. To register a simple
function is available: exception conversion that translates a C++ exception into a new Python exception
using the C++ exception's ``what()`` method, a helper function is available:
.. code-block:: cpp .. code-block:: cpp
@ -79,35 +90,44 @@ This call creates a Python exception class with the name ``PyExp`` in the given
module and automatically converts any encountered exceptions of type ``CppExp`` module and automatically converts any encountered exceptions of type ``CppExp``
into Python exceptions of type ``PyExp``. into Python exceptions of type ``PyExp``.
A matching function is available for registering a local exception translator:
.. code-block:: cpp
py::register_local_exception<CppExp>(module, "PyExp");
It is possible to specify base class for the exception using the third It is possible to specify base class for the exception using the third
parameter, a `handle`: parameter, a ``handle``:
.. code-block:: cpp .. code-block:: cpp
py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError); py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
py::register_local_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
Then `PyExp` can be caught both as `PyExp` and `RuntimeError`. Then ``PyExp`` can be caught both as ``PyExp`` and ``RuntimeError``.
The class objects of the built-in Python exceptions are listed in the Python The class objects of the built-in Python exceptions are listed in the Python
documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_. documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_.
The default base class is `PyExc_Exception`. The default base class is ``PyExc_Exception``.
When more advanced exception translation is needed, the function When more advanced exception translation is needed, the functions
``py::register_exception_translator(translator)`` can be used to register ``py::register_exception_translator(translator)`` and
``py::register_local_exception_translator(translator)`` can be used to register
functions that can translate arbitrary exception types (and which may include functions that can translate arbitrary exception types (and which may include
additional logic to do so). The function takes a stateless callable (e.g. a additional logic to do so). The functions takes a stateless callable (e.g. a
function pointer or a lambda function without captured variables) with the call function pointer or a lambda function without captured variables) with the call
signature ``void(std::exception_ptr)``. signature ``void(std::exception_ptr)``.
When a C++ exception is thrown, the registered exception translators are tried When a C++ exception is thrown, the registered exception translators are tried
in reverse order of registration (i.e. the last registered translator gets the in reverse order of registration (i.e. the last registered translator gets the
first shot at handling the exception). first shot at handling the exception). All local translators will be tried
before a global translator is tried.
Inside the translator, ``std::rethrow_exception`` should be used within Inside the translator, ``std::rethrow_exception`` should be used within
a try block to re-throw the exception. One or more catch clauses to catch a try block to re-throw the exception. One or more catch clauses to catch
the appropriate exceptions should then be used with each clause using the appropriate exceptions should then be used with each clause using
``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set ``py::set_error()`` (see below).
the python exception to a custom exception type (see below).
To declare a custom Python exception type, declare a ``py::exception`` variable To declare a custom Python exception type, declare a ``py::exception`` variable
and use this in the associated exception translator (note: it is often useful and use this in the associated exception translator (note: it is often useful
@ -121,14 +141,16 @@ standard python RuntimeError:
.. code-block:: cpp .. code-block:: cpp
static py::exception<MyCustomException> exc(m, "MyCustomError"); PYBIND11_CONSTINIT static py::gil_safe_call_once_and_store<py::object> exc_storage;
exc_storage.call_once_and_store_result(
[&]() { return py::exception<MyCustomException>(m, "MyCustomError"); });
py::register_exception_translator([](std::exception_ptr p) { py::register_exception_translator([](std::exception_ptr p) {
try { try {
if (p) std::rethrow_exception(p); if (p) std::rethrow_exception(p);
} catch (const MyCustomException &e) { } catch (const MyCustomException &e) {
exc(e.what()); py::set_error(exc_storage.get_stored(), e.what());
} catch (const OtherException &e) { } catch (const OtherException &e) {
PyErr_SetString(PyExc_RuntimeError, e.what()); py::set_error(PyExc_RuntimeError, e.what());
} }
}); });
@ -147,8 +169,7 @@ section.
.. note:: .. note::
Call either ``PyErr_SetString`` or a custom exception's call Call ``py::set_error()`` for every exception caught in a custom exception
operator (``exc(string)``) for every exception caught in a custom exception
translator. Failure to do so will cause Python to crash with ``SystemError: translator. Failure to do so will cause Python to crash with ``SystemError:
error return without exception set``. error return without exception set``.
@ -156,6 +177,60 @@ section.
may be explicitly (re-)thrown to delegate it to the other, may be explicitly (re-)thrown to delegate it to the other,
previously-declared existing exception translators. previously-declared existing exception translators.
Note that ``libc++`` and ``libstdc++`` `behave differently under macOS
<https://stackoverflow.com/questions/19496643/using-clang-fvisibility-hidden-and-typeinfo-and-type-erasure/28827430>`_
with ``-fvisibility=hidden``. Therefore exceptions that are used across ABI
boundaries need to be explicitly exported, as exercised in
``tests/test_exceptions.h``. See also:
"Problems with C++ exceptions" under `GCC Wiki <https://gcc.gnu.org/wiki/Visibility>`_.
Local vs Global Exception Translators
=====================================
When a global exception translator is registered, it will be applied across all
modules in the reverse order of registration. This can create behavior where the
order of module import influences how exceptions are translated.
If module1 has the following translator:
.. code-block:: cpp
py::register_exception_translator([](std::exception_ptr p) {
try {
if (p) std::rethrow_exception(p);
} catch (const std::invalid_argument &e) {
py::set_error(PyExc_ArgumentError, "module1 handled this");
}
}
and module2 has the following similar translator:
.. code-block:: cpp
py::register_exception_translator([](std::exception_ptr p) {
try {
if (p) std::rethrow_exception(p);
} catch (const std::invalid_argument &e) {
py::set_error(PyExc_ArgumentError, "module2 handled this");
}
}
then which translator handles the invalid_argument will be determined by the
order that module1 and module2 are imported. Since exception translators are
applied in the reverse order of registration, which ever module was imported
last will "win" and that translator will be applied.
If there are multiple pybind11 modules that share exception types (either
standard built-in or custom) loaded into a single python instance and
consistent error handling behavior is needed, then local translators should be
used.
Changing the previous example to use ``register_local_exception_translator``
would mean that when invalid_argument is thrown in the module2 code, the
module2 translator will always handle it, while in module1, the module1
translator will do the same.
.. _handling_python_exceptions_cpp: .. _handling_python_exceptions_cpp:
Handling exceptions from Python in C++ Handling exceptions from Python in C++
@ -188,7 +263,7 @@ For example:
} catch (py::error_already_set &e) { } catch (py::error_already_set &e) {
if (e.matches(PyExc_FileNotFoundError)) { if (e.matches(PyExc_FileNotFoundError)) {
py::print("missing.txt not found"); py::print("missing.txt not found");
} else if (e.match(PyExc_PermissionError)) { } else if (e.matches(PyExc_PermissionError)) {
py::print("missing.txt found but not accessible"); py::print("missing.txt found but not accessible");
} else { } else {
throw; throw;
@ -237,11 +312,11 @@ error protocol, which is outlined here.
After calling the Python C API, if Python returns an error, After calling the Python C API, if Python returns an error,
``throw py::error_already_set();``, which allows pybind11 to deal with the ``throw py::error_already_set();``, which allows pybind11 to deal with the
exception and pass it back to the Python interpreter. This includes calls to exception and pass it back to the Python interpreter. This includes calls to
the error setting functions such as ``PyErr_SetString``. the error setting functions such as ``py::set_error()``.
.. code-block:: cpp .. code-block:: cpp
PyErr_SetString(PyExc_TypeError, "C API type error demo"); py::set_error(PyExc_TypeError, "C API type error demo");
throw py::error_already_set(); throw py::error_already_set();
// But it would be easier to simply... // But it would be easier to simply...
@ -253,6 +328,34 @@ Alternately, to ignore the error, call `PyErr_Clear
Any Python error must be thrown or cleared, or Python/pybind11 will be left in Any Python error must be thrown or cleared, or Python/pybind11 will be left in
an invalid state. an invalid state.
Chaining exceptions ('raise from')
==================================
Python has a mechanism for indicating that exceptions were caused by other
exceptions:
.. code-block:: py
try:
print(1 / 0)
except Exception as exc:
raise RuntimeError("could not divide by zero") from exc
To do a similar thing in pybind11, you can use the ``py::raise_from`` function. It
sets the current python error indicator, so to continue propagating the exception
you should ``throw py::error_already_set()``.
.. code-block:: cpp
try {
py::eval("print(1 / 0"));
} catch (py::error_already_set &e) {
py::raise_from(e, PyExc_RuntimeError, "could not divide by zero");
throw py::error_already_set();
}
.. versionadded:: 2.8
.. _unraisable_exceptions: .. _unraisable_exceptions:
Handling unraisable exceptions Handling unraisable exceptions

View File

@ -16,7 +16,7 @@ lifetime of objects managed by them. This can lead to issues when creating
bindings for functions that return a non-trivial type. Just by looking at the bindings for functions that return a non-trivial type. Just by looking at the
type information, it is not clear whether Python should take charge of the type information, it is not clear whether Python should take charge of the
returned value and eventually free its resources, or if this is handled on the returned value and eventually free its resources, or if this is handled on the
C++ side. For this reason, pybind11 provides a several *return value policy* C++ side. For this reason, pybind11 provides several *return value policy*
annotations that can be passed to the :func:`module_::def` and annotations that can be passed to the :func:`module_::def` and
:func:`class_::def` functions. The default policy is :func:`class_::def` functions. The default policy is
:enum:`return_value_policy::automatic`. :enum:`return_value_policy::automatic`.
@ -50,7 +50,7 @@ implied transfer of ownership, i.e.:
.. code-block:: cpp .. code-block:: cpp
m.def("get_data", &get_data, return_value_policy::reference); m.def("get_data", &get_data, py::return_value_policy::reference);
On the other hand, this is not the right policy for many other situations, On the other hand, this is not the right policy for many other situations,
where ignoring ownership could lead to resource leaks. where ignoring ownership could lead to resource leaks.
@ -90,17 +90,18 @@ The following table provides an overview of available policies:
| | return value is referenced by Python. This is the default policy for | | | return value is referenced by Python. This is the default policy for |
| | property getters created via ``def_property``, ``def_readwrite``, etc. | | | property getters created via ``def_property``, ``def_readwrite``, etc. |
+--------------------------------------------------+----------------------------------------------------------------------------+ +--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy | | :enum:`return_value_policy::automatic` | This policy falls back to the policy |
| | :enum:`return_value_policy::take_ownership` when the return value is a | | | :enum:`return_value_policy::take_ownership` when the return value is a |
| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or | | | pointer. Otherwise, it uses :enum:`return_value_policy::move` or |
| | :enum:`return_value_policy::copy` for rvalue and lvalue references, | | | :enum:`return_value_policy::copy` for rvalue and lvalue references, |
| | respectively. See above for a description of what all of these different | | | respectively. See above for a description of what all of these different |
| | policies do. | | | policies do. This is the default policy for ``py::class_``-wrapped types. |
+--------------------------------------------------+----------------------------------------------------------------------------+ +--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the | | :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
| | return value is a pointer. This is the default conversion policy for | | | return value is a pointer. This is the default conversion policy for |
| | function arguments when calling Python functions manually from C++ code | | | function arguments when calling Python functions manually from C++ code |
| | (i.e. via handle::operator()). You probably won't need to use this. | | | (i.e. via ``handle::operator()``) and the casters in ``pybind11/stl.h``. |
| | You probably won't need to use this explicitly. |
+--------------------------------------------------+----------------------------------------------------------------------------+ +--------------------------------------------------+----------------------------------------------------------------------------+
Return value policies can also be applied to properties: Return value policies can also be applied to properties:
@ -119,7 +120,7 @@ targeted arguments can be passed through the :class:`cpp_function` constructor:
.. code-block:: cpp .. code-block:: cpp
class_<MyClass>(m, "MyClass") class_<MyClass>(m, "MyClass")
.def_property("data" .def_property("data",
py::cpp_function(&MyClass::getData, py::return_value_policy::copy), py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
py::cpp_function(&MyClass::setData) py::cpp_function(&MyClass::setData)
); );
@ -182,6 +183,9 @@ relies on the ability to create a *weak reference* to the nurse object. When
the nurse object is not a pybind11-registered type and does not support weak the nurse object is not a pybind11-registered type and does not support weak
references, an exception will be thrown. references, an exception will be thrown.
If you use an incorrect argument index, you will get a ``RuntimeError`` saying
``Could not activate keep_alive!``. You should review the indices you're using.
Consider the following example: here, the binding code for a list append Consider the following example: here, the binding code for a list append
operation ties the lifetime of the newly added element to the underlying operation ties the lifetime of the newly added element to the underlying
container: container:
@ -228,7 +232,7 @@ is equivalent to the following pseudocode:
}); });
The only requirement is that ``T`` is default-constructible, but otherwise any The only requirement is that ``T`` is default-constructible, but otherwise any
scope guard will work. This is very useful in combination with `gil_scoped_release`. scope guard will work. This is very useful in combination with ``gil_scoped_release``.
See :ref:`gil`. See :ref:`gil`.
Multiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The Multiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The
@ -251,7 +255,7 @@ For instance, the following statement iterates over a Python ``dict``:
.. code-block:: cpp .. code-block:: cpp
void print_dict(py::dict dict) { void print_dict(const py::dict& dict) {
/* Easily interact with Python types */ /* Easily interact with Python types */
for (auto item : dict) for (auto item : dict)
std::cout << "key=" << std::string(py::str(item.first)) << ", " std::cout << "key=" << std::string(py::str(item.first)) << ", "
@ -268,7 +272,7 @@ And used in Python as usual:
.. code-block:: pycon .. code-block:: pycon
>>> print_dict({'foo': 123, 'bar': 'hello'}) >>> print_dict({"foo": 123, "bar": "hello"})
key=foo, value=123 key=foo, value=123
key=bar, value=hello key=bar, value=hello
@ -289,7 +293,7 @@ Such functions can also be created using pybind11:
.. code-block:: cpp .. code-block:: cpp
void generic(py::args args, py::kwargs kwargs) { void generic(py::args args, const py::kwargs& kwargs) {
/// .. do something with args /// .. do something with args
if (kwargs) if (kwargs)
/// .. do something with kwargs /// .. do something with kwargs
@ -302,8 +306,9 @@ The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives
from ``py::dict``. from ``py::dict``.
You may also use just one or the other, and may combine these with other You may also use just one or the other, and may combine these with other
arguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last arguments. Note, however, that ``py::kwargs`` must always be the last argument
arguments accepted by the function. of the function, and ``py::args`` implies that any further arguments are
keyword-only (see :ref:`keyword_only_arguments`).
Please refer to the other examples for details on how to iterate over these, Please refer to the other examples for details on how to iterate over these,
and on how to cast their entries into C++ objects. A demonstration is also and on how to cast their entries into C++ objects. A demonstration is also
@ -362,10 +367,12 @@ like so:
py::class_<MyClass>("MyClass") py::class_<MyClass>("MyClass")
.def("myFunction", py::arg("arg") = static_cast<SomeType *>(nullptr)); .def("myFunction", py::arg("arg") = static_cast<SomeType *>(nullptr));
.. _keyword_only_arguments:
Keyword-only arguments Keyword-only arguments
====================== ======================
Python 3 introduced keyword-only arguments by specifying an unnamed ``*`` Python implements keyword-only arguments by specifying an unnamed ``*``
argument in a function definition: argument in a function definition:
.. code-block:: python .. code-block:: python
@ -373,10 +380,11 @@ argument in a function definition:
def f(a, *, b): # a can be positional or via keyword; b must be via keyword def f(a, *, b): # a can be positional or via keyword; b must be via keyword
pass pass
f(a=1, b=2) # good f(a=1, b=2) # good
f(b=2, a=1) # good f(b=2, a=1) # good
f(1, b=2) # good f(1, b=2) # good
f(1, 2) # TypeError: f() takes 1 positional argument but 2 were given f(1, 2) # TypeError: f() takes 1 positional argument but 2 were given
Pybind11 provides a ``py::kw_only`` object that allows you to implement Pybind11 provides a ``py::kw_only`` object that allows you to implement
the same behaviour by specifying the object between positional and keyword-only the same behaviour by specifying the object between positional and keyword-only
@ -387,11 +395,19 @@ argument annotations when registering the function:
m.def("f", [](int a, int b) { /* ... */ }, m.def("f", [](int a, int b) { /* ... */ },
py::arg("a"), py::kw_only(), py::arg("b")); py::arg("a"), py::kw_only(), py::arg("b"));
Note that you currently cannot combine this with a ``py::args`` argument. This
feature does *not* require Python 3 to work.
.. versionadded:: 2.6 .. versionadded:: 2.6
A ``py::args`` argument implies that any following arguments are keyword-only,
as if ``py::kw_only()`` had been specified in the same relative location of the
argument list as the ``py::args`` argument. The ``py::kw_only()`` may be
included to be explicit about this, but is not required.
.. versionchanged:: 2.9
This can now be combined with ``py::args``. Before, ``py::args`` could only
occur at the end of the argument list, or immediately before a ``py::kwargs``
argument at the end.
Positional-only arguments Positional-only arguments
========================= =========================
@ -524,6 +540,8 @@ The default behaviour when the tag is unspecified is to allow ``None``.
not allow ``None`` as argument. To pass optional argument of these copied types consider not allow ``None`` as argument. To pass optional argument of these copied types consider
using ``std::optional<T>`` using ``std::optional<T>``
.. _overload_resolution:
Overload resolution order Overload resolution order
========================= =========================
@ -559,3 +577,38 @@ prefers earlier-defined overloads to later-defined ones.
.. versionadded:: 2.6 .. versionadded:: 2.6
The ``py::prepend()`` tag. The ``py::prepend()`` tag.
Binding functions with template parameters
==========================================
You can bind functions that have template parameters. Here's a function:
.. code-block:: cpp
template <typename T>
void set(T t);
C++ templates cannot be instantiated at runtime, so you cannot bind the
non-instantiated function:
.. code-block:: cpp
// BROKEN (this will not compile)
m.def("set", &set);
You must bind each instantiated function template separately. You may bind
each instantiation with the same name, which will be treated the same as
an overloaded function:
.. code-block:: cpp
m.def("set", &set<int>);
m.def("set", &set<std::string>);
Sometimes it's more clear to bind them with separate names, which is also
an option:
.. code-block:: cpp
m.def("setInt", &set<int>);
m.def("setString", &set<std::string>);

View File

@ -39,15 +39,42 @@ The ``PYBIND11_MAKE_OPAQUE`` macro does *not* require the above workarounds.
Global Interpreter Lock (GIL) Global Interpreter Lock (GIL)
============================= =============================
When calling a C++ function from Python, the GIL is always held. The Python C API dictates that the Global Interpreter Lock (GIL) must always
be held by the current thread to safely access Python objects. As a result,
when Python calls into C++ via pybind11 the GIL must be held, and pybind11
will never implicitly release the GIL.
.. code-block:: cpp
void my_function() {
/* GIL is held when this function is called from Python */
}
PYBIND11_MODULE(example, m) {
m.def("my_function", &my_function);
}
pybind11 will ensure that the GIL is held when it knows that it is calling
Python code. For example, if a Python callback is passed to C++ code via
``std::function``, when C++ code calls the function the built-in wrapper
will acquire the GIL before calling the Python callback. Similarly, the
``PYBIND11_OVERRIDE`` family of macros will acquire the GIL before calling
back into Python.
When writing C++ code that is called from other C++ code, if that code accesses
Python state, it must explicitly acquire and release the GIL.
The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be
used to acquire and release the global interpreter lock in the body of a C++ used to acquire and release the global interpreter lock in the body of a C++
function call. In this way, long-running C++ code can be parallelized using function call. In this way, long-running C++ code can be parallelized using
multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this multiple Python threads, **but great care must be taken** when any
:class:`gil_scoped_release` appear: if there is any way that the C++ code
can access Python objects, :class:`gil_scoped_acquire` should be used to
reacquire the GIL. Taking :ref:`overriding_virtuals` as an example, this
could be realized as follows (important changes highlighted): could be realized as follows (important changes highlighted):
.. code-block:: cpp .. code-block:: cpp
:emphasize-lines: 8,9,31,32 :emphasize-lines: 8,30,31
class PyAnimal : public Animal { class PyAnimal : public Animal {
public: public:
@ -56,9 +83,7 @@ could be realized as follows (important changes highlighted):
/* Trampoline (need one for each virtual function) */ /* Trampoline (need one for each virtual function) */
std::string go(int n_times) { std::string go(int n_times) {
/* Acquire GIL before calling Python code */ /* PYBIND11_OVERRIDE_PURE will acquire the GIL before accessing Python state */
py::gil_scoped_acquire acquire;
PYBIND11_OVERRIDE_PURE( PYBIND11_OVERRIDE_PURE(
std::string, /* Return type */ std::string, /* Return type */
Animal, /* Parent class */ Animal, /* Parent class */
@ -78,13 +103,14 @@ could be realized as follows (important changes highlighted):
.def(py::init<>()); .def(py::init<>());
m.def("call_go", [](Animal *animal) -> std::string { m.def("call_go", [](Animal *animal) -> std::string {
/* Release GIL before calling into (potentially long-running) C++ code */ // GIL is held when called from Python code. Release GIL before
// calling into (potentially long-running) C++ code
py::gil_scoped_release release; py::gil_scoped_release release;
return call_go(animal); return call_go(animal);
}); });
} }
The ``call_go`` wrapper can also be simplified using the `call_guard` policy The ``call_go`` wrapper can also be simplified using the ``call_guard`` policy
(see :ref:`call_policies`) which yields the same result: (see :ref:`call_policies`) which yields the same result:
.. code-block:: cpp .. code-block:: cpp
@ -92,6 +118,34 @@ The ``call_go`` wrapper can also be simplified using the `call_guard` policy
m.def("call_go", &call_go, py::call_guard<py::gil_scoped_release>()); m.def("call_go", &call_go, py::call_guard<py::gil_scoped_release>());
Common Sources Of Global Interpreter Lock Errors
==================================================================
Failing to properly hold the Global Interpreter Lock (GIL) is one of the
more common sources of bugs within code that uses pybind11. If you are
running into GIL related errors, we highly recommend you consult the
following checklist.
- Do you have any global variables that are pybind11 objects or invoke
pybind11 functions in either their constructor or destructor? You are generally
not allowed to invoke any Python function in a global static context. We recommend
using lazy initialization and then intentionally leaking at the end of the program.
- Do you have any pybind11 objects that are members of other C++ structures? One
commonly overlooked requirement is that pybind11 objects have to increase their reference count
whenever their copy constructor is called. Thus, you need to be holding the GIL to invoke
the copy constructor of any C++ class that has a pybind11 member. This can sometimes be very
tricky to track for complicated programs Think carefully when you make a pybind11 object
a member in another struct.
- C++ destructors that invoke Python functions can be particularly troublesome as
destructors can sometimes get invoked in weird and unexpected circumstances as a result
of exceptions.
- You should try running your code in a debug build. That will enable additional assertions
within pybind11 that will throw exceptions on certain GIL handling errors
(reference counting operations).
Binding sequence data types, iterators, the slicing protocol, etc. Binding sequence data types, iterators, the slicing protocol, etc.
================================================================== ==================================================================
@ -298,6 +352,15 @@ The class ``options`` allows you to selectively suppress auto-generated signatur
m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers");
} }
pybind11 also appends all members of an enum to the resulting enum docstring.
This default behavior can be disabled by using the ``disable_enum_members_docstring()``
function of the ``options`` class.
With ``disable_user_defined_docstrings()`` all user defined docstrings of
``module_::def()``, ``class_::def()`` and ``enum_()`` are disabled, but the
function signatures and enum members are included in the docstring, unless they
are disabled separately.
Note that changes to the settings affect only function bindings created during the Note that changes to the settings affect only function bindings created during the
lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function, lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function,
the default settings are restored to prevent unwanted side effects. the default settings are restored to prevent unwanted side effects.
@ -335,3 +398,32 @@ before they are used as a parameter or return type of a function:
pyFoo.def(py::init<const ns::Bar&>()); pyFoo.def(py::init<const ns::Bar&>());
pyBar.def(py::init<const ns::Foo&>()); pyBar.def(py::init<const ns::Foo&>());
} }
Setting inner type hints in docstrings
======================================
When you use pybind11 wrappers for ``list``, ``dict``, and other generic python
types, the docstring will just display the generic type. You can convey the
inner types in the docstring by using a special 'typed' version of the generic
type.
.. code-block:: cpp
PYBIND11_MODULE(example, m) {
m.def("pass_list_of_str", [](py::typing::List<py::str> arg) {
// arg can be used just like py::list
));
}
The resulting docstring will be ``pass_list_of_str(arg0: list[str]) -> None``.
The following special types are available in ``pybind11/typing.h``:
* ``py::Tuple<Args...>``
* ``py::Dict<K, V>``
* ``py::List<V>``
* ``py::Set<V>``
* ``py::Callable<Signature>``
.. warning:: Just like in python, these are merely hints. They don't actually
enforce the types of their contents at runtime or compile time.

View File

@ -87,7 +87,7 @@ buffer objects (e.g. a NumPy matrix).
/* Request a buffer descriptor from Python */ /* Request a buffer descriptor from Python */
py::buffer_info info = b.request(); py::buffer_info info = b.request();
/* Some sanity checks ... */ /* Some basic validation checks ... */
if (info.format != py::format_descriptor<Scalar>::format()) if (info.format != py::format_descriptor<Scalar>::format())
throw std::runtime_error("Incompatible format: expected a double array!"); throw std::runtime_error("Incompatible format: expected a double array!");
@ -150,8 +150,10 @@ NumPy array containing double precision values.
When it is invoked with a different type (e.g. an integer or a list of When it is invoked with a different type (e.g. an integer or a list of
integers), the binding code will attempt to cast the input into a NumPy array integers), the binding code will attempt to cast the input into a NumPy array
of the requested type. Note that this feature requires the of the requested type. This feature requires the :file:`pybind11/numpy.h`
:file:`pybind11/numpy.h` header to be included. header to be included. Note that :file:`pybind11/numpy.h` does not depend on
the NumPy headers, and thus can be used without declaring a build-time
dependency on NumPy; NumPy>=1.7.0 is a runtime dependency.
Data in NumPy arrays is not guaranteed to packed in a dense manner; Data in NumPy arrays is not guaranteed to packed in a dense manner;
furthermore, entries can be separated by arbitrary column and row strides. furthermore, entries can be separated by arbitrary column and row strides.
@ -169,6 +171,31 @@ template parameter, and it ensures that non-conforming arguments are converted
into an array satisfying the specified requirements instead of trying the next into an array satisfying the specified requirements instead of trying the next
function overload. function overload.
There are several methods on arrays; the methods listed below under references
work, as well as the following functions based on the NumPy API:
- ``.dtype()`` returns the type of the contained values.
- ``.strides()`` returns a pointer to the strides of the array (optionally pass
an integer axis to get a number).
- ``.flags()`` returns the flag settings. ``.writable()`` and ``.owndata()``
are directly available.
- ``.offset_at()`` returns the offset (optionally pass indices).
- ``.squeeze()`` returns a view with length-1 axes removed.
- ``.view(dtype)`` returns a view of the array with a different dtype.
- ``.reshape({i, j, ...})`` returns a view of the array with a different shape.
``.resize({...})`` is also available.
- ``.index_at(i, j, ...)`` gets the count from the beginning to a given index.
There are also several methods for getting references (described below).
Structured types Structured types
================ ================
@ -231,8 +258,8 @@ by the compiler. The result is returned as a NumPy array of type
.. code-block:: pycon .. code-block:: pycon
>>> x = np.array([[1, 3],[5, 7]]) >>> x = np.array([[1, 3], [5, 7]])
>>> y = np.array([[2, 4],[6, 8]]) >>> y = np.array([[2, 4], [6, 8]])
>>> z = 3 >>> z = 3
>>> result = vectorized_func(x, y, z) >>> result = vectorized_func(x, y, z)
@ -343,21 +370,21 @@ The returned proxy object supports some of the same methods as ``py::array`` so
that it can be used as a drop-in replacement for some existing, index-checked that it can be used as a drop-in replacement for some existing, index-checked
uses of ``py::array``: uses of ``py::array``:
- ``r.ndim()`` returns the number of dimensions - ``.ndim()`` returns the number of dimensions
- ``r.data(1, 2, ...)`` and ``r.mutable_data(1, 2, ...)``` returns a pointer to - ``.data(1, 2, ...)`` and ``r.mutable_data(1, 2, ...)``` returns a pointer to
the ``const T`` or ``T`` data, respectively, at the given indices. The the ``const T`` or ``T`` data, respectively, at the given indices. The
latter is only available to proxies obtained via ``a.mutable_unchecked()``. latter is only available to proxies obtained via ``a.mutable_unchecked()``.
- ``itemsize()`` returns the size of an item in bytes, i.e. ``sizeof(T)``. - ``.itemsize()`` returns the size of an item in bytes, i.e. ``sizeof(T)``.
- ``ndim()`` returns the number of dimensions. - ``.ndim()`` returns the number of dimensions.
- ``shape(n)`` returns the size of dimension ``n`` - ``.shape(n)`` returns the size of dimension ``n``
- ``size()`` returns the total number of elements (i.e. the product of the shapes). - ``.size()`` returns the total number of elements (i.e. the product of the shapes).
- ``nbytes()`` returns the number of bytes used by the referenced elements - ``.nbytes()`` returns the number of bytes used by the referenced elements
(i.e. ``itemsize()`` times ``size()``). (i.e. ``itemsize()`` times ``size()``).
.. seealso:: .. seealso::
@ -368,15 +395,13 @@ uses of ``py::array``:
Ellipsis Ellipsis
======== ========
Python 3 provides a convenient ``...`` ellipsis notation that is often used to Python provides a convenient ``...`` ellipsis notation that is often used to
slice multidimensional arrays. For instance, the following snippet extracts the slice multidimensional arrays. For instance, the following snippet extracts the
middle dimensions of a tensor with the first and last index set to zero. middle dimensions of a tensor with the first and last index set to zero.
In Python 2, the syntactic sugar ``...`` is not available, but the singleton
``Ellipsis`` (of type ``ellipsis``) can still be used directly.
.. code-block:: python .. code-block:: python
a = # a NumPy array a = ... # a NumPy array
b = a[0, ..., 0] b = a[0, ..., 0]
The function ``py::ellipsis()`` function can be used to perform the same The function ``py::ellipsis()`` function can be used to perform the same
@ -387,8 +412,6 @@ operation on the C++ side:
py::array a = /* A NumPy array */; py::array a = /* A NumPy array */;
py::array b = a[py::make_tuple(0, py::ellipsis(), 0)]; py::array b = a[py::make_tuple(0, py::ellipsis(), 0)];
.. versionchanged:: 2.6
``py::ellipsis()`` is now also avaliable in Python 2.
Memory view Memory view
=========== ===========
@ -410,7 +433,7 @@ following:
{ 2, 4 }, // shape (rows, cols) { 2, 4 }, // shape (rows, cols)
{ sizeof(uint8_t) * 4, sizeof(uint8_t) } // strides in bytes { sizeof(uint8_t) * 4, sizeof(uint8_t) } // strides in bytes
); );
}) });
This approach is meant for providing a ``memoryview`` for a C/C++ buffer not This approach is meant for providing a ``memoryview`` for a C/C++ buffer not
managed by Python. The user is responsible for managing the lifetime of the managed by Python. The user is responsible for managing the lifetime of the
@ -426,11 +449,7 @@ We can also use ``memoryview::from_memory`` for a simple 1D contiguous buffer:
buffer, // buffer pointer buffer, // buffer pointer
sizeof(uint8_t) * 8 // buffer size sizeof(uint8_t) * 8 // buffer size
); );
}) });
.. note::
``memoryview::from_memory`` is not available in Python 2.
.. versionchanged:: 2.6 .. versionchanged:: 2.6
``memoryview::from_memory`` added. ``memoryview::from_memory`` added.

View File

@ -20,6 +20,40 @@ Available types include :class:`handle`, :class:`object`, :class:`bool_`,
Be sure to review the :ref:`pytypes_gotchas` before using this heavily in Be sure to review the :ref:`pytypes_gotchas` before using this heavily in
your C++ API. your C++ API.
.. _instantiating_compound_types:
Instantiating compound Python types from C++
============================================
Dictionaries can be initialized in the :class:`dict` constructor:
.. code-block:: cpp
using namespace pybind11::literals; // to bring in the `_a` literal
py::dict d("spam"_a=py::none(), "eggs"_a=42);
A tuple of python objects can be instantiated using :func:`py::make_tuple`:
.. code-block:: cpp
py::tuple tup = py::make_tuple(42, py::none(), "spam");
Each element is converted to a supported Python type.
A `simple namespace`_ can be instantiated using
.. code-block:: cpp
using namespace pybind11::literals; // to bring in the `_a` literal
py::object SimpleNamespace = py::module_::import("types").attr("SimpleNamespace");
py::object ns = SimpleNamespace("spam"_a=py::none(), "eggs"_a=42);
Attributes on a namespace can be modified with the :func:`py::delattr`,
:func:`py::getattr`, and :func:`py::setattr` functions. Simple namespaces can
be useful as lightweight stand-ins for class instances.
.. _simple namespace: https://docs.python.org/3/library/types.html#types.SimpleNamespace
.. _casting_back_and_forth: .. _casting_back_and_forth:
Casting back and forth Casting back and forth
@ -30,7 +64,7 @@ types to Python, which can be done using :func:`py::cast`:
.. code-block:: cpp .. code-block:: cpp
MyClass *cls = ..; MyClass *cls = ...;
py::object obj = py::cast(cls); py::object obj = py::cast(cls);
The reverse direction uses the following syntax: The reverse direction uses the following syntax:
@ -132,6 +166,7 @@ Keyword arguments are also supported. In Python, there is the usual call syntax:
def f(number, say, to): def f(number, say, to):
... # function code ... # function code
f(1234, say="hello", to=some_instance) # keyword call in Python f(1234, say="hello", to=some_instance) # keyword call in Python
In C++, the same call can be made using: In C++, the same call can be made using:

View File

@ -28,7 +28,7 @@ Capturing standard output from ostream
Often, a library will use the streams ``std::cout`` and ``std::cerr`` to print, Often, a library will use the streams ``std::cout`` and ``std::cerr`` to print,
but this does not play well with Python's standard ``sys.stdout`` and ``sys.stderr`` but this does not play well with Python's standard ``sys.stdout`` and ``sys.stderr``
redirection. Replacing a library's printing with `py::print <print>` may not redirection. Replacing a library's printing with ``py::print <print>`` may not
be feasible. This can be fixed using a guard around the library function that be feasible. This can be fixed using a guard around the library function that
redirects output to the corresponding Python streams: redirects output to the corresponding Python streams:
@ -47,15 +47,26 @@ redirects output to the corresponding Python streams:
call_noisy_func(); call_noisy_func();
}); });
.. warning::
The implementation in ``pybind11/iostream.h`` is NOT thread safe. Multiple
threads writing to a redirected ostream concurrently cause data races
and potentially buffer overflows. Therefore it is currently a requirement
that all (possibly) concurrent redirected ostream writes are protected by
a mutex. #HelpAppreciated: Work on iostream.h thread safety. For more
background see the discussions under
`PR #2982 <https://github.com/pybind/pybind11/pull/2982>`_ and
`PR #2995 <https://github.com/pybind/pybind11/pull/2995>`_.
This method respects flushes on the output streams and will flush if needed This method respects flushes on the output streams and will flush if needed
when the scoped guard is destroyed. This allows the output to be redirected in when the scoped guard is destroyed. This allows the output to be redirected in
real time, such as to a Jupyter notebook. The two arguments, the C++ stream and real time, such as to a Jupyter notebook. The two arguments, the C++ stream and
the Python output, are optional, and default to standard output if not given. An the Python output, are optional, and default to standard output if not given. An
extra type, `py::scoped_estream_redirect <scoped_estream_redirect>`, is identical extra type, ``py::scoped_estream_redirect <scoped_estream_redirect>``, is identical
except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful with except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful with
`py::call_guard`, which allows multiple items, but uses the default constructor: ``py::call_guard``, which allows multiple items, but uses the default constructor:
.. code-block:: py .. code-block:: cpp
// Alternative: Call single function using call guard // Alternative: Call single function using call guard
m.def("noisy_func", &call_noisy_function, m.def("noisy_func", &call_noisy_function,
@ -63,7 +74,7 @@ except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful wi
py::scoped_estream_redirect>()); py::scoped_estream_redirect>());
The redirection can also be done in Python with the addition of a context The redirection can also be done in Python with the addition of a context
manager, using the `py::add_ostream_redirect() <add_ostream_redirect>` function: manager, using the ``py::add_ostream_redirect() <add_ostream_redirect>`` function:
.. code-block:: cpp .. code-block:: cpp
@ -92,7 +103,7 @@ arguments to disable one of the streams if needed.
Evaluating Python expressions from strings and files Evaluating Python expressions from strings and files
==================================================== ====================================================
pybind11 provides the `eval`, `exec` and `eval_file` functions to evaluate pybind11 provides the ``eval``, ``exec`` and ``eval_file`` functions to evaluate
Python expressions and statements. The following example illustrates how they Python expressions and statements. The following example illustrates how they
can be used. can be used.

View File

@ -77,6 +77,7 @@ segmentation fault).
.. code-block:: python .. code-block:: python
from example import Parent from example import Parent
print(Parent().get_child()) print(Parent().get_child())
The problem is that ``Parent::get_child()`` returns a pointer to an instance of The problem is that ``Parent::get_child()`` returns a pointer to an instance of
@ -156,7 +157,7 @@ specialized:
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>); PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>);
// Only needed if the type's `.get()` goes by another name // Only needed if the type's `.get()` goes by another name
namespace pybind11 { namespace detail { namespace PYBIND11_NAMESPACE { namespace detail {
template <typename T> template <typename T>
struct holder_helper<SmartPtr<T>> { // <-- specialization struct holder_helper<SmartPtr<T>> { // <-- specialization
static const T *get(const SmartPtr<T> &p) { return p.getPointer(); } static const T *get(const SmartPtr<T> &p) { return p.getPointer(); }

View File

@ -32,8 +32,7 @@ The last line will both compile and run the tests.
Windows Windows
------- -------
On Windows, only **Visual Studio 2015** and newer are supported since pybind11 relies On Windows, only **Visual Studio 2017** and newer are supported.
on various C++11 language features that break older versions of Visual Studio.
.. Note:: .. Note::
@ -109,7 +108,7 @@ a file named :file:`example.cpp` with the following contents:
PYBIND11_MODULE(example, m) { PYBIND11_MODULE(example, m) {
m.doc() = "pybind11 example plugin"; // optional module docstring m.doc() = "pybind11 example plugin"; // optional module docstring
m.def("add", &add, "A function which adds two numbers"); m.def("add", &add, "A function that adds two numbers");
} }
.. [#f1] In practice, implementation and binding code will generally be located .. [#f1] In practice, implementation and binding code will generally be located
@ -136,7 +135,14 @@ On Linux, the above example can be compiled using the following command:
.. code-block:: bash .. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` $ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
.. note::
If you used :ref:`include_as_a_submodule` to get the pybind11 source, then
use ``$(python3-config --includes) -Iextern/pybind11/include`` instead of
``$(python3 -m pybind11 --includes)`` in the above compilation, as
explained in :ref:`building_manually`.
For more details on the required compiler flags on Linux and macOS, see For more details on the required compiler flags on Linux and macOS, see
:ref:`building_manually`. For complete cross-platform compilation instructions, :ref:`building_manually`. For complete cross-platform compilation instructions,
@ -159,12 +165,12 @@ load and execute the example:
.. code-block:: pycon .. code-block:: pycon
$ python $ python
Python 2.7.10 (default, Aug 22 2015, 20:33:39) Python 3.9.10 (main, Jan 15 2022, 11:48:04)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1)] on darwin [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information. Type "help", "copyright", "credits" or "license" for more information.
>>> import example >>> import example
>>> example.add(1, 2) >>> example.add(1, 2)
3L 3
>>> >>>
.. _keyword_args: .. _keyword_args:

View File

@ -1,8 +1,6 @@
# -*- coding: utf-8 -*-
import random
import os
import time
import datetime as dt import datetime as dt
import os
import random
nfns = 4 # Functions per class nfns = 4 # Functions per class
nargs = 4 # Arguments per function nargs = 4 # Arguments per function
@ -13,20 +11,20 @@ def generate_dummy_code_pybind11(nclasses=10):
bindings = "" bindings = ""
for cl in range(nclasses): for cl in range(nclasses):
decl += "class cl%03i;\n" % cl decl += f"class cl{cl:03};\n"
decl += "\n" decl += "\n"
for cl in range(nclasses): for cl in range(nclasses):
decl += "class cl%03i {\n" % cl decl += f"class {cl:03} {{\n"
decl += "public:\n" decl += "public:\n"
bindings += ' py::class_<cl%03i>(m, "cl%03i")\n' % (cl, cl) bindings += f' py::class_<cl{cl:03}>(m, "cl{cl:03}")\n'
for fn in range(nfns): for fn in range(nfns):
ret = random.randint(0, nclasses - 1) ret = random.randint(0, nclasses - 1)
params = [random.randint(0, nclasses - 1) for i in range(nargs)] params = [random.randint(0, nclasses - 1) for i in range(nargs)]
decl += " cl%03i *fn_%03i(" % (ret, fn) decl += f" cl{ret:03} *fn_{fn:03}("
decl += ", ".join("cl%03i *" % p for p in params) decl += ", ".join(f"cl{p:03} *" for p in params)
decl += ");\n" decl += ");\n"
bindings += ' .def("fn_%03i", &cl%03i::fn_%03i)\n' % (fn, cl, fn) bindings += f' .def("fn_{fn:03}", &cl{cl:03}::fn_{fn:03})\n'
decl += "};\n\n" decl += "};\n\n"
bindings += " ;\n" bindings += " ;\n"
@ -44,23 +42,20 @@ def generate_dummy_code_boost(nclasses=10):
bindings = "" bindings = ""
for cl in range(nclasses): for cl in range(nclasses):
decl += "class cl%03i;\n" % cl decl += f"class cl{cl:03};\n"
decl += "\n" decl += "\n"
for cl in range(nclasses): for cl in range(nclasses):
decl += "class cl%03i {\n" % cl decl += "class cl%03i {\n" % cl
decl += "public:\n" decl += "public:\n"
bindings += ' py::class_<cl%03i>("cl%03i")\n' % (cl, cl) bindings += f' py::class_<cl{cl:03}>("cl{cl:03}")\n'
for fn in range(nfns): for fn in range(nfns):
ret = random.randint(0, nclasses - 1) ret = random.randint(0, nclasses - 1)
params = [random.randint(0, nclasses - 1) for i in range(nargs)] params = [random.randint(0, nclasses - 1) for i in range(nargs)]
decl += " cl%03i *fn_%03i(" % (ret, fn) decl += f" cl{ret:03} *fn_{fn:03}("
decl += ", ".join("cl%03i *" % p for p in params) decl += ", ".join(f"cl{p:03} *" for p in params)
decl += ");\n" decl += ");\n"
bindings += ( bindings += f' .def("fn_{fn:03}", &cl{cl:03}::fn_{fn:03}, py::return_value_policy<py::manage_new_object>())\n'
' .def("fn_%03i", &cl%03i::fn_%03i, py::return_value_policy<py::manage_new_object>())\n'
% (fn, cl, fn)
)
decl += "};\n\n" decl += "};\n\n"
bindings += " ;\n" bindings += " ;\n"
@ -75,8 +70,8 @@ def generate_dummy_code_boost(nclasses=10):
for codegen in [generate_dummy_code_pybind11, generate_dummy_code_boost]: for codegen in [generate_dummy_code_pybind11, generate_dummy_code_boost]:
print("{") print("{")
for i in range(0, 10): for i in range(10):
nclasses = 2 ** i nclasses = 2**i
with open("test.cpp", "w") as f: with open("test.cpp", "w") as f:
f.write(codegen(nclasses)) f.write(codegen(nclasses))
n1 = dt.datetime.now() n1 = dt.datetime.now()

File diff suppressed because it is too large Load Diff

View File

@ -44,20 +44,30 @@ interactive Python session demonstrating this example is shown below:
% python % python
>>> import example >>> import example
>>> p = example.Pet('Molly') >>> p = example.Pet("Molly")
>>> print(p) >>> print(p)
<example.Pet object at 0x10cd98060> <example.Pet object at 0x10cd98060>
>>> p.getName() >>> p.getName()
u'Molly' 'Molly'
>>> p.setName('Charly') >>> p.setName("Charly")
>>> p.getName() >>> p.getName()
u'Charly' 'Charly'
.. seealso:: .. seealso::
Static member functions can be bound in the same way using Static member functions can be bound in the same way using
:func:`class_::def_static`. :func:`class_::def_static`.
.. note::
Binding C++ types in unnamed namespaces (also known as anonymous namespaces)
works reliably on many platforms, but not all. The `XFAIL_CONDITION` in
tests/test_unnamed_namespace_a.py encodes the currently known conditions.
For background see `#4319 <https://github.com/pybind/pybind11/pull/4319>`_.
If portability is a concern, it is therefore not recommended to bind C++
types in unnamed namespaces. It will be safest to manually pick unique
namespace names.
Keyword and default arguments Keyword and default arguments
============================= =============================
It is possible to specify keyword and default arguments using the syntax It is possible to specify keyword and default arguments using the syntax
@ -122,12 +132,12 @@ This makes it possible to write
.. code-block:: pycon .. code-block:: pycon
>>> p = example.Pet('Molly') >>> p = example.Pet("Molly")
>>> p.name >>> p.name
u'Molly' 'Molly'
>>> p.name = 'Charly' >>> p.name = "Charly"
>>> p.name >>> p.name
u'Charly' 'Charly'
Now suppose that ``Pet::name`` was a private internal variable Now suppose that ``Pet::name`` was a private internal variable
that can only be accessed via setters and getters. that can only be accessed via setters and getters.
@ -174,10 +184,10 @@ Native Python classes can pick up new attributes dynamically:
.. code-block:: pycon .. code-block:: pycon
>>> class Pet: >>> class Pet:
... name = 'Molly' ... name = "Molly"
... ...
>>> p = Pet() >>> p = Pet()
>>> p.name = 'Charly' # overwrite existing >>> p.name = "Charly" # overwrite existing
>>> p.age = 2 # dynamically add a new attribute >>> p.age = 2 # dynamically add a new attribute
By default, classes exported from C++ do not support this and the only writable By default, classes exported from C++ do not support this and the only writable
@ -195,7 +205,7 @@ Trying to set any other attribute results in an error:
.. code-block:: pycon .. code-block:: pycon
>>> p = example.Pet() >>> p = example.Pet()
>>> p.name = 'Charly' # OK, attribute defined in C++ >>> p.name = "Charly" # OK, attribute defined in C++
>>> p.age = 2 # fail >>> p.age = 2 # fail
AttributeError: 'Pet' object has no attribute 'age' AttributeError: 'Pet' object has no attribute 'age'
@ -213,7 +223,7 @@ Now everything works as expected:
.. code-block:: pycon .. code-block:: pycon
>>> p = example.Pet() >>> p = example.Pet()
>>> p.name = 'Charly' # OK, overwrite value in C++ >>> p.name = "Charly" # OK, overwrite value in C++
>>> p.age = 2 # OK, dynamically add a new attribute >>> p.age = 2 # OK, dynamically add a new attribute
>>> p.__dict__ # just like a native Python class >>> p.__dict__ # just like a native Python class
{'age': 2} {'age': 2}
@ -280,11 +290,11 @@ expose fields and methods of both types:
.. code-block:: pycon .. code-block:: pycon
>>> p = example.Dog('Molly') >>> p = example.Dog("Molly")
>>> p.name >>> p.name
u'Molly' 'Molly'
>>> p.bark() >>> p.bark()
u'woof!' 'woof!'
The C++ classes defined above are regular non-polymorphic types with an The C++ classes defined above are regular non-polymorphic types with an
inheritance relationship. This is reflected in Python: inheritance relationship. This is reflected in Python:
@ -332,7 +342,7 @@ will automatically recognize this:
>>> type(p) >>> type(p)
PolymorphicDog # automatically downcast PolymorphicDog # automatically downcast
>>> p.bark() >>> p.bark()
u'woof!' 'woof!'
Given a pointer to a polymorphic base, pybind11 performs automatic downcasting Given a pointer to a polymorphic base, pybind11 performs automatic downcasting
to the actual derived type. Note that this goes beyond the usual situation in to the actual derived type. Note that this goes beyond the usual situation in
@ -434,8 +444,7 @@ you can use ``py::detail::overload_cast_impl`` with an additional set of parenth
.def("set", overload_cast_<int>()(&Pet::set), "Set the pet's age") .def("set", overload_cast_<int>()(&Pet::set), "Set the pet's age")
.def("set", overload_cast_<const std::string &>()(&Pet::set), "Set the pet's name"); .def("set", overload_cast_<const std::string &>()(&Pet::set), "Set the pet's name");
.. [#cpp14] A compiler which supports the ``-std=c++14`` flag .. [#cpp14] A compiler which supports the ``-std=c++14`` flag.
or Visual Studio 2015 Update 2 and newer.
.. note:: .. note::
@ -446,8 +455,7 @@ you can use ``py::detail::overload_cast_impl`` with an additional set of parenth
Enumerations and internal types Enumerations and internal types
=============================== ===============================
Let's now suppose that the example class contains an internal enumeration type, Let's now suppose that the example class contains internal types like enumerations, e.g.:
e.g.:
.. code-block:: cpp .. code-block:: cpp
@ -457,10 +465,15 @@ e.g.:
Cat Cat
}; };
struct Attributes {
float age = 0;
};
Pet(const std::string &name, Kind type) : name(name), type(type) { } Pet(const std::string &name, Kind type) : name(name), type(type) { }
std::string name; std::string name;
Kind type; Kind type;
Attributes attr;
}; };
The binding code for this example looks as follows: The binding code for this example looks as follows:
@ -471,22 +484,28 @@ The binding code for this example looks as follows:
pet.def(py::init<const std::string &, Pet::Kind>()) pet.def(py::init<const std::string &, Pet::Kind>())
.def_readwrite("name", &Pet::name) .def_readwrite("name", &Pet::name)
.def_readwrite("type", &Pet::type); .def_readwrite("type", &Pet::type)
.def_readwrite("attr", &Pet::attr);
py::enum_<Pet::Kind>(pet, "Kind") py::enum_<Pet::Kind>(pet, "Kind")
.value("Dog", Pet::Kind::Dog) .value("Dog", Pet::Kind::Dog)
.value("Cat", Pet::Kind::Cat) .value("Cat", Pet::Kind::Cat)
.export_values(); .export_values();
To ensure that the ``Kind`` type is created within the scope of ``Pet``, the py::class_<Pet::Attributes>(pet, "Attributes")
``pet`` :class:`class_` instance must be supplied to the :class:`enum_`. .def(py::init<>())
.def_readwrite("age", &Pet::Attributes::age);
To ensure that the nested types ``Kind`` and ``Attributes`` are created within the scope of ``Pet``, the
``pet`` :class:`class_` instance must be supplied to the :class:`enum_` and :class:`class_`
constructor. The :func:`enum_::export_values` function exports the enum entries constructor. The :func:`enum_::export_values` function exports the enum entries
into the parent scope, which should be skipped for newer C++11-style strongly into the parent scope, which should be skipped for newer C++11-style strongly
typed enums. typed enums.
.. code-block:: pycon .. code-block:: pycon
>>> p = Pet('Lucy', Pet.Cat) >>> p = Pet("Lucy", Pet.Cat)
>>> p.type >>> p.type
Kind.Cat Kind.Cat
>>> int(p.type) >>> int(p.type)
@ -508,7 +527,7 @@ The ``name`` property returns the name of the enum value as a unicode string.
.. code-block:: pycon .. code-block:: pycon
>>> p = Pet( "Lucy", Pet.Cat ) >>> p = Pet("Lucy", Pet.Cat)
>>> pet_type = p.type >>> pet_type = p.type
>>> pet_type >>> pet_type
Pet.Cat Pet.Cat
@ -530,3 +549,7 @@ The ``name`` property returns the name of the enum value as a unicode string.
... ...
By default, these are omitted to conserve space. By default, these are omitted to conserve space.
.. warning::
Contrary to Python customs, enum values from the wrappers should not be compared using ``is``, but with ``==`` (see `#1177 <https://github.com/pybind/pybind11/issues/1177>`_ for background).

View File

@ -42,10 +42,7 @@ An example of a ``setup.py`` using pybind11's helpers:
), ),
] ]
setup( setup(..., ext_modules=ext_modules)
...,
ext_modules=ext_modules
)
If you want to do an automatic search for the highest supported C++ standard, If you want to do an automatic search for the highest supported C++ standard,
that is supported via a ``build_ext`` command override; it will only affect that is supported via a ``build_ext`` command override; it will only affect
@ -64,11 +61,20 @@ that is supported via a ``build_ext`` command override; it will only affect
), ),
] ]
setup( setup(..., cmdclass={"build_ext": build_ext}, ext_modules=ext_modules)
...,
cmdclass={"build_ext": build_ext}, If you have single-file extension modules that are directly stored in the
ext_modules=ext_modules Python source tree (``foo.cpp`` in the same directory as where a ``foo.py``
) would be located), you can also generate ``Pybind11Extensions`` using
``setup_helpers.intree_extensions``: ``intree_extensions(["path/to/foo.cpp",
...])`` returns a list of ``Pybind11Extensions`` which can be passed to
``ext_modules``, possibly after further customizing their attributes
(``libraries``, ``include_dirs``, etc.). By doing so, a ``foo.*.so`` extension
module will be generated and made available upon installation.
``intree_extension`` will automatically detect if you are using a ``src``-style
layout (as long as no namespace packages are involved), but you can also
explicitly pass ``package_dir`` to it (as in ``setuptools.setup``).
Since pybind11 does not require NumPy when building, a light-weight replacement Since pybind11 does not require NumPy when building, a light-weight replacement
for NumPy's parallel compilation distutils tool is included. Use it like this: for NumPy's parallel compilation distutils tool is included. Use it like this:
@ -84,22 +90,23 @@ for NumPy's parallel compilation distutils tool is included. Use it like this:
The argument is the name of an environment variable to control the number of The argument is the name of an environment variable to control the number of
threads, such as ``NPY_NUM_BUILD_JOBS`` (as used by NumPy), though you can set threads, such as ``NPY_NUM_BUILD_JOBS`` (as used by NumPy), though you can set
something different if you want. You can also pass ``default=N`` to set the something different if you want; ``CMAKE_BUILD_PARALLEL_LEVEL`` is another choice
default number of threads (0 will take the number of threads available) and a user might expect. You can also pass ``default=N`` to set the default number
``max=N``, the maximum number of threads; if you have a large extension you may of threads (0 will take the number of threads available) and ``max=N``, the
want set this to a memory dependent number. maximum number of threads; if you have a large extension you may want set this
to a memory dependent number.
If you are developing rapidly and have a lot of C++ files, you may want to If you are developing rapidly and have a lot of C++ files, you may want to
avoid rebuilding files that have not changed. For simple cases were you are avoid rebuilding files that have not changed. For simple cases were you are
using ``pip install -e .`` and do not have local headers, you can skip the using ``pip install -e .`` and do not have local headers, you can skip the
rebuild if a object file is newer than it's source (headers are not checked!) rebuild if an object file is newer than its source (headers are not checked!)
with the following: with the following:
.. code-block:: python .. code-block:: python
from pybind11.setup_helpers import ParallelCompile, naive_recompile from pybind11.setup_helpers import ParallelCompile, naive_recompile
SmartCompile("NPY_NUM_BUILD_JOBS", needs_recompile=naive_recompile).install() ParallelCompile("NPY_NUM_BUILD_JOBS", needs_recompile=naive_recompile).install()
If you have a more complex build, you can implement a smarter function and pass If you have a more complex build, you can implement a smarter function and pass
@ -136,7 +143,7 @@ Your ``pyproject.toml`` file will likely look something like this:
.. code-block:: toml .. code-block:: toml
[build-system] [build-system]
requires = ["setuptools", "wheel", "pybind11==2.6.0"] requires = ["setuptools>=42", "pybind11>=2.6.1"]
build-backend = "setuptools.build_meta" build-backend = "setuptools.build_meta"
.. note:: .. note::
@ -147,10 +154,12 @@ Your ``pyproject.toml`` file will likely look something like this:
in Python) using something like `cibuildwheel`_, remember that ``setup.py`` in Python) using something like `cibuildwheel`_, remember that ``setup.py``
and ``pyproject.toml`` are not even contained in the wheel, so this high and ``pyproject.toml`` are not even contained in the wheel, so this high
Pip requirement is only for source builds, and will not affect users of Pip requirement is only for source builds, and will not affect users of
your binary wheels. your binary wheels. If you are building SDists and wheels, then
`pypa-build`_ is the recommended official tool.
.. _PEP 517: https://www.python.org/dev/peps/pep-0517/ .. _PEP 517: https://www.python.org/dev/peps/pep-0517/
.. _cibuildwheel: https://cibuildwheel.readthedocs.io .. _cibuildwheel: https://cibuildwheel.readthedocs.io
.. _pypa-build: https://pypa-build.readthedocs.io/en/latest/
.. _setup_helpers-setup_requires: .. _setup_helpers-setup_requires:
@ -232,7 +241,7 @@ extension module can be created with just a few lines of code:
.. code-block:: cmake .. code-block:: cmake
cmake_minimum_required(VERSION 3.4...3.18) cmake_minimum_required(VERSION 3.5...3.27)
project(example LANGUAGES CXX) project(example LANGUAGES CXX)
add_subdirectory(pybind11) add_subdirectory(pybind11)
@ -252,6 +261,9 @@ PyPI integration, can be found in the [cmake_example]_ repository.
.. versionchanged:: 2.6 .. versionchanged:: 2.6
CMake 3.4+ is required. CMake 3.4+ is required.
.. versionchanged:: 2.11
CMake 3.5+ is required.
Further information can be found at :doc:`cmake/index`. Further information can be found at :doc:`cmake/index`.
pybind11_add_module pybind11_add_module
@ -331,7 +343,7 @@ standard explicitly with
set(CMAKE_CXX_STANDARD 14 CACHE STRING "C++ version selection") # or 11, 14, 17, 20 set(CMAKE_CXX_STANDARD 14 CACHE STRING "C++ version selection") # or 11, 14, 17, 20
set(CMAKE_CXX_STANDARD_REQUIRED ON) # optional, ensure standard is supported set(CMAKE_CXX_STANDARD_REQUIRED ON) # optional, ensure standard is supported
set(CMAKE_CXX_EXTENSIONS OFF) # optional, keep compiler extensionsn off set(CMAKE_CXX_EXTENSIONS OFF) # optional, keep compiler extensions off
The variables can also be set when calling CMake from the command line using The variables can also be set when calling CMake from the command line using
the ``-D<variable>=<value>`` flag. You can also manually set ``CXX_STANDARD`` the ``-D<variable>=<value>`` flag. You can also manually set ``CXX_STANDARD``
@ -401,16 +413,17 @@ can refer to the same [cmake_example]_ repository for a full sample project
FindPython mode FindPython mode
--------------- ---------------
CMake 3.12+ (3.15+ recommended) added a new module called FindPython that had a CMake 3.12+ (3.15+ recommended, 3.18.2+ ideal) added a new module called
highly improved search algorithm and modern targets and tools. If you use FindPython that had a highly improved search algorithm and modern targets
FindPython, pybind11 will detect this and use the existing targets instead: and tools. If you use FindPython, pybind11 will detect this and use the
existing targets instead:
.. code-block:: cmake .. code-block:: cmake
cmake_minumum_required(VERSION 3.15...3.18) cmake_minimum_required(VERSION 3.15...3.22)
project(example LANGUAGES CXX) project(example LANGUAGES CXX)
find_package(Python COMPONENTS Interpreter Development REQUIRED) find_package(Python 3.6 COMPONENTS Interpreter Development REQUIRED)
find_package(pybind11 CONFIG REQUIRED) find_package(pybind11 CONFIG REQUIRED)
# or add_subdirectory(pybind11) # or add_subdirectory(pybind11)
@ -423,9 +436,8 @@ algorithms from the CMake invocation, with ``-DPYBIND11_FINDPYTHON=ON``.
.. warning:: .. warning::
If you use FindPython2 and FindPython3 to dual-target Python, use the If you use FindPython to multi-target Python versions, use the individual
individual targets listed below, and avoid targets that directly include targets listed below, and avoid targets that directly include Python parts.
Python parts.
There are `many ways to hint or force a discovery of a specific Python There are `many ways to hint or force a discovery of a specific Python
installation <https://cmake.org/cmake/help/latest/module/FindPython.html>`_), installation <https://cmake.org/cmake/help/latest/module/FindPython.html>`_),
@ -433,6 +445,14 @@ setting ``Python_ROOT_DIR`` may be the most common one (though with
virtualenv/venv support, and Conda support, this tends to find the correct virtualenv/venv support, and Conda support, this tends to find the correct
Python version more often than the old system did). Python version more often than the old system did).
.. warning::
When the Python libraries (i.e. ``libpythonXX.a`` and ``libpythonXX.so``
on Unix) are not available, as is the case on a manylinux image, the
``Development`` component will not be resolved by ``FindPython``. When not
using the embedding functionality, CMake 3.18+ allows you to specify
``Development.Module`` instead of ``Development`` to resolve this issue.
.. versionadded:: 2.6 .. versionadded:: 2.6
Advanced: interface library targets Advanced: interface library targets
@ -444,11 +464,8 @@ available in all modes. The targets provided are:
``pybind11::headers`` ``pybind11::headers``
Just the pybind11 headers and minimum compile requirements Just the pybind11 headers and minimum compile requirements
``pybind11::python2_no_register``
Quiets the warning/error when mixing C++14 or higher and Python 2
``pybind11::pybind11`` ``pybind11::pybind11``
Python headers + ``pybind11::headers`` + ``pybind11::python2_no_register`` (Python 2 only) Python headers + ``pybind11::headers``
``pybind11::python_link_helper`` ``pybind11::python_link_helper``
Just the "linking" part of pybind11:module Just the "linking" part of pybind11:module
@ -457,7 +474,7 @@ available in all modes. The targets provided are:
Everything for extension modules - ``pybind11::pybind11`` + ``Python::Module`` (FindPython CMake 3.15+) or ``pybind11::python_link_helper`` Everything for extension modules - ``pybind11::pybind11`` + ``Python::Module`` (FindPython CMake 3.15+) or ``pybind11::python_link_helper``
``pybind11::embed`` ``pybind11::embed``
Everything for embedding the Python interpreter - ``pybind11::pybind11`` + ``Python::Embed`` (FindPython) or Python libs Everything for embedding the Python interpreter - ``pybind11::pybind11`` + ``Python::Python`` (FindPython) or Python libs
``pybind11::lto`` / ``pybind11::thin_lto`` ``pybind11::lto`` / ``pybind11::thin_lto``
An alternative to `INTERPROCEDURAL_OPTIMIZATION` for adding link-time optimization. An alternative to `INTERPROCEDURAL_OPTIMIZATION` for adding link-time optimization.
@ -481,7 +498,7 @@ You can use these targets to build complex applications. For example, the
.. code-block:: cmake .. code-block:: cmake
cmake_minimum_required(VERSION 3.4) cmake_minimum_required(VERSION 3.5...3.27)
project(example LANGUAGES CXX) project(example LANGUAGES CXX)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11) find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
@ -491,7 +508,10 @@ You can use these targets to build complex applications. For example, the
target_link_libraries(example PRIVATE pybind11::module pybind11::lto pybind11::windows_extras) target_link_libraries(example PRIVATE pybind11::module pybind11::lto pybind11::windows_extras)
pybind11_extension(example) pybind11_extension(example)
pybind11_strip(example) if(NOT MSVC AND NOT ${CMAKE_BUILD_TYPE} MATCHES Debug|RelWithDebInfo)
# Strip unnecessary sections of the binary on Linux/macOS
pybind11_strip(example)
endif()
set_target_properties(example PROPERTIES CXX_VISIBILITY_PRESET "hidden" set_target_properties(example PROPERTIES CXX_VISIBILITY_PRESET "hidden"
CUDA_VISIBILITY_PRESET "hidden") CUDA_VISIBILITY_PRESET "hidden")
@ -504,7 +524,7 @@ Instead of setting properties, you can set ``CMAKE_*`` variables to initialize t
compiler flags are provided to ensure high quality code generation. In compiler flags are provided to ensure high quality code generation. In
contrast to the ``pybind11_add_module()`` command, the CMake interface contrast to the ``pybind11_add_module()`` command, the CMake interface
provides a *composable* set of targets to ensure that you retain flexibility. provides a *composable* set of targets to ensure that you retain flexibility.
It can be expecially important to provide or set these properties; the It can be especially important to provide or set these properties; the
:ref:`FAQ <faq:symhidden>` contains an explanation on why these are needed. :ref:`FAQ <faq:symhidden>` contains an explanation on why these are needed.
.. versionadded:: 2.6 .. versionadded:: 2.6
@ -536,7 +556,7 @@ information about usage in C++, see :doc:`/advanced/embedding`.
.. code-block:: cmake .. code-block:: cmake
cmake_minimum_required(VERSION 3.4...3.18) cmake_minimum_required(VERSION 3.5...3.27)
project(example LANGUAGES CXX) project(example LANGUAGES CXX)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11) find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
@ -557,10 +577,7 @@ On Linux, you can compile an example such as the one given in
.. code-block:: bash .. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` $ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
The flags given here assume that you're using Python 3. For Python 2, just
change the executable appropriately (to ``python`` or ``python2``).
The ``python3 -m pybind11 --includes`` command fetches the include paths for The ``python3 -m pybind11 --includes`` command fetches the include paths for
both pybind11 and Python headers. This assumes that pybind11 has been installed both pybind11 and Python headers. This assumes that pybind11 has been installed
@ -568,19 +585,13 @@ using ``pip`` or ``conda``. If it hasn't, you can also manually specify
``-I <path-to-pybind11>/include`` together with the Python includes path ``-I <path-to-pybind11>/include`` together with the Python includes path
``python3-config --includes``. ``python3-config --includes``.
Note that Python 2.7 modules don't use a special suffix, so you should simply
use ``example.so`` instead of ``example`python3-config --extension-suffix```.
Besides, the ``--extension-suffix`` option may or may not be available, depending
on the distribution; in the latter case, the module extension can be manually
set to ``.so``.
On macOS: the build command is almost the same but it also requires passing On macOS: the build command is almost the same but it also requires passing
the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when
building the module: building the module:
.. code-block:: bash .. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` $ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
In general, it is advisable to include several additional build parameters In general, it is advisable to include several additional build parameters
that can considerably reduce the size of the created binary. Refer to section that can considerably reduce the size of the created binary. Refer to section
@ -628,3 +639,11 @@ cross-project dependency management. Additionally, it is able to autogenerate
customizable pybind11-based wrappers by parsing C++ header files. customizable pybind11-based wrappers by parsing C++ header files.
.. [robotpy-build] https://robotpy-build.readthedocs.io .. [robotpy-build] https://robotpy-build.readthedocs.io
[litgen]_ is an automatic python bindings generator with a focus on generating
documented and discoverable bindings: bindings will nicely reproduce the documentation
found in headers. It is is based on srcML (srcml.org), a highly scalable, multi-language
parsing tool with a developer centric approach. The API that you want to expose to python
must be C++14 compatible (but your implementation can use more modern constructs).
.. [litgen] https://pthom.github.io/litgen

View File

@ -1,5 +1,4 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*-
# #
# pybind11 documentation build configuration file, created by # pybind11 documentation build configuration file, created by
# sphinx-quickstart on Sun Oct 11 19:23:48 2015. # sphinx-quickstart on Sun Oct 11 19:23:48 2015.
@ -13,12 +12,11 @@
# All configuration values have a default; values that are commented out # All configuration values have a default; values that are commented out
# serve to show the default. # serve to show the default.
import sys
import os import os
import shlex
import subprocess
from pathlib import Path
import re import re
import subprocess
import sys
from pathlib import Path
DIR = Path(__file__).parent.resolve() DIR = Path(__file__).parent.resolve()
@ -37,6 +35,7 @@ DIR = Path(__file__).parent.resolve()
# ones. # ones.
extensions = [ extensions = [
"breathe", "breathe",
"sphinx_copybutton",
"sphinxcontrib.rsvgconverter", "sphinxcontrib.rsvgconverter",
"sphinxcontrib.moderncmakedomain", "sphinxcontrib.moderncmakedomain",
] ]
@ -127,23 +126,7 @@ todo_include_todos = False
# The theme to use for HTML and HTML Help pages. See the documentation for # The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes. # a list of builtin themes.
on_rtd = os.environ.get("READTHEDOCS", None) == "True" html_theme = "furo"
if not on_rtd: # only import and set the theme if we're building docs locally
import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
html_context = {"css_files": ["_static/theme_overrides.css"]}
else:
html_context = {
"css_files": [
"//media.readthedocs.org/css/sphinx_rtd_theme.css",
"//media.readthedocs.org/css/readthedocs-doc-embed.css",
"_static/theme_overrides.css",
]
}
# Theme options are theme-specific and customize the look and feel of a theme # Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the # further. For a list of options available for each theme, see the
@ -174,6 +157,10 @@ else:
# so a file named "default.css" will overwrite the builtin "default.css". # so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"] html_static_path = ["_static"]
html_css_files = [
"css/custom.css",
]
# Add any extra paths that contain custom files (such as robots.txt or # Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied # .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation. # directly to the root of the documentation.
@ -239,6 +226,8 @@ htmlhelp_basename = "pybind11doc"
# -- Options for LaTeX output --------------------------------------------- # -- Options for LaTeX output ---------------------------------------------
latex_engine = "pdflatex"
latex_elements = { latex_elements = {
# The paper size ('letterpaper' or 'a4paper'). # The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper', # 'papersize': 'letterpaper',
@ -344,29 +333,31 @@ def generate_doxygen_xml(app):
subprocess.call(["doxygen", "--version"]) subprocess.call(["doxygen", "--version"])
retcode = subprocess.call(["doxygen"], cwd=app.confdir) retcode = subprocess.call(["doxygen"], cwd=app.confdir)
if retcode < 0: if retcode < 0:
sys.stderr.write("doxygen error code: {}\n".format(-retcode)) sys.stderr.write(f"doxygen error code: {-retcode}\n")
except OSError as e: except OSError as e:
sys.stderr.write("doxygen execution failed: {}\n".format(e)) sys.stderr.write(f"doxygen execution failed: {e}\n")
def prepare(app): def prepare(app):
with open(DIR.parent / "README.rst") as f: with open(DIR.parent / "README.rst") as f:
contents = f.read() contents = f.read()
# Filter out section titles for index.rst for LaTeX
if app.builder.name == "latex": if app.builder.name == "latex":
# Remove badges and stuff from start
contents = contents[contents.find(r".. start") :]
# Filter out section titles for index.rst for LaTeX
contents = re.sub(r"^(.*)\n[-~]{3,}$", r"**\1**", contents, flags=re.MULTILINE) contents = re.sub(r"^(.*)\n[-~]{3,}$", r"**\1**", contents, flags=re.MULTILINE)
with open(DIR / "readme.rst", "w") as f: with open(DIR / "readme.rst", "w") as f:
f.write(contents) f.write(contents)
def clean_up(app, exception): def clean_up(app, exception): # noqa: ARG001
(DIR / "readme.rst").unlink() (DIR / "readme.rst").unlink()
def setup(app): def setup(app):
# Add hook for building doxygen xml when needed # Add hook for building doxygen xml when needed
app.connect("builder-inited", generate_doxygen_xml) app.connect("builder-inited", generate_doxygen_xml)

View File

@ -5,12 +5,10 @@ Frequently asked questions
=========================================================== ===========================================================
1. Make sure that the name specified in PYBIND11_MODULE is identical to the 1. Make sure that the name specified in PYBIND11_MODULE is identical to the
filename of the extension library (without suffixes such as .so) filename of the extension library (without suffixes such as ``.so``).
2. If the above did not fix the issue, you are likely using an incompatible 2. If the above did not fix the issue, you are likely using an incompatible
version of Python (for instance, the extension library was compiled against version of Python that does not match what you compiled with.
Python 2, while the interpreter is running on top of some version of Python
3, or vice versa).
"Symbol not found: ``__Py_ZeroStruct`` / ``_PyInstanceMethod_Type``" "Symbol not found: ``__Py_ZeroStruct`` / ``_PyInstanceMethod_Type``"
======================================================================== ========================================================================
@ -54,7 +52,7 @@ provided by the caller -- in fact, it does nothing at all.
.. code-block:: python .. code-block:: python
def increment(i): def increment(i):
i += 1 # nope.. i += 1 # nope..
pybind11 is also affected by such language-level conventions, which means that pybind11 is also affected by such language-level conventions, which means that
binding ``increment`` or ``increment_ptr`` will also create Python functions binding ``increment`` or ``increment_ptr`` will also create Python functions
@ -147,7 +145,7 @@ using C++14 template metaprogramming.
.. _`faq:hidden_visibility`: .. _`faq:hidden_visibility`:
"SomeClass declared with greater visibility than the type of its field SomeClass::member [-Wattributes]" "'SomeClass' declared with greater visibility than the type of its field 'SomeClass::member' [-Wattributes]"
============================================================================================================ ============================================================================================================
This error typically indicates that you are compiling without the required This error typically indicates that you are compiling without the required
@ -169,8 +167,8 @@ can be changed, but even if it isn't it is not always enough to guarantee
complete independence of the symbols involved when not using complete independence of the symbols involved when not using
``-fvisibility=hidden``. ``-fvisibility=hidden``.
Additionally, ``-fvisiblity=hidden`` can deliver considerably binary size Additionally, ``-fvisibility=hidden`` can deliver considerably binary size
savings. (See the following section for more details). savings. (See the following section for more details.)
.. _`faq:symhidden`: .. _`faq:symhidden`:
@ -180,7 +178,7 @@ How can I create smaller binaries?
To do its job, pybind11 extensively relies on a programming technique known as To do its job, pybind11 extensively relies on a programming technique known as
*template metaprogramming*, which is a way of performing computation at compile *template metaprogramming*, which is a way of performing computation at compile
time using type information. Template metaprogamming usually instantiates code time using type information. Template metaprogramming usually instantiates code
involving significant numbers of deeply nested types that are either completely involving significant numbers of deeply nested types that are either completely
removed or reduced to just a few instructions during the compiler's optimization removed or reduced to just a few instructions during the compiler's optimization
phase. However, due to the nested nature of these types, the resulting symbol phase. However, due to the nested nature of these types, the resulting symbol
@ -222,20 +220,6 @@ In addition to decreasing binary size, ``-fvisibility=hidden`` also avoids
potential serious issues when loading multiple modules and is required for potential serious issues when loading multiple modules and is required for
proper pybind operation. See the previous FAQ entry for more details. proper pybind operation. See the previous FAQ entry for more details.
Working with ancient Visual Studio 2008 builds on Windows
=========================================================
The official Windows distributions of Python are compiled using truly
ancient versions of Visual Studio that lack good C++11 support. Some users
implicitly assume that it would be impossible to load a plugin built with
Visual Studio 2015 into a Python distribution that was compiled using Visual
Studio 2008. However, no such issue exists: it's perfectly legitimate to
interface DLLs that are built with different compilers and/or C libraries.
Common gotchas to watch out for involve not ``free()``-ing memory region
that that were ``malloc()``-ed in another shared library, using data
structures with incompatible ABIs, and so on. pybind11 is very careful not
to make these types of mistakes.
How can I properly handle Ctrl-C in long-running functions? How can I properly handle Ctrl-C in long-running functions?
=========================================================== ===========================================================
@ -289,27 +273,7 @@ Conflicts can arise, however, when using pybind11 in a project that *also* uses
the CMake Python detection in a system with several Python versions installed. the CMake Python detection in a system with several Python versions installed.
This difference may cause inconsistencies and errors if *both* mechanisms are This difference may cause inconsistencies and errors if *both* mechanisms are
used in the same project. Consider the following CMake code executed in a used in the same project.
system with Python 2.7 and 3.x installed:
.. code-block:: cmake
find_package(PythonInterp)
find_package(PythonLibs)
find_package(pybind11)
It will detect Python 2.7 and pybind11 will pick it as well.
In contrast this code:
.. code-block:: cmake
find_package(pybind11)
find_package(PythonInterp)
find_package(PythonLibs)
will detect Python 3.x for pybind11 and may crash on
``find_package(PythonLibs)`` afterwards.
There are three possible solutions: There are three possible solutions:
@ -320,7 +284,8 @@ There are three possible solutions:
COMPONENTS Interpreter Development)`` on modern CMake (3.12+, 3.15+ better, COMPONENTS Interpreter Development)`` on modern CMake (3.12+, 3.15+ better,
3.18.2+ best). Pybind11 in these cases uses the new CMake FindPython instead 3.18.2+ best). Pybind11 in these cases uses the new CMake FindPython instead
of the old, deprecated search tools, and these modules are much better at of the old, deprecated search tools, and these modules are much better at
finding the correct Python. finding the correct Python. If FindPythonLibs/Interp are not available
(CMake 3.27+), then this will be ignored and FindPython will be used.
3. Set ``PYBIND11_NOPYTHON`` to ``TRUE``. Pybind11 will not search for Python. 3. Set ``PYBIND11_NOPYTHON`` to ``TRUE``. Pybind11 will not search for Python.
However, you will have to use the target-based system, and do more setup However, you will have to use the target-based system, and do more setup
yourself, because it does not know about or include things that depend on yourself, because it does not know about or include things that depend on

View File

@ -8,6 +8,8 @@ There are several ways to get the pybind11 source, which lives at
developers recommend one of the first three ways listed here, submodule, PyPI, developers recommend one of the first three ways listed here, submodule, PyPI,
or conda-forge, for obtaining pybind11. or conda-forge, for obtaining pybind11.
.. _include_as_a_submodule:
Include as a submodule Include as a submodule
====================== ======================
@ -16,7 +18,7 @@ as a submodule. From your git repository, use:
.. code-block:: bash .. code-block:: bash
git submodule add ../../pybind/pybind11 extern/pybind11 -b stable git submodule add -b stable ../../pybind/pybind11 extern/pybind11
git submodule update --init git submodule update --init
This assumes you are placing your dependencies in ``extern/``, and that you are This assumes you are placing your dependencies in ``extern/``, and that you are

View File

@ -57,16 +57,16 @@ clean, well written patch would likely be accepted to solve them.
Python 3.9.0 warning Python 3.9.0 warning
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
Combining older versions of pybind11 (< 2.6.0) with Python on 3.9.0 will Combining older versions of pybind11 (< 2.6.0) with Python on exactly 3.9.0
trigger undefined behavior that typically manifests as crashes during will trigger undefined behavior that typically manifests as crashes during
interpreter shutdown (but could also destroy your data. **You have been interpreter shutdown (but could also destroy your data. **You have been
warned**). warned**).
This issue has been This issue was `fixed in Python <https://github.com/python/cpython/pull/22670>`_.
`fixed in Python <https://github.com/python/cpython/pull/22670>`_. As a As a mitigation for this bug, pybind11 2.6.0 or newer includes a workaround
mitigation until 3.9.1 is released and commonly used, pybind11 (2.6.0 or newer) specifically when Python 3.9.0 is detected at runtime, leaking about 50 bytes
includes a temporary workaround specifically when Python 3.9.0 is detected at of memory when a callback function is garbage collected. For reference, the
runtime, leaking about 50 bytes of memory when a callback function is garbage pybind11 test suite has about 2,000 such callbacks, but only 49 are garbage
collected. For reference; the pybind11 test suite has about 2,000 such collected before the end-of-process. Wheels (even if built with Python 3.9.0)
callbacks, but only 49 are garbage collected before the end-of-process. Wheels will correctly avoid the leak when run in Python 3.9.1, and this does not
built with Python 3.9.0 will correctly avoid the leak when run in Python 3.9.1. affect other 3.X versions.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 60 KiB

View File

@ -52,6 +52,20 @@ Convenience classes for specific Python types
.. doxygengroup:: pytypes .. doxygengroup:: pytypes
:members: :members:
Convenience functions converting to Python types
================================================
.. doxygenfunction:: make_tuple(Args&&...)
.. doxygenfunction:: make_iterator(Iterator, Sentinel, Extra &&...)
.. doxygenfunction:: make_iterator(Type &, Extra&&...)
.. doxygenfunction:: make_key_iterator(Iterator, Sentinel, Extra &&...)
.. doxygenfunction:: make_key_iterator(Type &, Extra&&...)
.. doxygenfunction:: make_value_iterator(Iterator, Sentinel, Extra &&...)
.. doxygenfunction:: make_value_iterator(Type &, Extra&&...)
.. _extras: .. _extras:
Passing extra arguments to ``def`` or ``class_`` Passing extra arguments to ``def`` or ``class_``
@ -110,7 +124,6 @@ Exceptions
.. doxygenclass:: builtin_exception .. doxygenclass:: builtin_exception
:members: :members:
Literals Literals
======== ========

View File

@ -15,63 +15,129 @@ For example:
For beta, ``PYBIND11_VERSION_PATCH`` should be ``Z.b1``. RC's can be ``Z.rc1``. For beta, ``PYBIND11_VERSION_PATCH`` should be ``Z.b1``. RC's can be ``Z.rc1``.
Always include the dot (even though PEP 440 allows it to be dropped). For a Always include the dot (even though PEP 440 allows it to be dropped). For a
final release, this must be a simple integer. final release, this must be a simple integer. There is also
``PYBIND11_VERSION_HEX`` just below that needs to be updated.
To release a new version of pybind11: To release a new version of pybind11:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you don't have nox, you should either use ``pipx run nox`` instead, or use
``pipx install nox`` or ``brew install nox`` (Unix).
- Update the version number - Update the version number
- Update ``PYBIND11_VERSION_MAJOR`` etc. in - Update ``PYBIND11_VERSION_MAJOR`` etc. in
``include/pybind11/detail/common.h``. PATCH should be a simple integer. ``include/pybind11/detail/common.h``. PATCH should be a simple integer.
- Update ``pybind11/_version.py`` (match above)
- Ensure that all the information in ``setup.py`` is up-to-date. - Update ``PYBIND11_VERSION_HEX`` just below as well.
- Add release date in ``docs/changelog.rst``.
- ``git add`` and ``git commit``, ``git push``. **Ensure CI passes**. (If it - Update ``pybind11/_version.py`` (match above).
- Run ``nox -s tests_packaging`` to ensure this was done correctly.
- Ensure that all the information in ``setup.cfg`` is up-to-date, like
supported Python versions.
- Add release date in ``docs/changelog.rst`` and integrate the output of
``nox -s make_changelog``.
- Note that the ``nox -s make_changelog`` command inspects
`needs changelog <https://github.com/pybind/pybind11/pulls?q=is%3Apr+is%3Aclosed+label%3A%22needs+changelog%22>`_.
- Manually clear the ``needs changelog`` labels using the GitHub web
interface (very easy: start by clicking the link above).
- ``git add`` and ``git commit``, ``git push``. **Ensure CI passes**. (If it
fails due to a known flake issue, either ignore or restart CI.) fails due to a known flake issue, either ignore or restart CI.)
- Add a release branch if this is a new minor version
- ``git checkout -b vX.Y``, ``git push -u origin vX.Y`` - Add a release branch if this is a new MINOR version, or update the existing
release branch if it is a patch version
- New branch: ``git checkout -b vX.Y``, ``git push -u origin vX.Y``
- Update branch: ``git checkout vX.Y``, ``git merge <release branch>``, ``git push``
- Update tags (optional; if you skip this, the GitHub release makes a - Update tags (optional; if you skip this, the GitHub release makes a
non-annotated tag for you) non-annotated tag for you)
- ``git tag -a vX.Y.Z -m 'vX.Y.Z release'``.
- ``git push --tags``. - ``git tag -a vX.Y.Z -m 'vX.Y.Z release'``
- ``grep ^__version__ pybind11/_version.py``
- Last-minute consistency check: same as tag?
- ``git push --tags``
- Update stable - Update stable
- ``git checkout stable``
- ``git merge master`` - ``git checkout stable``
- ``git push``
- ``git merge -X theirs vX.Y.Z``
- ``git diff vX.Y.Z``
- Carefully review and reconcile any diffs. There should be none.
- ``git push``
- Make a GitHub release (this shows up in the UI, sends new release - Make a GitHub release (this shows up in the UI, sends new release
notifications to users watching releases, and also uploads PyPI packages). notifications to users watching releases, and also uploads PyPI packages).
(Note: if you do not use an existing tag, this creates a new lightweight tag (Note: if you do not use an existing tag, this creates a new lightweight tag
for you, so you could skip the above step). for you, so you could skip the above step.)
- GUI method: click "Create a new release" on the far right, fill in the tag
name (if you didn't tag above, it will be made here), fill in a release - GUI method: Under `releases <https://github.com/pybind/pybind11/releases>`_
name like "Version X.Y.Z", and optionally copy-and-paste the changelog into click "Draft a new release" on the far right, fill in the tag name
the description (processed as markdown by Pandoc). Check "pre-release" if (if you didn't tag above, it will be made here), fill in a release name
this is a beta/RC. like "Version X.Y.Z", and copy-and-paste the markdown-formatted (!) changelog
into the description. You can use ``cat docs/changelog.rst | pandoc -f rst -t gfm``,
then manually remove line breaks and strip links to PRs and issues,
e.g. to a bare ``#1234``, without the surrounding ``<...>_`` hyperlink markup.
Check "pre-release" if this is a beta/RC.
- CLI method: with ``gh`` installed, run ``gh release create vX.Y.Z -t "Version X.Y.Z"`` - CLI method: with ``gh`` installed, run ``gh release create vX.Y.Z -t "Version X.Y.Z"``
If this is a pre-release, add ``-p``. If this is a pre-release, add ``-p``.
- Get back to work - Get back to work
- Make sure you are on master, not somewhere else: ``git checkout master`` - Make sure you are on master, not somewhere else: ``git checkout master``
- Update version macros in ``include/pybind11/detail/common.h`` (set PATCH to - Update version macros in ``include/pybind11/detail/common.h`` (set PATCH to
``0.dev1`` and increment MINOR). ``0.dev1`` and increment MINOR).
- Update ``_version.py`` to match
- Add a plot for in-development updates in ``docs/changelog.rst``. - Update ``pybind11/_version.py`` to match.
- Run ``nox -s tests_packaging`` to ensure this was done correctly.
- If the release was a new MINOR version, add a new ``IN DEVELOPMENT``
section in ``docs/changelog.rst``.
- ``git add``, ``git commit``, ``git push`` - ``git add``, ``git commit``, ``git push``
If a version branch is updated, remember to set PATCH to ``1.dev1``. If a version branch is updated, remember to set PATCH to ``1.dev1``.
If you'd like to bump homebrew, run:
.. code-block:: console
brew bump-formula-pr --url https://github.com/pybind/pybind11/archive/vX.Y.Z.tar.gz
Conda-forge should automatically make a PR in a few hours, and automatically
merge it if there are no issues.
Manual packaging Manual packaging
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
If you need to manually upload releases, you can download the releases from the job artifacts and upload them with twine. You can also make the files locally (not recommended in general, as your local directory is more likely to be "dirty" and SDists love picking up random unrelated/hidden files); this is the procedure: If you need to manually upload releases, you can download the releases from
the job artifacts and upload them with twine. You can also make the files
locally (not recommended in general, as your local directory is more likely
to be "dirty" and SDists love picking up random unrelated/hidden files);
this is the procedure:
.. code-block:: bash .. code-block:: bash
python3 -m pip install build nox -s build
python3 -m build
PYBIND11_SDIST_GLOBAL=1 python3 -m build
twine upload dist/* twine upload dist/*
This makes SDists and wheels, and the final line uploads them. This makes SDists and wheels, and the final line uploads them.

View File

@ -1,7 +1,6 @@
breathe==4.20.0 breathe==4.34.0
commonmark==0.9.1 furo==2022.6.21
recommonmark==0.6.0 sphinx==5.0.2
sphinx==3.2.1 sphinx-copybutton==0.5.0
sphinx_rtd_theme==0.5.0 sphinxcontrib-moderncmakedomain==3.21.4
sphinxcontrib-moderncmakedomain==3.13 sphinxcontrib-svg2pdfconverter==1.2.0
sphinxcontrib-svg2pdfconverter==1.1.0

View File

@ -8,6 +8,88 @@ to a new version. But it goes into more detail. This includes things like
deprecated APIs and their replacements, build system changes, general code deprecated APIs and their replacements, build system changes, general code
modernization and other useful information. modernization and other useful information.
.. _upgrade-guide-2.12:
v2.12
=====
NumPy support has been upgraded to support the 2.x series too. The two relevant
changes are that:
* ``dtype.flags()`` is now a ``uint64`` and ``dtype.alignment()`` an
``ssize_t`` (and NumPy may return an larger than integer value for
``itemsize()`` in NumPy 2.x).
* The long deprecated NumPy function ``PyArray_GetArrayParamsFromObject``
function is not available anymore.
Due to NumPy changes, you may experience difficulties updating to NumPy 2.
Please see the [NumPy 2 migration guide](https://numpy.org/devdocs/numpy_2_0_migration_guide.html) for details.
For example, a more direct change could be that the default integer ``"int_"``
(and ``"uint"``) is now ``ssize_t`` and not ``long`` (affects 64bit windows).
If you want to only support NumPy 1.x for now and are having problems due to
the two internal changes listed above, you can define
``PYBIND11_NUMPY_1_ONLY`` to disable the new support for now. Make sure you
define this on all pybind11 compile units, since it could be a source of ODR
violations if used inconsistently. This option will be removed in the future,
so adapting your code is highly recommended.
.. _upgrade-guide-2.11:
v2.11
=====
* The minimum version of CMake is now 3.5. A future version will likely move to
requiring something like CMake 3.15. Note that CMake 3.27 is removing the
long-deprecated support for ``FindPythonInterp`` if you set 3.27 as the
minimum or maximum supported version. To prepare for that future, CMake 3.15+
using ``FindPython`` or setting ``PYBIND11_FINDPYTHON`` is highly recommended,
otherwise pybind11 will automatically switch to using ``FindPython`` if
``FindPythonInterp`` is not available.
.. _upgrade-guide-2.9:
v2.9
====
* Any usage of the recently added ``py::make_simple_namespace`` should be
converted to using ``py::module_::import("types").attr("SimpleNamespace")``
instead.
* The use of ``_`` in custom type casters can now be replaced with the more
readable ``const_name`` instead. The old ``_`` shortcut has been retained
unless it is being used as a macro (like for gettext).
.. _upgrade-guide-2.7:
v2.7
====
*Before* v2.7, ``py::str`` can hold ``PyUnicodeObject`` or ``PyBytesObject``,
and ``py::isinstance<str>()`` is ``true`` for both ``py::str`` and
``py::bytes``. Starting with v2.7, ``py::str`` exclusively holds
``PyUnicodeObject`` (`#2409 <https://github.com/pybind/pybind11/pull/2409>`_),
and ``py::isinstance<str>()`` is ``true`` only for ``py::str``. To help in
the transition of user code, the ``PYBIND11_STR_LEGACY_PERMISSIVE`` macro
is provided as an escape hatch to go back to the legacy behavior. This macro
will be removed in future releases. Two types of required fixes are expected
to be common:
* Accidental use of ``py::str`` instead of ``py::bytes``, masked by the legacy
behavior. These are probably very easy to fix, by changing from
``py::str`` to ``py::bytes``.
* Reliance on py::isinstance<str>(obj) being ``true`` for
``py::bytes``. This is likely to be easy to fix in most cases by adding
``|| py::isinstance<bytes>(obj)``, but a fix may be more involved, e.g. if
``py::isinstance<T>`` appears in a template. Such situations will require
careful review and custom fixes.
.. _upgrade-guide-2.6: .. _upgrade-guide-2.6:
v2.6 v2.6
@ -192,7 +274,7 @@ way to get and set object state. See :ref:`pickling` for details.
... ...
.def(py::pickle( .def(py::pickle(
[](const Foo &self) { // __getstate__ [](const Foo &self) { // __getstate__
return py::make_tuple(f.value1(), f.value2(), ...); // unchanged return py::make_tuple(self.value1(), self.value2(), ...); // unchanged
}, },
[](py::tuple t) { // __setstate__, note: no `self` argument [](py::tuple t) { // __setstate__, note: no `self` argument
return new Foo(t[0].cast<std::string>(), ...); return new Foo(t[0].cast<std::string>(), ...);
@ -256,7 +338,7 @@ Within pybind11's CMake build system, ``pybind11_add_module`` has always been
setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's
being applied unconditionally, even in debug mode and it can no longer be opted being applied unconditionally, even in debug mode and it can no longer be opted
out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also
adds this flag to it's interface. The ``pybind11::embed`` target is unchanged. adds this flag to its interface. The ``pybind11::embed`` target is unchanged.
The most significant change here is for the ``pybind11::module`` target. If you The most significant change here is for the ``pybind11::module`` target. If you
were previously relying on default visibility, i.e. if your Python module was were previously relying on default visibility, i.e. if your Python module was
@ -484,7 +566,7 @@ include a declaration of the form:
PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>) PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)
Continuing to do so wont cause an error or even a deprecation warning, Continuing to do so won't cause an error or even a deprecation warning,
but it's completely redundant. but it's completely redundant.

View File

@ -10,72 +10,116 @@
#pragma once #pragma once
#include "detail/common.h"
#include "cast.h" #include "cast.h"
#include <functional>
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
/// \addtogroup annotations /// \addtogroup annotations
/// @{ /// @{
/// Annotation for methods /// Annotation for methods
struct is_method { handle class_; is_method(const handle &c) : class_(c) { } }; struct is_method {
handle class_;
explicit is_method(const handle &c) : class_(c) {}
};
/// Annotation for setters
struct is_setter {};
/// Annotation for operators /// Annotation for operators
struct is_operator { }; struct is_operator {};
/// Annotation for classes that cannot be subclassed /// Annotation for classes that cannot be subclassed
struct is_final { }; struct is_final {};
/// Annotation for parent scope /// Annotation for parent scope
struct scope { handle value; scope(const handle &s) : value(s) { } }; struct scope {
handle value;
explicit scope(const handle &s) : value(s) {}
};
/// Annotation for documentation /// Annotation for documentation
struct doc { const char *value; doc(const char *value) : value(value) { } }; struct doc {
const char *value;
explicit doc(const char *value) : value(value) {}
};
/// Annotation for function names /// Annotation for function names
struct name { const char *value; name(const char *value) : value(value) { } }; struct name {
const char *value;
explicit name(const char *value) : value(value) {}
};
/// Annotation indicating that a function is an overload associated with a given "sibling" /// Annotation indicating that a function is an overload associated with a given "sibling"
struct sibling { handle value; sibling(const handle &value) : value(value.ptr()) { } }; struct sibling {
handle value;
explicit sibling(const handle &value) : value(value.ptr()) {}
};
/// Annotation indicating that a class derives from another given type /// Annotation indicating that a class derives from another given type
template <typename T> struct base { template <typename T>
struct base {
PYBIND11_DEPRECATED("base<T>() was deprecated in favor of specifying 'T' as a template argument to class_") PYBIND11_DEPRECATED(
base() { } // NOLINT(modernize-use-equals-default): breaks MSVC 2015 when adding an attribute "base<T>() was deprecated in favor of specifying 'T' as a template argument to class_")
base() = default;
}; };
/// Keep patient alive while nurse lives /// Keep patient alive while nurse lives
template <size_t Nurse, size_t Patient> struct keep_alive { }; template <size_t Nurse, size_t Patient>
struct keep_alive {};
/// Annotation indicating that a class is involved in a multiple inheritance relationship /// Annotation indicating that a class is involved in a multiple inheritance relationship
struct multiple_inheritance { }; struct multiple_inheritance {};
/// Annotation which enables dynamic attributes, i.e. adds `__dict__` to a class /// Annotation which enables dynamic attributes, i.e. adds `__dict__` to a class
struct dynamic_attr { }; struct dynamic_attr {};
/// Annotation which enables the buffer protocol for a type /// Annotation which enables the buffer protocol for a type
struct buffer_protocol { }; struct buffer_protocol {};
/// Annotation which requests that a special metaclass is created for a type /// Annotation which requests that a special metaclass is created for a type
struct metaclass { struct metaclass {
handle value; handle value;
PYBIND11_DEPRECATED("py::metaclass() is no longer required. It's turned on by default now.") PYBIND11_DEPRECATED("py::metaclass() is no longer required. It's turned on by default now.")
metaclass() { } // NOLINT(modernize-use-equals-default): breaks MSVC 2015 when adding an attribute metaclass() = default;
/// Override pybind11's default metaclass /// Override pybind11's default metaclass
explicit metaclass(handle value) : value(value) { } explicit metaclass(handle value) : value(value) {}
};
/// Specifies a custom callback with signature `void (PyHeapTypeObject*)` that
/// may be used to customize the Python type.
///
/// The callback is invoked immediately before `PyType_Ready`.
///
/// Note: This is an advanced interface, and uses of it may require changes to
/// work with later versions of pybind11. You may wish to consult the
/// implementation of `make_new_python_type` in `detail/classes.h` to understand
/// the context in which the callback will be run.
struct custom_type_setup {
using callback = std::function<void(PyHeapTypeObject *heap_type)>;
explicit custom_type_setup(callback value) : value(std::move(value)) {}
callback value;
}; };
/// Annotation that marks a class as local to the module: /// Annotation that marks a class as local to the module:
struct module_local { const bool value; constexpr module_local(bool v = true) : value(v) { } }; struct module_local {
const bool value;
constexpr explicit module_local(bool v = true) : value(v) {}
};
/// Annotation to mark enums as an arithmetic type /// Annotation to mark enums as an arithmetic type
struct arithmetic { }; struct arithmetic {};
/// Mark a function for addition at the beginning of the existing overload chain instead of the end /// Mark a function for addition at the beginning of the existing overload chain instead of the end
struct prepend { }; struct prepend {};
/** \rst /** \rst
A call policy which places one or more guard variables (``Ts...``) around the function call. A call policy which places one or more guard variables (``Ts...``) around the function call.
@ -95,9 +139,13 @@ struct prepend { };
return foo(args...); // forwarded arguments return foo(args...); // forwarded arguments
}); });
\endrst */ \endrst */
template <typename... Ts> struct call_guard; template <typename... Ts>
struct call_guard;
template <> struct call_guard<> { using type = detail::void_type; }; template <>
struct call_guard<> {
using type = detail::void_type;
};
template <typename T> template <typename T>
struct call_guard<T> { struct call_guard<T> {
@ -122,8 +170,9 @@ PYBIND11_NAMESPACE_BEGIN(detail)
enum op_id : int; enum op_id : int;
enum op_type : int; enum op_type : int;
struct undefined_t; struct undefined_t;
template <op_id id, op_type ot, typename L = undefined_t, typename R = undefined_t> struct op_; template <op_id id, op_type ot, typename L = undefined_t, typename R = undefined_t>
inline void keep_alive_impl(size_t Nurse, size_t Patient, function_call &call, handle ret); struct op_;
void keep_alive_impl(size_t Nurse, size_t Patient, function_call &call, handle ret);
/// Internal data structure which holds metadata about a keyword argument /// Internal data structure which holds metadata about a keyword argument
struct argument_record { struct argument_record {
@ -134,15 +183,16 @@ struct argument_record {
bool none : 1; ///< True if None is allowed when loading bool none : 1; ///< True if None is allowed when loading
argument_record(const char *name, const char *descr, handle value, bool convert, bool none) argument_record(const char *name, const char *descr, handle value, bool convert, bool none)
: name(name), descr(descr), value(value), convert(convert), none(none) { } : name(name), descr(descr), value(value), convert(convert), none(none) {}
}; };
/// Internal data structure which holds metadata about a bound function (signature, overloads, etc.) /// Internal data structure which holds metadata about a bound function (signature, overloads,
/// etc.)
struct function_record { struct function_record {
function_record() function_record()
: is_constructor(false), is_new_style_constructor(false), is_stateless(false), : is_constructor(false), is_new_style_constructor(false), is_stateless(false),
is_operator(false), is_method(false), has_args(false), is_operator(false), is_method(false), is_setter(false), has_args(false),
has_kwargs(false), has_kw_only_args(false), prepend(false) { } has_kwargs(false), prepend(false) {}
/// Function name /// Function name
char *name = nullptr; /* why no C++ strings? They generate heavier code.. */ char *name = nullptr; /* why no C++ strings? They generate heavier code.. */
@ -157,13 +207,13 @@ struct function_record {
std::vector<argument_record> args; std::vector<argument_record> args;
/// Pointer to lambda function which converts arguments and performs the actual call /// Pointer to lambda function which converts arguments and performs the actual call
handle (*impl) (function_call &) = nullptr; handle (*impl)(function_call &) = nullptr;
/// Storage for the wrapped function pointer and captured data, if any /// Storage for the wrapped function pointer and captured data, if any
void *data[3] = { }; void *data[3] = {};
/// Pointer to custom destructor for 'data' (if needed) /// Pointer to custom destructor for 'data' (if needed)
void (*free_data) (function_record *ptr) = nullptr; void (*free_data)(function_record *ptr) = nullptr;
/// Return value policy associated with this function /// Return value policy associated with this function
return_value_policy policy = return_value_policy::automatic; return_value_policy policy = return_value_policy::automatic;
@ -183,23 +233,24 @@ struct function_record {
/// True if this is a method /// True if this is a method
bool is_method : 1; bool is_method : 1;
/// True if this is a setter
bool is_setter : 1;
/// True if the function has a '*args' argument /// True if the function has a '*args' argument
bool has_args : 1; bool has_args : 1;
/// True if the function has a '**kwargs' argument /// True if the function has a '**kwargs' argument
bool has_kwargs : 1; bool has_kwargs : 1;
/// True once a 'py::kw_only' is encountered (any following args are keyword-only)
bool has_kw_only_args : 1;
/// True if this function is to be inserted at the beginning of the overload resolution chain /// True if this function is to be inserted at the beginning of the overload resolution chain
bool prepend : 1; bool prepend : 1;
/// Number of arguments (including py::args and/or py::kwargs, if present) /// Number of arguments (including py::args and/or py::kwargs, if present)
std::uint16_t nargs; std::uint16_t nargs;
/// Number of trailing arguments (counted in `nargs`) that are keyword-only /// Number of leading positional arguments, which are terminated by a py::args or py::kwargs
std::uint16_t nargs_kw_only = 0; /// argument or by a py::kw_only annotation.
std::uint16_t nargs_pos = 0;
/// Number of leading arguments (counted in `nargs`) that are positional-only /// Number of leading arguments (counted in `nargs`) that are positional-only
std::uint16_t nargs_pos_only = 0; std::uint16_t nargs_pos_only = 0;
@ -221,7 +272,7 @@ struct function_record {
struct type_record { struct type_record {
PYBIND11_NOINLINE type_record() PYBIND11_NOINLINE type_record()
: multiple_inheritance(false), dynamic_attr(false), buffer_protocol(false), : multiple_inheritance(false), dynamic_attr(false), buffer_protocol(false),
default_holder(true), module_local(false), is_final(false) { } default_holder(true), module_local(false), is_final(false) {}
/// Handle to the parent scope /// Handle to the parent scope
handle scope; handle scope;
@ -259,6 +310,9 @@ struct type_record {
/// Custom metaclass (optional) /// Custom metaclass (optional)
handle metaclass; handle metaclass;
/// Custom type setup.
custom_type_setup::callback custom_type_setup_callback;
/// Multiple inheritance marker /// Multiple inheritance marker
bool multiple_inheritance : 1; bool multiple_inheritance : 1;
@ -277,42 +331,45 @@ struct type_record {
/// Is the class inheritable from python classes? /// Is the class inheritable from python classes?
bool is_final : 1; bool is_final : 1;
PYBIND11_NOINLINE void add_base(const std::type_info &base, void *(*caster)(void *)) { PYBIND11_NOINLINE void add_base(const std::type_info &base, void *(*caster)(void *) ) {
auto base_info = detail::get_type_info(base, false); auto *base_info = detail::get_type_info(base, false);
if (!base_info) { if (!base_info) {
std::string tname(base.name()); std::string tname(base.name());
detail::clean_type_id(tname); detail::clean_type_id(tname);
pybind11_fail("generic_type: type \"" + std::string(name) + pybind11_fail("generic_type: type \"" + std::string(name)
"\" referenced unknown base type \"" + tname + "\""); + "\" referenced unknown base type \"" + tname + "\"");
} }
if (default_holder != base_info->default_holder) { if (default_holder != base_info->default_holder) {
std::string tname(base.name()); std::string tname(base.name());
detail::clean_type_id(tname); detail::clean_type_id(tname);
pybind11_fail("generic_type: type \"" + std::string(name) + "\" " + pybind11_fail("generic_type: type \"" + std::string(name) + "\" "
(default_holder ? "does not have" : "has") + + (default_holder ? "does not have" : "has")
" a non-default holder type while its base \"" + tname + "\" " + + " a non-default holder type while its base \"" + tname + "\" "
(base_info->default_holder ? "does not" : "does")); + (base_info->default_holder ? "does not" : "does"));
} }
bases.append((PyObject *) base_info->type); bases.append((PyObject *) base_info->type);
if (base_info->type->tp_dictoffset != 0) #if PY_VERSION_HEX < 0x030B0000
dynamic_attr = true; dynamic_attr |= base_info->type->tp_dictoffset != 0;
#else
dynamic_attr |= (base_info->type->tp_flags & Py_TPFLAGS_MANAGED_DICT) != 0;
#endif
if (caster) if (caster) {
base_info->implicit_casts.emplace_back(type, caster); base_info->implicit_casts.emplace_back(type, caster);
}
} }
}; };
inline function_call::function_call(const function_record &f, handle p) : inline function_call::function_call(const function_record &f, handle p) : func(f), parent(p) {
func(f), parent(p) {
args.reserve(f.nargs); args.reserve(f.nargs);
args_convert.reserve(f.nargs); args_convert.reserve(f.nargs);
} }
/// Tag for a new-style `__init__` defined in `detail/init.h` /// Tag for a new-style `__init__` defined in `detail/init.h`
struct is_new_style_constructor { }; struct is_new_style_constructor {};
/** /**
* Partial template specializations to process custom attributes provided to * Partial template specializations to process custom attributes provided to
@ -320,129 +377,183 @@ struct is_new_style_constructor { };
* fields in the type_record and function_record data structures or executed at * fields in the type_record and function_record data structures or executed at
* runtime to deal with custom call policies (e.g. keep_alive). * runtime to deal with custom call policies (e.g. keep_alive).
*/ */
template <typename T, typename SFINAE = void> struct process_attribute; template <typename T, typename SFINAE = void>
struct process_attribute;
template <typename T> struct process_attribute_default { template <typename T>
struct process_attribute_default {
/// Default implementation: do nothing /// Default implementation: do nothing
static void init(const T &, function_record *) { } static void init(const T &, function_record *) {}
static void init(const T &, type_record *) { } static void init(const T &, type_record *) {}
static void precall(function_call &) { } static void precall(function_call &) {}
static void postcall(function_call &, handle) { } static void postcall(function_call &, handle) {}
}; };
/// Process an attribute specifying the function's name /// Process an attribute specifying the function's name
template <> struct process_attribute<name> : process_attribute_default<name> { template <>
struct process_attribute<name> : process_attribute_default<name> {
static void init(const name &n, function_record *r) { r->name = const_cast<char *>(n.value); } static void init(const name &n, function_record *r) { r->name = const_cast<char *>(n.value); }
}; };
/// Process an attribute specifying the function's docstring /// Process an attribute specifying the function's docstring
template <> struct process_attribute<doc> : process_attribute_default<doc> { template <>
struct process_attribute<doc> : process_attribute_default<doc> {
static void init(const doc &n, function_record *r) { r->doc = const_cast<char *>(n.value); } static void init(const doc &n, function_record *r) { r->doc = const_cast<char *>(n.value); }
}; };
/// Process an attribute specifying the function's docstring (provided as a C-style string) /// Process an attribute specifying the function's docstring (provided as a C-style string)
template <> struct process_attribute<const char *> : process_attribute_default<const char *> { template <>
struct process_attribute<const char *> : process_attribute_default<const char *> {
static void init(const char *d, function_record *r) { r->doc = const_cast<char *>(d); } static void init(const char *d, function_record *r) { r->doc = const_cast<char *>(d); }
static void init(const char *d, type_record *r) { r->doc = const_cast<char *>(d); } static void init(const char *d, type_record *r) { r->doc = d; }
}; };
template <> struct process_attribute<char *> : process_attribute<const char *> { }; template <>
struct process_attribute<char *> : process_attribute<const char *> {};
/// Process an attribute indicating the function's return value policy /// Process an attribute indicating the function's return value policy
template <> struct process_attribute<return_value_policy> : process_attribute_default<return_value_policy> { template <>
struct process_attribute<return_value_policy> : process_attribute_default<return_value_policy> {
static void init(const return_value_policy &p, function_record *r) { r->policy = p; } static void init(const return_value_policy &p, function_record *r) { r->policy = p; }
}; };
/// Process an attribute which indicates that this is an overloaded function associated with a given sibling /// Process an attribute which indicates that this is an overloaded function associated with a
template <> struct process_attribute<sibling> : process_attribute_default<sibling> { /// given sibling
template <>
struct process_attribute<sibling> : process_attribute_default<sibling> {
static void init(const sibling &s, function_record *r) { r->sibling = s.value; } static void init(const sibling &s, function_record *r) { r->sibling = s.value; }
}; };
/// Process an attribute which indicates that this function is a method /// Process an attribute which indicates that this function is a method
template <> struct process_attribute<is_method> : process_attribute_default<is_method> { template <>
static void init(const is_method &s, function_record *r) { r->is_method = true; r->scope = s.class_; } struct process_attribute<is_method> : process_attribute_default<is_method> {
static void init(const is_method &s, function_record *r) {
r->is_method = true;
r->scope = s.class_;
}
};
/// Process an attribute which indicates that this function is a setter
template <>
struct process_attribute<is_setter> : process_attribute_default<is_setter> {
static void init(const is_setter &, function_record *r) { r->is_setter = true; }
}; };
/// Process an attribute which indicates the parent scope of a method /// Process an attribute which indicates the parent scope of a method
template <> struct process_attribute<scope> : process_attribute_default<scope> { template <>
struct process_attribute<scope> : process_attribute_default<scope> {
static void init(const scope &s, function_record *r) { r->scope = s.value; } static void init(const scope &s, function_record *r) { r->scope = s.value; }
}; };
/// Process an attribute which indicates that this function is an operator /// Process an attribute which indicates that this function is an operator
template <> struct process_attribute<is_operator> : process_attribute_default<is_operator> { template <>
struct process_attribute<is_operator> : process_attribute_default<is_operator> {
static void init(const is_operator &, function_record *r) { r->is_operator = true; } static void init(const is_operator &, function_record *r) { r->is_operator = true; }
}; };
template <> struct process_attribute<is_new_style_constructor> : process_attribute_default<is_new_style_constructor> { template <>
static void init(const is_new_style_constructor &, function_record *r) { r->is_new_style_constructor = true; } struct process_attribute<is_new_style_constructor>
: process_attribute_default<is_new_style_constructor> {
static void init(const is_new_style_constructor &, function_record *r) {
r->is_new_style_constructor = true;
}
}; };
inline void process_kw_only_arg(const arg &a, function_record *r) { inline void check_kw_only_arg(const arg &a, function_record *r) {
if (!a.name || strlen(a.name) == 0) if (r->args.size() > r->nargs_pos && (!a.name || a.name[0] == '\0')) {
pybind11_fail("arg(): cannot specify an unnamed argument after an kw_only() annotation"); pybind11_fail("arg(): cannot specify an unnamed argument after a kw_only() annotation or "
++r->nargs_kw_only; "args() argument");
}
}
inline void append_self_arg_if_needed(function_record *r) {
if (r->is_method && r->args.empty()) {
r->args.emplace_back("self", nullptr, handle(), /*convert=*/true, /*none=*/false);
}
} }
/// Process a keyword argument attribute (*without* a default value) /// Process a keyword argument attribute (*without* a default value)
template <> struct process_attribute<arg> : process_attribute_default<arg> { template <>
struct process_attribute<arg> : process_attribute_default<arg> {
static void init(const arg &a, function_record *r) { static void init(const arg &a, function_record *r) {
if (r->is_method && r->args.empty()) append_self_arg_if_needed(r);
r->args.emplace_back("self", nullptr, handle(), true /*convert*/, false /*none not allowed*/);
r->args.emplace_back(a.name, nullptr, handle(), !a.flag_noconvert, a.flag_none); r->args.emplace_back(a.name, nullptr, handle(), !a.flag_noconvert, a.flag_none);
if (r->has_kw_only_args) process_kw_only_arg(a, r); check_kw_only_arg(a, r);
} }
}; };
/// Process a keyword argument attribute (*with* a default value) /// Process a keyword argument attribute (*with* a default value)
template <> struct process_attribute<arg_v> : process_attribute_default<arg_v> { template <>
struct process_attribute<arg_v> : process_attribute_default<arg_v> {
static void init(const arg_v &a, function_record *r) { static void init(const arg_v &a, function_record *r) {
if (r->is_method && r->args.empty()) if (r->is_method && r->args.empty()) {
r->args.emplace_back("self", nullptr /*descr*/, handle() /*parent*/, true /*convert*/, false /*none not allowed*/); r->args.emplace_back(
"self", /*descr=*/nullptr, /*parent=*/handle(), /*convert=*/true, /*none=*/false);
}
if (!a.value) { if (!a.value) {
#if !defined(NDEBUG) #if defined(PYBIND11_DETAILED_ERROR_MESSAGES)
std::string descr("'"); std::string descr("'");
if (a.name) descr += std::string(a.name) + ": "; if (a.name) {
descr += std::string(a.name) + ": ";
}
descr += a.type + "'"; descr += a.type + "'";
if (r->is_method) { if (r->is_method) {
if (r->name) if (r->name) {
descr += " in method '" + (std::string) str(r->scope) + "." + (std::string) r->name + "'"; descr += " in method '" + (std::string) str(r->scope) + "."
else + (std::string) r->name + "'";
} else {
descr += " in method of '" + (std::string) str(r->scope) + "'"; descr += " in method of '" + (std::string) str(r->scope) + "'";
}
} else if (r->name) { } else if (r->name) {
descr += " in function '" + (std::string) r->name + "'"; descr += " in function '" + (std::string) r->name + "'";
} }
pybind11_fail("arg(): could not convert default argument " pybind11_fail("arg(): could not convert default argument " + descr
+ descr + " into a Python object (type not registered yet?)"); + " into a Python object (type not registered yet?)");
#else #else
pybind11_fail("arg(): could not convert default argument " pybind11_fail("arg(): could not convert default argument "
"into a Python object (type not registered yet?). " "into a Python object (type not registered yet?). "
"Compile in debug mode for more information."); "#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for "
"more information.");
#endif #endif
} }
r->args.emplace_back(a.name, a.descr, a.value.inc_ref(), !a.flag_noconvert, a.flag_none); r->args.emplace_back(a.name, a.descr, a.value.inc_ref(), !a.flag_noconvert, a.flag_none);
if (r->has_kw_only_args) process_kw_only_arg(a, r); check_kw_only_arg(a, r);
} }
}; };
/// Process a keyword-only-arguments-follow pseudo argument /// Process a keyword-only-arguments-follow pseudo argument
template <> struct process_attribute<kw_only> : process_attribute_default<kw_only> { template <>
struct process_attribute<kw_only> : process_attribute_default<kw_only> {
static void init(const kw_only &, function_record *r) { static void init(const kw_only &, function_record *r) {
r->has_kw_only_args = true; append_self_arg_if_needed(r);
if (r->has_args && r->nargs_pos != static_cast<std::uint16_t>(r->args.size())) {
pybind11_fail("Mismatched args() and kw_only(): they must occur at the same relative "
"argument location (or omit kw_only() entirely)");
}
r->nargs_pos = static_cast<std::uint16_t>(r->args.size());
} }
}; };
/// Process a positional-only-argument maker /// Process a positional-only-argument maker
template <> struct process_attribute<pos_only> : process_attribute_default<pos_only> { template <>
struct process_attribute<pos_only> : process_attribute_default<pos_only> {
static void init(const pos_only &, function_record *r) { static void init(const pos_only &, function_record *r) {
append_self_arg_if_needed(r);
r->nargs_pos_only = static_cast<std::uint16_t>(r->args.size()); r->nargs_pos_only = static_cast<std::uint16_t>(r->args.size());
if (r->nargs_pos_only > r->nargs_pos) {
pybind11_fail("pos_only(): cannot follow a py::args() argument");
}
// It also can't follow a kw_only, but a static_assert in pybind11.h checks that
} }
}; };
/// Process a parent class attribute. Single inheritance only (class_ itself already guarantees that) /// Process a parent class attribute. Single inheritance only (class_ itself already guarantees
/// that)
template <typename T> template <typename T>
struct process_attribute<T, enable_if_t<is_pyobject<T>::value>> : process_attribute_default<handle> { struct process_attribute<T, enable_if_t<is_pyobject<T>::value>>
: process_attribute_default<handle> {
static void init(const handle &h, type_record *r) { r->bases.append(h); } static void init(const handle &h, type_record *r) { r->bases.append(h); }
}; };
@ -455,7 +566,9 @@ struct process_attribute<base<T>> : process_attribute_default<base<T>> {
/// Process a multiple inheritance attribute /// Process a multiple inheritance attribute
template <> template <>
struct process_attribute<multiple_inheritance> : process_attribute_default<multiple_inheritance> { struct process_attribute<multiple_inheritance> : process_attribute_default<multiple_inheritance> {
static void init(const multiple_inheritance &, type_record *r) { r->multiple_inheritance = true; } static void init(const multiple_inheritance &, type_record *r) {
r->multiple_inheritance = true;
}
}; };
template <> template <>
@ -463,6 +576,13 @@ struct process_attribute<dynamic_attr> : process_attribute_default<dynamic_attr>
static void init(const dynamic_attr &, type_record *r) { r->dynamic_attr = true; } static void init(const dynamic_attr &, type_record *r) { r->dynamic_attr = true; }
}; };
template <>
struct process_attribute<custom_type_setup> {
static void init(const custom_type_setup &value, type_record *r) {
r->custom_type_setup_callback = value.value;
}
};
template <> template <>
struct process_attribute<is_final> : process_attribute_default<is_final> { struct process_attribute<is_final> : process_attribute_default<is_final> {
static void init(const is_final &, type_record *r) { r->is_final = true; } static void init(const is_final &, type_record *r) { r->is_final = true; }
@ -494,41 +614,59 @@ template <>
struct process_attribute<arithmetic> : process_attribute_default<arithmetic> {}; struct process_attribute<arithmetic> : process_attribute_default<arithmetic> {};
template <typename... Ts> template <typename... Ts>
struct process_attribute<call_guard<Ts...>> : process_attribute_default<call_guard<Ts...>> { }; struct process_attribute<call_guard<Ts...>> : process_attribute_default<call_guard<Ts...>> {};
/** /**
* Process a keep_alive call policy -- invokes keep_alive_impl during the * Process a keep_alive call policy -- invokes keep_alive_impl during the
* pre-call handler if both Nurse, Patient != 0 and use the post-call handler * pre-call handler if both Nurse, Patient != 0 and use the post-call handler
* otherwise * otherwise
*/ */
template <size_t Nurse, size_t Patient> struct process_attribute<keep_alive<Nurse, Patient>> : public process_attribute_default<keep_alive<Nurse, Patient>> { template <size_t Nurse, size_t Patient>
struct process_attribute<keep_alive<Nurse, Patient>>
: public process_attribute_default<keep_alive<Nurse, Patient>> {
template <size_t N = Nurse, size_t P = Patient, enable_if_t<N != 0 && P != 0, int> = 0> template <size_t N = Nurse, size_t P = Patient, enable_if_t<N != 0 && P != 0, int> = 0>
static void precall(function_call &call) { keep_alive_impl(Nurse, Patient, call, handle()); } static void precall(function_call &call) {
keep_alive_impl(Nurse, Patient, call, handle());
}
template <size_t N = Nurse, size_t P = Patient, enable_if_t<N != 0 && P != 0, int> = 0> template <size_t N = Nurse, size_t P = Patient, enable_if_t<N != 0 && P != 0, int> = 0>
static void postcall(function_call &, handle) { } static void postcall(function_call &, handle) {}
template <size_t N = Nurse, size_t P = Patient, enable_if_t<N == 0 || P == 0, int> = 0> template <size_t N = Nurse, size_t P = Patient, enable_if_t<N == 0 || P == 0, int> = 0>
static void precall(function_call &) { } static void precall(function_call &) {}
template <size_t N = Nurse, size_t P = Patient, enable_if_t<N == 0 || P == 0, int> = 0> template <size_t N = Nurse, size_t P = Patient, enable_if_t<N == 0 || P == 0, int> = 0>
static void postcall(function_call &call, handle ret) { keep_alive_impl(Nurse, Patient, call, ret); } static void postcall(function_call &call, handle ret) {
keep_alive_impl(Nurse, Patient, call, ret);
}
}; };
/// Recursively iterate over variadic template arguments /// Recursively iterate over variadic template arguments
template <typename... Args> struct process_attributes { template <typename... Args>
static void init(const Args&... args, function_record *r) { struct process_attributes {
int unused[] = { 0, (process_attribute<typename std::decay<Args>::type>::init(args, r), 0) ... }; static void init(const Args &...args, function_record *r) {
ignore_unused(unused); PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(r);
PYBIND11_WORKAROUND_INCORRECT_GCC_UNUSED_BUT_SET_PARAMETER(r);
using expander = int[];
(void) expander{
0, ((void) process_attribute<typename std::decay<Args>::type>::init(args, r), 0)...};
} }
static void init(const Args&... args, type_record *r) { static void init(const Args &...args, type_record *r) {
int unused[] = { 0, (process_attribute<typename std::decay<Args>::type>::init(args, r), 0) ... }; PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(r);
ignore_unused(unused); PYBIND11_WORKAROUND_INCORRECT_GCC_UNUSED_BUT_SET_PARAMETER(r);
using expander = int[];
(void) expander{0,
(process_attribute<typename std::decay<Args>::type>::init(args, r), 0)...};
} }
static void precall(function_call &call) { static void precall(function_call &call) {
int unused[] = { 0, (process_attribute<typename std::decay<Args>::type>::precall(call), 0) ... }; PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(call);
ignore_unused(unused); using expander = int[];
(void) expander{0,
(process_attribute<typename std::decay<Args>::type>::precall(call), 0)...};
} }
static void postcall(function_call &call, handle fn_ret) { static void postcall(function_call &call, handle fn_ret) {
int unused[] = { 0, (process_attribute<typename std::decay<Args>::type>::postcall(call, fn_ret), 0) ... }; PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(call, fn_ret);
ignore_unused(unused); PYBIND11_WORKAROUND_INCORRECT_GCC_UNUSED_BUT_SET_PARAMETER(fn_ret);
using expander = int[];
(void) expander{
0, (process_attribute<typename std::decay<Args>::type>::postcall(call, fn_ret), 0)...};
} }
}; };
@ -542,9 +680,10 @@ using extract_guard_t = typename exactly_one_t<is_call_guard, call_guard<>, Extr
/// Check the number of named arguments at compile time /// Check the number of named arguments at compile time
template <typename... Extra, template <typename... Extra,
size_t named = constexpr_sum(std::is_base_of<arg, Extra>::value...), size_t named = constexpr_sum(std::is_base_of<arg, Extra>::value...),
size_t self = constexpr_sum(std::is_same<is_method, Extra>::value...)> size_t self = constexpr_sum(std::is_same<is_method, Extra>::value...)>
constexpr bool expected_num_args(size_t nargs, bool has_args, bool has_kwargs) { constexpr bool expected_num_args(size_t nargs, bool has_args, bool has_kwargs) {
return named == 0 || (self + named + has_args + has_kwargs) == nargs; PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(nargs, has_args, has_kwargs);
return named == 0 || (self + named + size_t(has_args) + size_t(has_kwargs)) == nargs;
} }
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)

View File

@ -19,9 +19,11 @@ PYBIND11_NAMESPACE_BEGIN(detail)
inline std::vector<ssize_t> c_strides(const std::vector<ssize_t> &shape, ssize_t itemsize) { inline std::vector<ssize_t> c_strides(const std::vector<ssize_t> &shape, ssize_t itemsize) {
auto ndim = shape.size(); auto ndim = shape.size();
std::vector<ssize_t> strides(ndim, itemsize); std::vector<ssize_t> strides(ndim, itemsize);
if (ndim > 0) if (ndim > 0) {
for (size_t i = ndim - 1; i > 0; --i) for (size_t i = ndim - 1; i > 0; --i) {
strides[i - 1] = strides[i] * shape[i]; strides[i - 1] = strides[i] * shape[i];
}
}
return strides; return strides;
} }
@ -29,11 +31,15 @@ inline std::vector<ssize_t> c_strides(const std::vector<ssize_t> &shape, ssize_t
inline std::vector<ssize_t> f_strides(const std::vector<ssize_t> &shape, ssize_t itemsize) { inline std::vector<ssize_t> f_strides(const std::vector<ssize_t> &shape, ssize_t itemsize) {
auto ndim = shape.size(); auto ndim = shape.size();
std::vector<ssize_t> strides(ndim, itemsize); std::vector<ssize_t> strides(ndim, itemsize);
for (size_t i = 1; i < ndim; ++i) for (size_t i = 1; i < ndim; ++i) {
strides[i] = strides[i - 1] * shape[i - 1]; strides[i] = strides[i - 1] * shape[i - 1];
}
return strides; return strides;
} }
template <typename T, typename SFINAE = void>
struct compare_buffer_info;
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
/// Information record describing a Python buffer object /// Information record describing a Python buffer object
@ -41,61 +47,89 @@ struct buffer_info {
void *ptr = nullptr; // Pointer to the underlying storage void *ptr = nullptr; // Pointer to the underlying storage
ssize_t itemsize = 0; // Size of individual items in bytes ssize_t itemsize = 0; // Size of individual items in bytes
ssize_t size = 0; // Total number of entries ssize_t size = 0; // Total number of entries
std::string format; // For homogeneous buffers, this should be set to format_descriptor<T>::format() std::string format; // For homogeneous buffers, this should be set to
// format_descriptor<T>::format()
ssize_t ndim = 0; // Number of dimensions ssize_t ndim = 0; // Number of dimensions
std::vector<ssize_t> shape; // Shape of the tensor (1 entry per dimension) std::vector<ssize_t> shape; // Shape of the tensor (1 entry per dimension)
std::vector<ssize_t> strides; // Number of bytes between adjacent entries (for each per dimension) std::vector<ssize_t> strides; // Number of bytes between adjacent entries
// (for each per dimension)
bool readonly = false; // flag to indicate if the underlying storage may be written to bool readonly = false; // flag to indicate if the underlying storage may be written to
buffer_info() = default; buffer_info() = default;
buffer_info(void *ptr, ssize_t itemsize, const std::string &format, ssize_t ndim, buffer_info(void *ptr,
detail::any_container<ssize_t> shape_in, detail::any_container<ssize_t> strides_in, bool readonly=false) ssize_t itemsize,
: ptr(ptr), itemsize(itemsize), size(1), format(format), ndim(ndim), const std::string &format,
shape(std::move(shape_in)), strides(std::move(strides_in)), readonly(readonly) { ssize_t ndim,
if (ndim != (ssize_t) shape.size() || ndim != (ssize_t) strides.size()) detail::any_container<ssize_t> shape_in,
detail::any_container<ssize_t> strides_in,
bool readonly = false)
: ptr(ptr), itemsize(itemsize), size(1), format(format), ndim(ndim),
shape(std::move(shape_in)), strides(std::move(strides_in)), readonly(readonly) {
if (ndim != (ssize_t) shape.size() || ndim != (ssize_t) strides.size()) {
pybind11_fail("buffer_info: ndim doesn't match shape and/or strides length"); pybind11_fail("buffer_info: ndim doesn't match shape and/or strides length");
for (size_t i = 0; i < (size_t) ndim; ++i) }
for (size_t i = 0; i < (size_t) ndim; ++i) {
size *= shape[i]; size *= shape[i];
}
} }
template <typename T> template <typename T>
buffer_info(T *ptr, detail::any_container<ssize_t> shape_in, detail::any_container<ssize_t> strides_in, bool readonly=false) buffer_info(T *ptr,
: buffer_info(private_ctr_tag(), ptr, sizeof(T), format_descriptor<T>::format(), static_cast<ssize_t>(shape_in->size()), std::move(shape_in), std::move(strides_in), readonly) { } detail::any_container<ssize_t> shape_in,
detail::any_container<ssize_t> strides_in,
bool readonly = false)
: buffer_info(private_ctr_tag(),
ptr,
sizeof(T),
format_descriptor<T>::format(),
static_cast<ssize_t>(shape_in->size()),
std::move(shape_in),
std::move(strides_in),
readonly) {}
buffer_info(void *ptr, ssize_t itemsize, const std::string &format, ssize_t size, bool readonly=false) buffer_info(void *ptr,
: buffer_info(ptr, itemsize, format, 1, {size}, {itemsize}, readonly) { } ssize_t itemsize,
const std::string &format,
ssize_t size,
bool readonly = false)
: buffer_info(ptr, itemsize, format, 1, {size}, {itemsize}, readonly) {}
template <typename T> template <typename T>
buffer_info(T *ptr, ssize_t size, bool readonly=false) buffer_info(T *ptr, ssize_t size, bool readonly = false)
: buffer_info(ptr, sizeof(T), format_descriptor<T>::format(), size, readonly) { } : buffer_info(ptr, sizeof(T), format_descriptor<T>::format(), size, readonly) {}
template <typename T> template <typename T>
buffer_info(const T *ptr, ssize_t size, bool readonly=true) buffer_info(const T *ptr, ssize_t size, bool readonly = true)
: buffer_info(const_cast<T*>(ptr), sizeof(T), format_descriptor<T>::format(), size, readonly) { } : buffer_info(
const_cast<T *>(ptr), sizeof(T), format_descriptor<T>::format(), size, readonly) {}
explicit buffer_info(Py_buffer *view, bool ownview = true) explicit buffer_info(Py_buffer *view, bool ownview = true)
: buffer_info(view->buf, view->itemsize, view->format, view->ndim, : buffer_info(
view->buf,
view->itemsize,
view->format,
view->ndim,
{view->shape, view->shape + view->ndim}, {view->shape, view->shape + view->ndim},
/* Though buffer::request() requests PyBUF_STRIDES, ctypes objects /* Though buffer::request() requests PyBUF_STRIDES, ctypes objects
* ignore this flag and return a view with NULL strides. * ignore this flag and return a view with NULL strides.
* When strides are NULL, build them manually. */ * When strides are NULL, build them manually. */
view->strides view->strides
? std::vector<ssize_t>(view->strides, view->strides + view->ndim) ? std::vector<ssize_t>(view->strides, view->strides + view->ndim)
: detail::c_strides({view->shape, view->shape + view->ndim}, view->itemsize), : detail::c_strides({view->shape, view->shape + view->ndim}, view->itemsize),
view->readonly) { (view->readonly != 0)) {
// NOLINTNEXTLINE(cppcoreguidelines-prefer-member-initializer)
this->m_view = view; this->m_view = view;
// NOLINTNEXTLINE(cppcoreguidelines-prefer-member-initializer)
this->ownview = ownview; this->ownview = ownview;
} }
buffer_info(const buffer_info &) = delete; buffer_info(const buffer_info &) = delete;
buffer_info& operator=(const buffer_info &) = delete; buffer_info &operator=(const buffer_info &) = delete;
buffer_info(buffer_info &&other) { buffer_info(buffer_info &&other) noexcept { (*this) = std::move(other); }
(*this) = std::move(other);
}
buffer_info& operator=(buffer_info &&rhs) { buffer_info &operator=(buffer_info &&rhs) noexcept {
ptr = rhs.ptr; ptr = rhs.ptr;
itemsize = rhs.itemsize; itemsize = rhs.itemsize;
size = rhs.size; size = rhs.size;
@ -110,17 +144,39 @@ struct buffer_info {
} }
~buffer_info() { ~buffer_info() {
if (m_view && ownview) { PyBuffer_Release(m_view); delete m_view; } if (m_view && ownview) {
PyBuffer_Release(m_view);
delete m_view;
}
} }
Py_buffer *view() const { return m_view; } Py_buffer *view() const { return m_view; }
Py_buffer *&view() { return m_view; } Py_buffer *&view() { return m_view; }
private:
struct private_ctr_tag { };
buffer_info(private_ctr_tag, void *ptr, ssize_t itemsize, const std::string &format, ssize_t ndim, /* True if the buffer item type is equivalent to `T`. */
detail::any_container<ssize_t> &&shape_in, detail::any_container<ssize_t> &&strides_in, bool readonly) // To define "equivalent" by example:
: buffer_info(ptr, itemsize, format, ndim, std::move(shape_in), std::move(strides_in), readonly) { } // `buffer_info::item_type_is_equivalent_to<int>(b)` and
// `buffer_info::item_type_is_equivalent_to<long>(b)` may both be true
// on some platforms, but `int` and `unsigned` will never be equivalent.
// For the ground truth, please inspect `detail::compare_buffer_info<>`.
template <typename T>
bool item_type_is_equivalent_to() const {
return detail::compare_buffer_info<T>::compare(*this);
}
private:
struct private_ctr_tag {};
buffer_info(private_ctr_tag,
void *ptr,
ssize_t itemsize,
const std::string &format,
ssize_t ndim,
detail::any_container<ssize_t> &&shape_in,
detail::any_container<ssize_t> &&strides_in,
bool readonly)
: buffer_info(
ptr, itemsize, format, ndim, std::move(shape_in), std::move(strides_in), readonly) {}
Py_buffer *m_view = nullptr; Py_buffer *m_view = nullptr;
bool ownview = false; bool ownview = false;
@ -128,17 +184,23 @@ private:
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
template <typename T, typename SFINAE = void> struct compare_buffer_info { template <typename T, typename SFINAE>
static bool compare(const buffer_info& b) { struct compare_buffer_info {
static bool compare(const buffer_info &b) {
// NOLINTNEXTLINE(bugprone-sizeof-expression) Needed for `PyObject *`
return b.format == format_descriptor<T>::format() && b.itemsize == (ssize_t) sizeof(T); return b.format == format_descriptor<T>::format() && b.itemsize == (ssize_t) sizeof(T);
} }
}; };
template <typename T> struct compare_buffer_info<T, detail::enable_if_t<std::is_integral<T>::value>> { template <typename T>
static bool compare(const buffer_info& b) { struct compare_buffer_info<T, detail::enable_if_t<std::is_integral<T>::value>> {
return (size_t) b.itemsize == sizeof(T) && (b.format == format_descriptor<T>::value || static bool compare(const buffer_info &b) {
((sizeof(T) == sizeof(long)) && b.format == (std::is_unsigned<T>::value ? "L" : "l")) || return (size_t) b.itemsize == sizeof(T)
((sizeof(T) == sizeof(size_t)) && b.format == (std::is_unsigned<T>::value ? "N" : "n"))); && (b.format == format_descriptor<T>::value
|| ((sizeof(T) == sizeof(long))
&& b.format == (std::is_unsigned<T>::value ? "L" : "l"))
|| ((sizeof(T) == sizeof(size_t))
&& b.format == (std::is_unsigned<T>::value ? "N" : "n")));
} }
}; };

File diff suppressed because it is too large Load Diff

View File

@ -11,62 +11,63 @@
#pragma once #pragma once
#include "pybind11.h" #include "pybind11.h"
#include <chrono>
#include <cmath> #include <cmath>
#include <ctime> #include <ctime>
#include <chrono>
#include <datetime.h> #include <datetime.h>
#include <mutex>
// Backport the PyDateTime_DELTA functions from Python3.3 if required
#ifndef PyDateTime_DELTA_GET_DAYS
#define PyDateTime_DELTA_GET_DAYS(o) (((PyDateTime_Delta*)o)->days)
#endif
#ifndef PyDateTime_DELTA_GET_SECONDS
#define PyDateTime_DELTA_GET_SECONDS(o) (((PyDateTime_Delta*)o)->seconds)
#endif
#ifndef PyDateTime_DELTA_GET_MICROSECONDS
#define PyDateTime_DELTA_GET_MICROSECONDS(o) (((PyDateTime_Delta*)o)->microseconds)
#endif
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
template <typename type> class duration_caster { template <typename type>
class duration_caster {
public: public:
using rep = typename type::rep; using rep = typename type::rep;
using period = typename type::period; using period = typename type::period;
using days = std::chrono::duration<uint_fast32_t, std::ratio<86400>>; // signed 25 bits required by the standard.
using days = std::chrono::duration<int_least32_t, std::ratio<86400>>;
bool load(handle src, bool) { bool load(handle src, bool) {
using namespace std::chrono; using namespace std::chrono;
// Lazy initialise the PyDateTime import // Lazy initialise the PyDateTime import
if (!PyDateTimeAPI) { PyDateTime_IMPORT; } if (!PyDateTimeAPI) {
PyDateTime_IMPORT;
}
if (!src) return false; if (!src) {
return false;
}
// If invoked with datetime.delta object // If invoked with datetime.delta object
if (PyDelta_Check(src.ptr())) { if (PyDelta_Check(src.ptr())) {
value = type(duration_cast<duration<rep, period>>( value = type(duration_cast<duration<rep, period>>(
days(PyDateTime_DELTA_GET_DAYS(src.ptr())) days(PyDateTime_DELTA_GET_DAYS(src.ptr()))
+ seconds(PyDateTime_DELTA_GET_SECONDS(src.ptr())) + seconds(PyDateTime_DELTA_GET_SECONDS(src.ptr()))
+ microseconds(PyDateTime_DELTA_GET_MICROSECONDS(src.ptr())))); + microseconds(PyDateTime_DELTA_GET_MICROSECONDS(src.ptr()))));
return true; return true;
} }
// If invoked with a float we assume it is seconds and convert // If invoked with a float we assume it is seconds and convert
else if (PyFloat_Check(src.ptr())) { if (PyFloat_Check(src.ptr())) {
value = type(duration_cast<duration<rep, period>>(duration<double>(PyFloat_AsDouble(src.ptr())))); value = type(duration_cast<duration<rep, period>>(
duration<double>(PyFloat_AsDouble(src.ptr()))));
return true; return true;
} }
else return false; return false;
} }
// If this is a duration just return it back // If this is a duration just return it back
static const std::chrono::duration<rep, period>& get_duration(const std::chrono::duration<rep, period> &src) { static const std::chrono::duration<rep, period> &
get_duration(const std::chrono::duration<rep, period> &src) {
return src; return src;
} }
// If this is a time_point get the time_since_epoch // If this is a time_point get the time_since_epoch
template <typename Clock> static std::chrono::duration<rep, period> get_duration(const std::chrono::time_point<Clock, std::chrono::duration<rep, period>> &src) { template <typename Clock>
static std::chrono::duration<rep, period>
get_duration(const std::chrono::time_point<Clock, std::chrono::duration<rep, period>> &src) {
return src.time_since_epoch(); return src.time_since_epoch();
} }
@ -78,9 +79,12 @@ public:
auto d = get_duration(src); auto d = get_duration(src);
// Lazy initialise the PyDateTime import // Lazy initialise the PyDateTime import
if (!PyDateTimeAPI) { PyDateTime_IMPORT; } if (!PyDateTimeAPI) {
PyDateTime_IMPORT;
}
// Declare these special duration types so the conversions happen with the correct primitive types (int) // Declare these special duration types so the conversions happen with the correct
// primitive types (int)
using dd_t = duration<int, std::ratio<86400>>; using dd_t = duration<int, std::ratio<86400>>;
using ss_t = duration<int, std::ratio<1>>; using ss_t = duration<int, std::ratio<1>>;
using us_t = duration<int, std::micro>; using us_t = duration<int, std::micro>;
@ -92,79 +96,109 @@ public:
return PyDelta_FromDSU(dd.count(), ss.count(), us.count()); return PyDelta_FromDSU(dd.count(), ss.count(), us.count());
} }
PYBIND11_TYPE_CASTER(type, _("datetime.timedelta")); PYBIND11_TYPE_CASTER(type, const_name("datetime.timedelta"));
}; };
inline std::tm *localtime_thread_safe(const std::time_t *time, std::tm *buf) {
#if (defined(__STDC_LIB_EXT1__) && defined(__STDC_WANT_LIB_EXT1__)) || defined(_MSC_VER)
if (localtime_s(buf, time))
return nullptr;
return buf;
#else
static std::mutex mtx;
std::lock_guard<std::mutex> lock(mtx);
std::tm *tm_ptr = std::localtime(time);
if (tm_ptr != nullptr) {
*buf = *tm_ptr;
}
return tm_ptr;
#endif
}
// This is for casting times on the system clock into datetime.datetime instances // This is for casting times on the system clock into datetime.datetime instances
template <typename Duration> class type_caster<std::chrono::time_point<std::chrono::system_clock, Duration>> { template <typename Duration>
class type_caster<std::chrono::time_point<std::chrono::system_clock, Duration>> {
public: public:
using type = std::chrono::time_point<std::chrono::system_clock, Duration>; using type = std::chrono::time_point<std::chrono::system_clock, Duration>;
bool load(handle src, bool) { bool load(handle src, bool) {
using namespace std::chrono; using namespace std::chrono;
// Lazy initialise the PyDateTime import // Lazy initialise the PyDateTime import
if (!PyDateTimeAPI) { PyDateTime_IMPORT; } if (!PyDateTimeAPI) {
PyDateTime_IMPORT;
}
if (!src) return false; if (!src) {
return false;
}
std::tm cal; std::tm cal;
microseconds msecs; microseconds msecs;
if (PyDateTime_Check(src.ptr())) { if (PyDateTime_Check(src.ptr())) {
cal.tm_sec = PyDateTime_DATE_GET_SECOND(src.ptr()); cal.tm_sec = PyDateTime_DATE_GET_SECOND(src.ptr());
cal.tm_min = PyDateTime_DATE_GET_MINUTE(src.ptr()); cal.tm_min = PyDateTime_DATE_GET_MINUTE(src.ptr());
cal.tm_hour = PyDateTime_DATE_GET_HOUR(src.ptr()); cal.tm_hour = PyDateTime_DATE_GET_HOUR(src.ptr());
cal.tm_mday = PyDateTime_GET_DAY(src.ptr()); cal.tm_mday = PyDateTime_GET_DAY(src.ptr());
cal.tm_mon = PyDateTime_GET_MONTH(src.ptr()) - 1; cal.tm_mon = PyDateTime_GET_MONTH(src.ptr()) - 1;
cal.tm_year = PyDateTime_GET_YEAR(src.ptr()) - 1900; cal.tm_year = PyDateTime_GET_YEAR(src.ptr()) - 1900;
cal.tm_isdst = -1; cal.tm_isdst = -1;
msecs = microseconds(PyDateTime_DATE_GET_MICROSECOND(src.ptr())); msecs = microseconds(PyDateTime_DATE_GET_MICROSECOND(src.ptr()));
} else if (PyDate_Check(src.ptr())) { } else if (PyDate_Check(src.ptr())) {
cal.tm_sec = 0; cal.tm_sec = 0;
cal.tm_min = 0; cal.tm_min = 0;
cal.tm_hour = 0; cal.tm_hour = 0;
cal.tm_mday = PyDateTime_GET_DAY(src.ptr()); cal.tm_mday = PyDateTime_GET_DAY(src.ptr());
cal.tm_mon = PyDateTime_GET_MONTH(src.ptr()) - 1; cal.tm_mon = PyDateTime_GET_MONTH(src.ptr()) - 1;
cal.tm_year = PyDateTime_GET_YEAR(src.ptr()) - 1900; cal.tm_year = PyDateTime_GET_YEAR(src.ptr()) - 1900;
cal.tm_isdst = -1; cal.tm_isdst = -1;
msecs = microseconds(0); msecs = microseconds(0);
} else if (PyTime_Check(src.ptr())) { } else if (PyTime_Check(src.ptr())) {
cal.tm_sec = PyDateTime_TIME_GET_SECOND(src.ptr()); cal.tm_sec = PyDateTime_TIME_GET_SECOND(src.ptr());
cal.tm_min = PyDateTime_TIME_GET_MINUTE(src.ptr()); cal.tm_min = PyDateTime_TIME_GET_MINUTE(src.ptr());
cal.tm_hour = PyDateTime_TIME_GET_HOUR(src.ptr()); cal.tm_hour = PyDateTime_TIME_GET_HOUR(src.ptr());
cal.tm_mday = 1; // This date (day, month, year) = (1, 0, 70) cal.tm_mday = 1; // This date (day, month, year) = (1, 0, 70)
cal.tm_mon = 0; // represents 1-Jan-1970, which is the first cal.tm_mon = 0; // represents 1-Jan-1970, which is the first
cal.tm_year = 70; // earliest available date for Python's datetime cal.tm_year = 70; // earliest available date for Python's datetime
cal.tm_isdst = -1; cal.tm_isdst = -1;
msecs = microseconds(PyDateTime_TIME_GET_MICROSECOND(src.ptr())); msecs = microseconds(PyDateTime_TIME_GET_MICROSECOND(src.ptr()));
} else {
return false;
} }
else return false;
value = time_point_cast<Duration>(system_clock::from_time_t(std::mktime(&cal)) + msecs); value = time_point_cast<Duration>(system_clock::from_time_t(std::mktime(&cal)) + msecs);
return true; return true;
} }
static handle cast(const std::chrono::time_point<std::chrono::system_clock, Duration> &src, return_value_policy /* policy */, handle /* parent */) { static handle cast(const std::chrono::time_point<std::chrono::system_clock, Duration> &src,
return_value_policy /* policy */,
handle /* parent */) {
using namespace std::chrono; using namespace std::chrono;
// Lazy initialise the PyDateTime import // Lazy initialise the PyDateTime import
if (!PyDateTimeAPI) { PyDateTime_IMPORT; } if (!PyDateTimeAPI) {
PyDateTime_IMPORT;
}
// Get out microseconds, and make sure they are positive, to avoid bug in eastern hemisphere time zones // Get out microseconds, and make sure they are positive, to avoid bug in eastern
// (cfr. https://github.com/pybind/pybind11/issues/2417) // hemisphere time zones (cfr. https://github.com/pybind/pybind11/issues/2417)
using us_t = duration<int, std::micro>; using us_t = duration<int, std::micro>;
auto us = duration_cast<us_t>(src.time_since_epoch() % seconds(1)); auto us = duration_cast<us_t>(src.time_since_epoch() % seconds(1));
if (us.count() < 0) if (us.count() < 0) {
us += seconds(1); us += seconds(1);
}
// Subtract microseconds BEFORE `system_clock::to_time_t`, because: // Subtract microseconds BEFORE `system_clock::to_time_t`, because:
// > If std::time_t has lower precision, it is implementation-defined whether the value is rounded or truncated. // > If std::time_t has lower precision, it is implementation-defined whether the value is
// (https://en.cppreference.com/w/cpp/chrono/system_clock/to_time_t) // rounded or truncated. (https://en.cppreference.com/w/cpp/chrono/system_clock/to_time_t)
std::time_t tt = system_clock::to_time_t(time_point_cast<system_clock::duration>(src - us)); std::time_t tt
// this function uses static memory so it's best to copy it out asap just in case = system_clock::to_time_t(time_point_cast<system_clock::duration>(src - us));
// otherwise other code that is using localtime may break this (not just python code)
std::tm localtime = *std::localtime(&tt);
std::tm localtime;
std::tm *localtime_ptr = localtime_thread_safe(&tt, &localtime);
if (!localtime_ptr) {
throw cast_error("Unable to represent system_clock in local time");
}
return PyDateTime_FromDateAndTime(localtime.tm_year + 1900, return PyDateTime_FromDateAndTime(localtime.tm_year + 1900,
localtime.tm_mon + 1, localtime.tm_mon + 1,
localtime.tm_mday, localtime.tm_mday,
@ -173,19 +207,19 @@ public:
localtime.tm_sec, localtime.tm_sec,
us.count()); us.count());
} }
PYBIND11_TYPE_CASTER(type, _("datetime.datetime")); PYBIND11_TYPE_CASTER(type, const_name("datetime.datetime"));
}; };
// Other clocks that are not the system clock are not measured as datetime.datetime objects // Other clocks that are not the system clock are not measured as datetime.datetime objects
// since they are not measured on calendar time. So instead we just make them timedeltas // since they are not measured on calendar time. So instead we just make them timedeltas
// Or if they have passed us a time as a float we convert that // Or if they have passed us a time as a float we convert that
template <typename Clock, typename Duration> class type_caster<std::chrono::time_point<Clock, Duration>> template <typename Clock, typename Duration>
: public duration_caster<std::chrono::time_point<Clock, Duration>> { class type_caster<std::chrono::time_point<Clock, Duration>>
}; : public duration_caster<std::chrono::time_point<Clock, Duration>> {};
template <typename Rep, typename Period> class type_caster<std::chrono::duration<Rep, Period>> template <typename Rep, typename Period>
: public duration_caster<std::chrono::duration<Rep, Period>> { class type_caster<std::chrono::duration<Rep, Period>>
}; : public duration_caster<std::chrono::duration<Rep, Period>> {};
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -10,42 +10,50 @@
#pragma once #pragma once
#include "pybind11.h" #include "pybind11.h"
#include <complex> #include <complex>
/// glibc defines I as a macro which breaks things, e.g., boost template names /// glibc defines I as a macro which breaks things, e.g., boost template names
#ifdef I #ifdef I
# undef I # undef I
#endif #endif
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
template <typename T> struct format_descriptor<std::complex<T>, detail::enable_if_t<std::is_floating_point<T>::value>> { template <typename T>
struct format_descriptor<std::complex<T>, detail::enable_if_t<std::is_floating_point<T>::value>> {
static constexpr const char c = format_descriptor<T>::c; static constexpr const char c = format_descriptor<T>::c;
static constexpr const char value[3] = { 'Z', c, '\0' }; static constexpr const char value[3] = {'Z', c, '\0'};
static std::string format() { return std::string(value); } static std::string format() { return std::string(value); }
}; };
#ifndef PYBIND11_CPP17 #ifndef PYBIND11_CPP17
template <typename T> constexpr const char format_descriptor< template <typename T>
std::complex<T>, detail::enable_if_t<std::is_floating_point<T>::value>>::value[3]; constexpr const char
format_descriptor<std::complex<T>,
detail::enable_if_t<std::is_floating_point<T>::value>>::value[3];
#endif #endif
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
template <typename T> struct is_fmt_numeric<std::complex<T>, detail::enable_if_t<std::is_floating_point<T>::value>> { template <typename T>
struct is_fmt_numeric<std::complex<T>, detail::enable_if_t<std::is_floating_point<T>::value>> {
static constexpr bool value = true; static constexpr bool value = true;
static constexpr int index = is_fmt_numeric<T>::index + 3; static constexpr int index = is_fmt_numeric<T>::index + 3;
}; };
template <typename T> class type_caster<std::complex<T>> { template <typename T>
class type_caster<std::complex<T>> {
public: public:
bool load(handle src, bool convert) { bool load(handle src, bool convert) {
if (!src) if (!src) {
return false; return false;
if (!convert && !PyComplex_Check(src.ptr())) }
if (!convert && !PyComplex_Check(src.ptr())) {
return false; return false;
}
Py_complex result = PyComplex_AsCComplex(src.ptr()); Py_complex result = PyComplex_AsCComplex(src.ptr());
if (result.real == -1.0 && PyErr_Occurred()) { if (result.real == -1.0 && PyErr_Occurred()) {
PyErr_Clear(); PyErr_Clear();
@ -55,11 +63,12 @@ public:
return true; return true;
} }
static handle cast(const std::complex<T> &src, return_value_policy /* policy */, handle /* parent */) { static handle
cast(const std::complex<T> &src, return_value_policy /* policy */, handle /* parent */) {
return PyComplex_FromDoubles((double) src.real(), (double) src.imag()); return PyComplex_FromDoubles((double) src.real(), (double) src.imag());
} }
PYBIND11_TYPE_CASTER(std::complex<T>, _("complex")); PYBIND11_TYPE_CASTER(std::complex<T>, const_name("complex"));
}; };
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -15,13 +15,14 @@
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
#if PY_VERSION_HEX >= 0x03030000 && !defined(PYPY_VERSION) #if !defined(PYPY_VERSION)
# define PYBIND11_BUILTIN_QUALNAME # define PYBIND11_BUILTIN_QUALNAME
# define PYBIND11_SET_OLDPY_QUALNAME(obj, nameobj) # define PYBIND11_SET_OLDPY_QUALNAME(obj, nameobj)
#else #else
// In pre-3.3 Python, we still set __qualname__ so that we can produce reliable function type // In PyPy, we still set __qualname__ so that we can produce reliable function type
// signatures; in 3.3+ this macro expands to nothing: // signatures; in CPython this macro expands to nothing:
# define PYBIND11_SET_OLDPY_QUALNAME(obj, nameobj) setattr((PyObject *) obj, "__qualname__", nameobj) # define PYBIND11_SET_OLDPY_QUALNAME(obj, nameobj) \
setattr((PyObject *) obj, "__qualname__", nameobj)
#endif #endif
inline std::string get_fully_qualified_tp_name(PyTypeObject *type) { inline std::string get_fully_qualified_tp_name(PyTypeObject *type) {
@ -54,6 +55,9 @@ extern "C" inline int pybind11_static_set(PyObject *self, PyObject *obj, PyObjec
return PyProperty_Type.tp_descr_set(self, cls, value); return PyProperty_Type.tp_descr_set(self, cls, value);
} }
// Forward declaration to use in `make_static_property_type()`
inline void enable_dynamic_attributes(PyHeapTypeObject *heap_type);
/** A `static_property` is the same as a `property` but the `__get__()` and `__set__()` /** A `static_property` is the same as a `property` but the `__get__()` and `__set__()`
methods are modified to always use the object type instead of a concrete instance. methods are modified to always use the object type instead of a concrete instance.
Return value: New reference. */ Return value: New reference. */
@ -65,24 +69,32 @@ inline PyTypeObject *make_static_property_type() {
issue no Python C API calls which could potentially invoke the issue no Python C API calls which could potentially invoke the
garbage collector (the GC will call type_traverse(), which will in garbage collector (the GC will call type_traverse(), which will in
turn find the newly constructed type in an invalid state) */ turn find the newly constructed type in an invalid state) */
auto heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0); auto *heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0);
if (!heap_type) if (!heap_type) {
pybind11_fail("make_static_property_type(): error allocating type!"); pybind11_fail("make_static_property_type(): error allocating type!");
}
heap_type->ht_name = name_obj.inc_ref().ptr(); heap_type->ht_name = name_obj.inc_ref().ptr();
#ifdef PYBIND11_BUILTIN_QUALNAME # ifdef PYBIND11_BUILTIN_QUALNAME
heap_type->ht_qualname = name_obj.inc_ref().ptr(); heap_type->ht_qualname = name_obj.inc_ref().ptr();
#endif # endif
auto type = &heap_type->ht_type; auto *type = &heap_type->ht_type;
type->tp_name = name; type->tp_name = name;
type->tp_base = type_incref(&PyProperty_Type); type->tp_base = type_incref(&PyProperty_Type);
type->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE; type->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE;
type->tp_descr_get = pybind11_static_get; type->tp_descr_get = pybind11_static_get;
type->tp_descr_set = pybind11_static_set; type->tp_descr_set = pybind11_static_set;
if (PyType_Ready(type) < 0) # if PY_VERSION_HEX >= 0x030C0000
// Since Python-3.12 property-derived types are required to
// have dynamic attributes (to set `__doc__`)
enable_dynamic_attributes(heap_type);
# endif
if (PyType_Ready(type) < 0) {
pybind11_fail("make_static_property_type(): failure in PyType_Ready()!"); pybind11_fail("make_static_property_type(): failure in PyType_Ready()!");
}
setattr((PyObject *) type, "__module__", str("pybind11_builtins")); setattr((PyObject *) type, "__module__", str("pybind11_builtins"));
PYBIND11_SET_OLDPY_QUALNAME(type, name_obj); PYBIND11_SET_OLDPY_QUALNAME(type, name_obj);
@ -98,15 +110,17 @@ inline PyTypeObject *make_static_property_type() {
inline PyTypeObject *make_static_property_type() { inline PyTypeObject *make_static_property_type() {
auto d = dict(); auto d = dict();
PyObject *result = PyRun_String(R"(\ PyObject *result = PyRun_String(R"(\
class pybind11_static_property(property): class pybind11_static_property(property):
def __get__(self, obj, cls): def __get__(self, obj, cls):
return property.__get__(self, cls, cls) return property.__get__(self, cls, cls)
def __set__(self, obj, value): def __set__(self, obj, value):
cls = obj if isinstance(obj, type) else type(obj) cls = obj if isinstance(obj, type) else type(obj)
property.__set__(self, cls, value) property.__set__(self, cls, value)
)", Py_file_input, d.ptr(), d.ptr() )",
); Py_file_input,
d.ptr(),
d.ptr());
if (result == nullptr) if (result == nullptr)
throw error_already_set(); throw error_already_set();
Py_DECREF(result); Py_DECREF(result);
@ -119,7 +133,7 @@ inline PyTypeObject *make_static_property_type() {
By default, Python replaces the `static_property` itself, but for wrapped C++ types By default, Python replaces the `static_property` itself, but for wrapped C++ types
we need to call `static_property.__set__()` in order to propagate the new value to we need to call `static_property.__set__()` in order to propagate the new value to
the underlying C++ data structure. */ the underlying C++ data structure. */
extern "C" inline int pybind11_meta_setattro(PyObject* obj, PyObject* name, PyObject* value) { extern "C" inline int pybind11_meta_setattro(PyObject *obj, PyObject *name, PyObject *value) {
// Use `_PyType_Lookup()` instead of `PyObject_GetAttr()` in order to get the raw // Use `_PyType_Lookup()` instead of `PyObject_GetAttr()` in order to get the raw
// descriptor (`property`) instead of calling `tp_descr_get` (`property.__get__()`). // descriptor (`property`) instead of calling `tp_descr_get` (`property.__get__()`).
PyObject *descr = _PyType_Lookup((PyTypeObject *) obj, name); PyObject *descr = _PyType_Lookup((PyTypeObject *) obj, name);
@ -128,9 +142,10 @@ extern "C" inline int pybind11_meta_setattro(PyObject* obj, PyObject* name, PyOb
// 1. `Type.static_prop = value` --> descr_set: `Type.static_prop.__set__(value)` // 1. `Type.static_prop = value` --> descr_set: `Type.static_prop.__set__(value)`
// 2. `Type.static_prop = other_static_prop` --> setattro: replace existing `static_prop` // 2. `Type.static_prop = other_static_prop` --> setattro: replace existing `static_prop`
// 3. `Type.regular_attribute = value` --> setattro: regular attribute assignment // 3. `Type.regular_attribute = value` --> setattro: regular attribute assignment
const auto static_prop = (PyObject *) get_internals().static_property_type; auto *const static_prop = (PyObject *) get_internals().static_property_type;
const auto call_descr_set = descr && value && PyObject_IsInstance(descr, static_prop) const auto call_descr_set = (descr != nullptr) && (value != nullptr)
&& !PyObject_IsInstance(value, static_prop); && (PyObject_IsInstance(descr, static_prop) != 0)
&& (PyObject_IsInstance(value, static_prop) == 0);
if (call_descr_set) { if (call_descr_set) {
// Call `static_property.__set__()` instead of replacing the `static_property`. // Call `static_property.__set__()` instead of replacing the `static_property`.
#if !defined(PYPY_VERSION) #if !defined(PYPY_VERSION)
@ -149,7 +164,6 @@ extern "C" inline int pybind11_meta_setattro(PyObject* obj, PyObject* name, PyOb
} }
} }
#if PY_MAJOR_VERSION >= 3
/** /**
* Python 3's PyInstanceMethod_Type hides itself via its tp_descr_get, which prevents aliasing * Python 3's PyInstanceMethod_Type hides itself via its tp_descr_get, which prevents aliasing
* methods via cls.attr("m2") = cls.attr("m1"): instead the tp_descr_get returns a plain function, * methods via cls.attr("m2") = cls.attr("m1"): instead the tp_descr_get returns a plain function,
@ -162,11 +176,8 @@ extern "C" inline PyObject *pybind11_meta_getattro(PyObject *obj, PyObject *name
Py_INCREF(descr); Py_INCREF(descr);
return descr; return descr;
} }
else { return PyType_Type.tp_getattro(obj, name);
return PyType_Type.tp_getattro(obj, name);
}
} }
#endif
/// metaclass `__call__` function that is used to create all pybind11 objects. /// metaclass `__call__` function that is used to create all pybind11 objects.
extern "C" inline PyObject *pybind11_meta_call(PyObject *type, PyObject *args, PyObject *kwargs) { extern "C" inline PyObject *pybind11_meta_call(PyObject *type, PyObject *args, PyObject *kwargs) {
@ -177,13 +188,12 @@ extern "C" inline PyObject *pybind11_meta_call(PyObject *type, PyObject *args, P
return nullptr; return nullptr;
} }
// This must be a pybind11 instance
auto instance = reinterpret_cast<detail::instance *>(self);
// Ensure that the base __init__ function(s) were called // Ensure that the base __init__ function(s) were called
for (const auto &vh : values_and_holders(instance)) { values_and_holders vhs(self);
if (!vh.holder_constructed()) { for (const auto &vh : vhs) {
PyErr_Format(PyExc_TypeError, "%.200s.__init__() must be called when overriding __init__", if (!vh.holder_constructed() && !vhs.is_redundant_value_and_holder(vh)) {
PyErr_Format(PyExc_TypeError,
"%.200s.__init__() must be called when overriding __init__",
get_fully_qualified_tp_name(vh.type->type).c_str()); get_fully_qualified_tp_name(vh.type->type).c_str());
Py_DECREF(self); Py_DECREF(self);
return nullptr; return nullptr;
@ -202,27 +212,28 @@ extern "C" inline void pybind11_meta_dealloc(PyObject *obj) {
// 1) be found in internals.registered_types_py // 1) be found in internals.registered_types_py
// 2) have exactly one associated `detail::type_info` // 2) have exactly one associated `detail::type_info`
auto found_type = internals.registered_types_py.find(type); auto found_type = internals.registered_types_py.find(type);
if (found_type != internals.registered_types_py.end() && if (found_type != internals.registered_types_py.end() && found_type->second.size() == 1
found_type->second.size() == 1 && && found_type->second[0]->type == type) {
found_type->second[0]->type == type) {
auto *tinfo = found_type->second[0]; auto *tinfo = found_type->second[0];
auto tindex = std::type_index(*tinfo->cpptype); auto tindex = std::type_index(*tinfo->cpptype);
internals.direct_conversions.erase(tindex); internals.direct_conversions.erase(tindex);
if (tinfo->module_local) if (tinfo->module_local) {
registered_local_types_cpp().erase(tindex); get_local_internals().registered_types_cpp.erase(tindex);
else } else {
internals.registered_types_cpp.erase(tindex); internals.registered_types_cpp.erase(tindex);
}
internals.registered_types_py.erase(tinfo->type); internals.registered_types_py.erase(tinfo->type);
// Actually just `std::erase_if`, but that's only available in C++20 // Actually just `std::erase_if`, but that's only available in C++20
auto &cache = internals.inactive_override_cache; auto &cache = internals.inactive_override_cache;
for (auto it = cache.begin(), last = cache.end(); it != last; ) { for (auto it = cache.begin(), last = cache.end(); it != last;) {
if (it->first == (PyObject *) tinfo->type) if (it->first == (PyObject *) tinfo->type) {
it = cache.erase(it); it = cache.erase(it);
else } else {
++it; ++it;
}
} }
delete tinfo; delete tinfo;
@ -234,7 +245,7 @@ extern "C" inline void pybind11_meta_dealloc(PyObject *obj) {
/** This metaclass is assigned by default to all pybind11 types and is required in order /** This metaclass is assigned by default to all pybind11 types and is required in order
for static properties to function correctly. Users may override this using `py::metaclass`. for static properties to function correctly. Users may override this using `py::metaclass`.
Return value: New reference. */ Return value: New reference. */
inline PyTypeObject* make_default_metaclass() { inline PyTypeObject *make_default_metaclass() {
constexpr auto *name = "pybind11_type"; constexpr auto *name = "pybind11_type";
auto name_obj = reinterpret_steal<object>(PYBIND11_FROM_STRING(name)); auto name_obj = reinterpret_steal<object>(PYBIND11_FROM_STRING(name));
@ -242,16 +253,17 @@ inline PyTypeObject* make_default_metaclass() {
issue no Python C API calls which could potentially invoke the issue no Python C API calls which could potentially invoke the
garbage collector (the GC will call type_traverse(), which will in garbage collector (the GC will call type_traverse(), which will in
turn find the newly constructed type in an invalid state) */ turn find the newly constructed type in an invalid state) */
auto heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0); auto *heap_type = (PyHeapTypeObject *) PyType_Type.tp_alloc(&PyType_Type, 0);
if (!heap_type) if (!heap_type) {
pybind11_fail("make_default_metaclass(): error allocating metaclass!"); pybind11_fail("make_default_metaclass(): error allocating metaclass!");
}
heap_type->ht_name = name_obj.inc_ref().ptr(); heap_type->ht_name = name_obj.inc_ref().ptr();
#ifdef PYBIND11_BUILTIN_QUALNAME #ifdef PYBIND11_BUILTIN_QUALNAME
heap_type->ht_qualname = name_obj.inc_ref().ptr(); heap_type->ht_qualname = name_obj.inc_ref().ptr();
#endif #endif
auto type = &heap_type->ht_type; auto *type = &heap_type->ht_type;
type->tp_name = name; type->tp_name = name;
type->tp_base = type_incref(&PyType_Type); type->tp_base = type_incref(&PyType_Type);
type->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE; type->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE;
@ -259,14 +271,13 @@ inline PyTypeObject* make_default_metaclass() {
type->tp_call = pybind11_meta_call; type->tp_call = pybind11_meta_call;
type->tp_setattro = pybind11_meta_setattro; type->tp_setattro = pybind11_meta_setattro;
#if PY_MAJOR_VERSION >= 3
type->tp_getattro = pybind11_meta_getattro; type->tp_getattro = pybind11_meta_getattro;
#endif
type->tp_dealloc = pybind11_meta_dealloc; type->tp_dealloc = pybind11_meta_dealloc;
if (PyType_Ready(type) < 0) if (PyType_Ready(type) < 0) {
pybind11_fail("make_default_metaclass(): failure in PyType_Ready()!"); pybind11_fail("make_default_metaclass(): failure in PyType_Ready()!");
}
setattr((PyObject *) type, "__module__", str("pybind11_builtins")); setattr((PyObject *) type, "__module__", str("pybind11_builtins"));
PYBIND11_SET_OLDPY_QUALNAME(type, name_obj); PYBIND11_SET_OLDPY_QUALNAME(type, name_obj);
@ -276,16 +287,20 @@ inline PyTypeObject* make_default_metaclass() {
/// For multiple inheritance types we need to recursively register/deregister base pointers for any /// For multiple inheritance types we need to recursively register/deregister base pointers for any
/// base classes with pointers that are difference from the instance value pointer so that we can /// base classes with pointers that are difference from the instance value pointer so that we can
/// correctly recognize an offset base class pointer. This calls a function with any offset base ptrs. /// correctly recognize an offset base class pointer. This calls a function with any offset base
inline void traverse_offset_bases(void *valueptr, const detail::type_info *tinfo, instance *self, /// ptrs.
bool (*f)(void * /*parentptr*/, instance * /*self*/)) { inline void traverse_offset_bases(void *valueptr,
const detail::type_info *tinfo,
instance *self,
bool (*f)(void * /*parentptr*/, instance * /*self*/)) {
for (handle h : reinterpret_borrow<tuple>(tinfo->type->tp_bases)) { for (handle h : reinterpret_borrow<tuple>(tinfo->type->tp_bases)) {
if (auto parent_tinfo = get_type_info((PyTypeObject *) h.ptr())) { if (auto *parent_tinfo = get_type_info((PyTypeObject *) h.ptr())) {
for (auto &c : parent_tinfo->implicit_casts) { for (auto &c : parent_tinfo->implicit_casts) {
if (c.first == tinfo->cpptype) { if (c.first == tinfo->cpptype) {
auto *parentptr = c.second(valueptr); auto *parentptr = c.second(valueptr);
if (parentptr != valueptr) if (parentptr != valueptr) {
f(parentptr, self); f(parentptr, self);
}
traverse_offset_bases(parentptr, parent_tinfo, self, f); traverse_offset_bases(parentptr, parent_tinfo, self, f);
break; break;
} }
@ -312,36 +327,36 @@ inline bool deregister_instance_impl(void *ptr, instance *self) {
inline void register_instance(instance *self, void *valptr, const type_info *tinfo) { inline void register_instance(instance *self, void *valptr, const type_info *tinfo) {
register_instance_impl(valptr, self); register_instance_impl(valptr, self);
if (!tinfo->simple_ancestors) if (!tinfo->simple_ancestors) {
traverse_offset_bases(valptr, tinfo, self, register_instance_impl); traverse_offset_bases(valptr, tinfo, self, register_instance_impl);
}
} }
inline bool deregister_instance(instance *self, void *valptr, const type_info *tinfo) { inline bool deregister_instance(instance *self, void *valptr, const type_info *tinfo) {
bool ret = deregister_instance_impl(valptr, self); bool ret = deregister_instance_impl(valptr, self);
if (!tinfo->simple_ancestors) if (!tinfo->simple_ancestors) {
traverse_offset_bases(valptr, tinfo, self, deregister_instance_impl); traverse_offset_bases(valptr, tinfo, self, deregister_instance_impl);
}
return ret; return ret;
} }
/// Instance creation function for all pybind11 types. It allocates the internal instance layout for /// Instance creation function for all pybind11 types. It allocates the internal instance layout
/// holding C++ objects and holders. Allocation is done lazily (the first time the instance is cast /// for holding C++ objects and holders. Allocation is done lazily (the first time the instance is
/// to a reference or pointer), and initialization is done by an `__init__` function. /// cast to a reference or pointer), and initialization is done by an `__init__` function.
inline PyObject *make_new_instance(PyTypeObject *type) { inline PyObject *make_new_instance(PyTypeObject *type) {
#if defined(PYPY_VERSION) #if defined(PYPY_VERSION)
// PyPy gets tp_basicsize wrong (issue 2482) under multiple inheritance when the first inherited // PyPy gets tp_basicsize wrong (issue 2482) under multiple inheritance when the first
// object is a a plain Python type (i.e. not derived from an extension type). Fix it. // inherited object is a plain Python type (i.e. not derived from an extension type). Fix it.
ssize_t instance_size = static_cast<ssize_t>(sizeof(instance)); ssize_t instance_size = static_cast<ssize_t>(sizeof(instance));
if (type->tp_basicsize < instance_size) { if (type->tp_basicsize < instance_size) {
type->tp_basicsize = instance_size; type->tp_basicsize = instance_size;
} }
#endif #endif
PyObject *self = type->tp_alloc(type, 0); PyObject *self = type->tp_alloc(type, 0);
auto inst = reinterpret_cast<instance *>(self); auto *inst = reinterpret_cast<instance *>(self);
// Allocate the value/holder internals: // Allocate the value/holder internals:
inst->allocate_layout(); inst->allocate_layout();
inst->owned = true;
return self; return self;
} }
@ -357,20 +372,20 @@ extern "C" inline PyObject *pybind11_object_new(PyTypeObject *type, PyObject *,
extern "C" inline int pybind11_object_init(PyObject *self, PyObject *, PyObject *) { extern "C" inline int pybind11_object_init(PyObject *self, PyObject *, PyObject *) {
PyTypeObject *type = Py_TYPE(self); PyTypeObject *type = Py_TYPE(self);
std::string msg = get_fully_qualified_tp_name(type) + ": No constructor defined!"; std::string msg = get_fully_qualified_tp_name(type) + ": No constructor defined!";
PyErr_SetString(PyExc_TypeError, msg.c_str()); set_error(PyExc_TypeError, msg.c_str());
return -1; return -1;
} }
inline void add_patient(PyObject *nurse, PyObject *patient) { inline void add_patient(PyObject *nurse, PyObject *patient) {
auto &internals = get_internals(); auto &internals = get_internals();
auto instance = reinterpret_cast<detail::instance *>(nurse); auto *instance = reinterpret_cast<detail::instance *>(nurse);
instance->has_patients = true; instance->has_patients = true;
Py_INCREF(patient); Py_INCREF(patient);
internals.patients[nurse].push_back(patient); internals.patients[nurse].push_back(patient);
} }
inline void clear_patients(PyObject *self) { inline void clear_patients(PyObject *self) {
auto instance = reinterpret_cast<detail::instance *>(self); auto *instance = reinterpret_cast<detail::instance *>(self);
auto &internals = get_internals(); auto &internals = get_internals();
auto pos = internals.patients.find(self); auto pos = internals.patients.find(self);
assert(pos != internals.patients.end()); assert(pos != internals.patients.end());
@ -380,14 +395,15 @@ inline void clear_patients(PyObject *self) {
auto patients = std::move(pos->second); auto patients = std::move(pos->second);
internals.patients.erase(pos); internals.patients.erase(pos);
instance->has_patients = false; instance->has_patients = false;
for (PyObject *&patient : patients) for (PyObject *&patient : patients) {
Py_CLEAR(patient); Py_CLEAR(patient);
}
} }
/// Clears all internal data from the instance and removes it from registered instances in /// Clears all internal data from the instance and removes it from registered instances in
/// preparation for deallocation. /// preparation for deallocation.
inline void clear_instance(PyObject *self) { inline void clear_instance(PyObject *self) {
auto instance = reinterpret_cast<detail::instance *>(self); auto *instance = reinterpret_cast<detail::instance *>(self);
// Deallocate any values/holders, if present: // Deallocate any values/holders, if present:
for (auto &v_h : values_and_holders(instance)) { for (auto &v_h : values_and_holders(instance)) {
@ -395,33 +411,48 @@ inline void clear_instance(PyObject *self) {
// We have to deregister before we call dealloc because, for virtual MI types, we still // We have to deregister before we call dealloc because, for virtual MI types, we still
// need to be able to get the parent pointers. // need to be able to get the parent pointers.
if (v_h.instance_registered() && !deregister_instance(instance, v_h.value_ptr(), v_h.type)) if (v_h.instance_registered()
pybind11_fail("pybind11_object_dealloc(): Tried to deallocate unregistered instance!"); && !deregister_instance(instance, v_h.value_ptr(), v_h.type)) {
pybind11_fail(
"pybind11_object_dealloc(): Tried to deallocate unregistered instance!");
}
if (instance->owned || v_h.holder_constructed()) if (instance->owned || v_h.holder_constructed()) {
v_h.type->dealloc(v_h); v_h.type->dealloc(v_h);
}
} }
} }
// Deallocate the value/holder layout internals: // Deallocate the value/holder layout internals:
instance->deallocate_layout(); instance->deallocate_layout();
if (instance->weakrefs) if (instance->weakrefs) {
PyObject_ClearWeakRefs(self); PyObject_ClearWeakRefs(self);
}
PyObject **dict_ptr = _PyObject_GetDictPtr(self); PyObject **dict_ptr = _PyObject_GetDictPtr(self);
if (dict_ptr) if (dict_ptr) {
Py_CLEAR(*dict_ptr); Py_CLEAR(*dict_ptr);
}
if (instance->has_patients) if (instance->has_patients) {
clear_patients(self); clear_patients(self);
}
} }
/// Instance destructor function for all pybind11 types. It calls `type_info.dealloc` /// Instance destructor function for all pybind11 types. It calls `type_info.dealloc`
/// to destroy the C++ object itself, while the rest is Python bookkeeping. /// to destroy the C++ object itself, while the rest is Python bookkeeping.
extern "C" inline void pybind11_object_dealloc(PyObject *self) { extern "C" inline void pybind11_object_dealloc(PyObject *self) {
auto *type = Py_TYPE(self);
// If this is a GC tracked object, untrack it first
// Note that the track call is implicitly done by the
// default tp_alloc, which we never override.
if (PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC) != 0) {
PyObject_GC_UnTrack(self);
}
clear_instance(self); clear_instance(self);
auto type = Py_TYPE(self);
type->tp_free(self); type->tp_free(self);
#if PY_VERSION_HEX < 0x03080000 #if PY_VERSION_HEX < 0x03080000
@ -439,6 +470,8 @@ extern "C" inline void pybind11_object_dealloc(PyObject *self) {
#endif #endif
} }
std::string error_string();
/** Create the type which can be used as a common base for all classes. This is /** Create the type which can be used as a common base for all classes. This is
needed in order to satisfy Python's requirements for multiple inheritance. needed in order to satisfy Python's requirements for multiple inheritance.
Return value: New reference. */ Return value: New reference. */
@ -450,16 +483,17 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass) {
issue no Python C API calls which could potentially invoke the issue no Python C API calls which could potentially invoke the
garbage collector (the GC will call type_traverse(), which will in garbage collector (the GC will call type_traverse(), which will in
turn find the newly constructed type in an invalid state) */ turn find the newly constructed type in an invalid state) */
auto heap_type = (PyHeapTypeObject *) metaclass->tp_alloc(metaclass, 0); auto *heap_type = (PyHeapTypeObject *) metaclass->tp_alloc(metaclass, 0);
if (!heap_type) if (!heap_type) {
pybind11_fail("make_object_base_type(): error allocating type!"); pybind11_fail("make_object_base_type(): error allocating type!");
}
heap_type->ht_name = name_obj.inc_ref().ptr(); heap_type->ht_name = name_obj.inc_ref().ptr();
#ifdef PYBIND11_BUILTIN_QUALNAME #ifdef PYBIND11_BUILTIN_QUALNAME
heap_type->ht_qualname = name_obj.inc_ref().ptr(); heap_type->ht_qualname = name_obj.inc_ref().ptr();
#endif #endif
auto type = &heap_type->ht_type; auto *type = &heap_type->ht_type;
type->tp_name = name; type->tp_name = name;
type->tp_base = type_incref(&PyBaseObject_Type); type->tp_base = type_incref(&PyBaseObject_Type);
type->tp_basicsize = static_cast<ssize_t>(sizeof(instance)); type->tp_basicsize = static_cast<ssize_t>(sizeof(instance));
@ -472,8 +506,9 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass) {
/* Support weak references (needed for the keep_alive feature) */ /* Support weak references (needed for the keep_alive feature) */
type->tp_weaklistoffset = offsetof(instance, weakrefs); type->tp_weaklistoffset = offsetof(instance, weakrefs);
if (PyType_Ready(type) < 0) if (PyType_Ready(type) < 0) {
pybind11_fail("PyType_Ready failed in make_object_base_type():" + error_string()); pybind11_fail("PyType_Ready failed in make_object_base_type(): " + error_string());
}
setattr((PyObject *) type, "__module__", str("pybind11_builtins")); setattr((PyObject *) type, "__module__", str("pybind11_builtins"));
PYBIND11_SET_OLDPY_QUALNAME(type, name_obj); PYBIND11_SET_OLDPY_QUALNAME(type, name_obj);
@ -482,56 +517,56 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass) {
return (PyObject *) heap_type; return (PyObject *) heap_type;
} }
/// dynamic_attr: Support for `d = instance.__dict__`.
extern "C" inline PyObject *pybind11_get_dict(PyObject *self, void *) {
PyObject *&dict = *_PyObject_GetDictPtr(self);
if (!dict)
dict = PyDict_New();
Py_XINCREF(dict);
return dict;
}
/// dynamic_attr: Support for `instance.__dict__ = dict()`.
extern "C" inline int pybind11_set_dict(PyObject *self, PyObject *new_dict, void *) {
if (!PyDict_Check(new_dict)) {
PyErr_Format(PyExc_TypeError, "__dict__ must be set to a dictionary, not a '%.200s'",
get_fully_qualified_tp_name(Py_TYPE(new_dict)).c_str());
return -1;
}
PyObject *&dict = *_PyObject_GetDictPtr(self);
Py_INCREF(new_dict);
Py_CLEAR(dict);
dict = new_dict;
return 0;
}
/// dynamic_attr: Allow the garbage collector to traverse the internal instance `__dict__`. /// dynamic_attr: Allow the garbage collector to traverse the internal instance `__dict__`.
extern "C" inline int pybind11_traverse(PyObject *self, visitproc visit, void *arg) { extern "C" inline int pybind11_traverse(PyObject *self, visitproc visit, void *arg) {
#if PY_VERSION_HEX >= 0x030D0000
PyObject_VisitManagedDict(self, visit, arg);
#else
PyObject *&dict = *_PyObject_GetDictPtr(self); PyObject *&dict = *_PyObject_GetDictPtr(self);
Py_VISIT(dict); Py_VISIT(dict);
#endif
// https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_traverse
#if PY_VERSION_HEX >= 0x03090000
Py_VISIT(Py_TYPE(self));
#endif
return 0; return 0;
} }
/// dynamic_attr: Allow the GC to clear the dictionary. /// dynamic_attr: Allow the GC to clear the dictionary.
extern "C" inline int pybind11_clear(PyObject *self) { extern "C" inline int pybind11_clear(PyObject *self) {
#if PY_VERSION_HEX >= 0x030D0000
PyObject_ClearManagedDict(self);
#else
PyObject *&dict = *_PyObject_GetDictPtr(self); PyObject *&dict = *_PyObject_GetDictPtr(self);
Py_CLEAR(dict); Py_CLEAR(dict);
#endif
return 0; return 0;
} }
/// Give instances of this type a `__dict__` and opt into garbage collection. /// Give instances of this type a `__dict__` and opt into garbage collection.
inline void enable_dynamic_attributes(PyHeapTypeObject *heap_type) { inline void enable_dynamic_attributes(PyHeapTypeObject *heap_type) {
auto type = &heap_type->ht_type; auto *type = &heap_type->ht_type;
type->tp_flags |= Py_TPFLAGS_HAVE_GC; type->tp_flags |= Py_TPFLAGS_HAVE_GC;
type->tp_dictoffset = type->tp_basicsize; // place dict at the end #if PY_VERSION_HEX < 0x030B0000
type->tp_basicsize += (ssize_t)sizeof(PyObject *); // and allocate enough space for it type->tp_dictoffset = type->tp_basicsize; // place dict at the end
type->tp_basicsize += (ssize_t) sizeof(PyObject *); // and allocate enough space for it
#else
type->tp_flags |= Py_TPFLAGS_MANAGED_DICT;
#endif
type->tp_traverse = pybind11_traverse; type->tp_traverse = pybind11_traverse;
type->tp_clear = pybind11_clear; type->tp_clear = pybind11_clear;
static PyGetSetDef getset[] = { static PyGetSetDef getset[] = {{
{const_cast<char*>("__dict__"), pybind11_get_dict, pybind11_set_dict, nullptr, nullptr}, #if PY_VERSION_HEX < 0x03070000
{nullptr, nullptr, nullptr, nullptr, nullptr} const_cast<char *>("__dict__"),
}; #else
"__dict__",
#endif
PyObject_GenericGetDict,
PyObject_GenericSetDict,
nullptr,
nullptr},
{nullptr, nullptr, nullptr, nullptr, nullptr}};
type->tp_getset = getset; type->tp_getset = getset;
} }
@ -541,38 +576,42 @@ extern "C" inline int pybind11_getbuffer(PyObject *obj, Py_buffer *view, int fla
type_info *tinfo = nullptr; type_info *tinfo = nullptr;
for (auto type : reinterpret_borrow<tuple>(Py_TYPE(obj)->tp_mro)) { for (auto type : reinterpret_borrow<tuple>(Py_TYPE(obj)->tp_mro)) {
tinfo = get_type_info((PyTypeObject *) type.ptr()); tinfo = get_type_info((PyTypeObject *) type.ptr());
if (tinfo && tinfo->get_buffer) if (tinfo && tinfo->get_buffer) {
break; break;
}
} }
if (view == nullptr || !tinfo || !tinfo->get_buffer) { if (view == nullptr || !tinfo || !tinfo->get_buffer) {
if (view) if (view) {
view->obj = nullptr; view->obj = nullptr;
PyErr_SetString(PyExc_BufferError, "pybind11_getbuffer(): Internal error"); }
set_error(PyExc_BufferError, "pybind11_getbuffer(): Internal error");
return -1; return -1;
} }
std::memset(view, 0, sizeof(Py_buffer)); std::memset(view, 0, sizeof(Py_buffer));
buffer_info *info = tinfo->get_buffer(obj, tinfo->get_buffer_data); buffer_info *info = tinfo->get_buffer(obj, tinfo->get_buffer_data);
if ((flags & PyBUF_WRITABLE) == PyBUF_WRITABLE && info->readonly) {
delete info;
// view->obj = nullptr; // Was just memset to 0, so not necessary
set_error(PyExc_BufferError, "Writable buffer requested for readonly storage");
return -1;
}
view->obj = obj; view->obj = obj;
view->ndim = 1; view->ndim = 1;
view->internal = info; view->internal = info;
view->buf = info->ptr; view->buf = info->ptr;
view->itemsize = info->itemsize; view->itemsize = info->itemsize;
view->len = view->itemsize; view->len = view->itemsize;
for (auto s : info->shape) for (auto s : info->shape) {
view->len *= s; view->len *= s;
view->readonly = info->readonly;
if ((flags & PyBUF_WRITABLE) == PyBUF_WRITABLE && info->readonly) {
if (view)
view->obj = nullptr;
PyErr_SetString(PyExc_BufferError, "Writable buffer requested for readonly storage");
return -1;
} }
if ((flags & PyBUF_FORMAT) == PyBUF_FORMAT) view->readonly = static_cast<int>(info->readonly);
if ((flags & PyBUF_FORMAT) == PyBUF_FORMAT) {
view->format = const_cast<char *>(info->format.c_str()); view->format = const_cast<char *>(info->format.c_str());
}
if ((flags & PyBUF_STRIDES) == PyBUF_STRIDES) { if ((flags & PyBUF_STRIDES) == PyBUF_STRIDES) {
view->ndim = (int) info->ndim; view->ndim = (int) info->ndim;
view->strides = &info->strides[0]; view->strides = info->strides.data();
view->shape = &info->shape[0]; view->shape = info->shape.data();
} }
Py_INCREF(view->obj); Py_INCREF(view->obj);
return 0; return 0;
@ -586,9 +625,6 @@ extern "C" inline void pybind11_releasebuffer(PyObject *, Py_buffer *view) {
/// Give this type a buffer interface. /// Give this type a buffer interface.
inline void enable_buffer_protocol(PyHeapTypeObject *heap_type) { inline void enable_buffer_protocol(PyHeapTypeObject *heap_type) {
heap_type->ht_type.tp_as_buffer = &heap_type->as_buffer; heap_type->ht_type.tp_as_buffer = &heap_type->as_buffer;
#if PY_MAJOR_VERSION < 3
heap_type->ht_type.tp_flags |= Py_TPFLAGS_HAVE_NEWBUFFER;
#endif
heap_type->as_buffer.bf_getbuffer = pybind11_getbuffer; heap_type->as_buffer.bf_getbuffer = pybind11_getbuffer;
heap_type->as_buffer.bf_releasebuffer = pybind11_releasebuffer; heap_type->as_buffer.bf_releasebuffer = pybind11_releasebuffer;
@ -596,70 +632,68 @@ inline void enable_buffer_protocol(PyHeapTypeObject *heap_type) {
/** Create a brand new Python type according to the `type_record` specification. /** Create a brand new Python type according to the `type_record` specification.
Return value: New reference. */ Return value: New reference. */
inline PyObject* make_new_python_type(const type_record &rec) { inline PyObject *make_new_python_type(const type_record &rec) {
auto name = reinterpret_steal<object>(PYBIND11_FROM_STRING(rec.name)); auto name = reinterpret_steal<object>(PYBIND11_FROM_STRING(rec.name));
auto qualname = name; auto qualname = name;
if (rec.scope && !PyModule_Check(rec.scope.ptr()) && hasattr(rec.scope, "__qualname__")) { if (rec.scope && !PyModule_Check(rec.scope.ptr()) && hasattr(rec.scope, "__qualname__")) {
#if PY_MAJOR_VERSION >= 3
qualname = reinterpret_steal<object>( qualname = reinterpret_steal<object>(
PyUnicode_FromFormat("%U.%U", rec.scope.attr("__qualname__").ptr(), name.ptr())); PyUnicode_FromFormat("%U.%U", rec.scope.attr("__qualname__").ptr(), name.ptr()));
#else
qualname = str(rec.scope.attr("__qualname__").cast<std::string>() + "." + rec.name);
#endif
} }
object module_; object module_;
if (rec.scope) { if (rec.scope) {
if (hasattr(rec.scope, "__module__")) if (hasattr(rec.scope, "__module__")) {
module_ = rec.scope.attr("__module__"); module_ = rec.scope.attr("__module__");
else if (hasattr(rec.scope, "__name__")) } else if (hasattr(rec.scope, "__name__")) {
module_ = rec.scope.attr("__name__"); module_ = rec.scope.attr("__name__");
}
} }
auto full_name = c_str( const auto *full_name = c_str(
#if !defined(PYPY_VERSION) #if !defined(PYPY_VERSION)
module_ ? str(module_).cast<std::string>() + "." + rec.name : module_ ? str(module_).cast<std::string>() + "." + rec.name :
#endif #endif
rec.name); rec.name);
char *tp_doc = nullptr; char *tp_doc = nullptr;
if (rec.doc && options::show_user_defined_docstrings()) { if (rec.doc && options::show_user_defined_docstrings()) {
/* Allocate memory for docstring (using PyObject_MALLOC, since /* Allocate memory for docstring (using PyObject_MALLOC, since
Python will free this later on) */ Python will free this later on) */
size_t size = strlen(rec.doc) + 1; size_t size = std::strlen(rec.doc) + 1;
tp_doc = (char *) PyObject_MALLOC(size); tp_doc = (char *) PyObject_MALLOC(size);
memcpy((void *) tp_doc, rec.doc, size); std::memcpy((void *) tp_doc, rec.doc, size);
} }
auto &internals = get_internals(); auto &internals = get_internals();
auto bases = tuple(rec.bases); auto bases = tuple(rec.bases);
auto base = (bases.empty()) ? internals.instance_base auto *base = (bases.empty()) ? internals.instance_base : bases[0].ptr();
: bases[0].ptr();
/* Danger zone: from now (and until PyType_Ready), make sure to /* Danger zone: from now (and until PyType_Ready), make sure to
issue no Python C API calls which could potentially invoke the issue no Python C API calls which could potentially invoke the
garbage collector (the GC will call type_traverse(), which will in garbage collector (the GC will call type_traverse(), which will in
turn find the newly constructed type in an invalid state) */ turn find the newly constructed type in an invalid state) */
auto metaclass = rec.metaclass.ptr() ? (PyTypeObject *) rec.metaclass.ptr() auto *metaclass
: internals.default_metaclass; = rec.metaclass.ptr() ? (PyTypeObject *) rec.metaclass.ptr() : internals.default_metaclass;
auto heap_type = (PyHeapTypeObject *) metaclass->tp_alloc(metaclass, 0); auto *heap_type = (PyHeapTypeObject *) metaclass->tp_alloc(metaclass, 0);
if (!heap_type) if (!heap_type) {
pybind11_fail(std::string(rec.name) + ": Unable to create type object!"); pybind11_fail(std::string(rec.name) + ": Unable to create type object!");
}
heap_type->ht_name = name.release().ptr(); heap_type->ht_name = name.release().ptr();
#ifdef PYBIND11_BUILTIN_QUALNAME #ifdef PYBIND11_BUILTIN_QUALNAME
heap_type->ht_qualname = qualname.inc_ref().ptr(); heap_type->ht_qualname = qualname.inc_ref().ptr();
#endif #endif
auto type = &heap_type->ht_type; auto *type = &heap_type->ht_type;
type->tp_name = full_name; type->tp_name = full_name;
type->tp_doc = tp_doc; type->tp_doc = tp_doc;
type->tp_base = type_incref((PyTypeObject *)base); type->tp_base = type_incref((PyTypeObject *) base);
type->tp_basicsize = static_cast<ssize_t>(sizeof(instance)); type->tp_basicsize = static_cast<ssize_t>(sizeof(instance));
if (!bases.empty()) if (!bases.empty()) {
type->tp_bases = bases.release().ptr(); type->tp_bases = bases.release().ptr();
}
/* Don't inherit base __init__ */ /* Don't inherit base __init__ */
type->tp_init = pybind11_object_init; type->tp_init = pybind11_object_init;
@ -668,38 +702,42 @@ inline PyObject* make_new_python_type(const type_record &rec) {
type->tp_as_number = &heap_type->as_number; type->tp_as_number = &heap_type->as_number;
type->tp_as_sequence = &heap_type->as_sequence; type->tp_as_sequence = &heap_type->as_sequence;
type->tp_as_mapping = &heap_type->as_mapping; type->tp_as_mapping = &heap_type->as_mapping;
#if PY_VERSION_HEX >= 0x03050000
type->tp_as_async = &heap_type->as_async; type->tp_as_async = &heap_type->as_async;
#endif
/* Flags */ /* Flags */
type->tp_flags |= Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HEAPTYPE; type->tp_flags |= Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HEAPTYPE;
#if PY_MAJOR_VERSION < 3 if (!rec.is_final) {
type->tp_flags |= Py_TPFLAGS_CHECKTYPES;
#endif
if (!rec.is_final)
type->tp_flags |= Py_TPFLAGS_BASETYPE; type->tp_flags |= Py_TPFLAGS_BASETYPE;
}
if (rec.dynamic_attr) if (rec.dynamic_attr) {
enable_dynamic_attributes(heap_type); enable_dynamic_attributes(heap_type);
}
if (rec.buffer_protocol) if (rec.buffer_protocol) {
enable_buffer_protocol(heap_type); enable_buffer_protocol(heap_type);
}
if (PyType_Ready(type) < 0) if (rec.custom_type_setup_callback) {
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed (" + error_string() + ")!"); rec.custom_type_setup_callback(heap_type);
}
assert(rec.dynamic_attr ? PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC) if (PyType_Ready(type) < 0) {
: !PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC)); pybind11_fail(std::string(rec.name) + ": PyType_Ready failed: " + error_string());
}
assert(!rec.dynamic_attr || PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));
/* Register type with the parent scope */ /* Register type with the parent scope */
if (rec.scope) if (rec.scope) {
setattr(rec.scope, rec.name, (PyObject *) type); setattr(rec.scope, rec.name, (PyObject *) type);
else } else {
Py_INCREF(type); // Keep it alive forever (reference leak) Py_INCREF(type); // Keep it alive forever (reference leak)
}
if (module_) // Needed by pydoc if (module_) { // Needed by pydoc
setattr((PyObject *) type, "__module__", module_); setattr((PyObject *) type, "__module__", module_);
}
PYBIND11_SET_OLDPY_QUALNAME(type, qualname); PYBIND11_SET_OLDPY_QUALNAME(type, qualname);

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,77 @@
// Copyright (c) 2024 The pybind Community.
#pragma once
#include <pybind11/pytypes.h>
#include "common.h"
#include "internals.h"
#include <typeinfo>
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail)
// Forward declaration needed here: Refactoring opportunity.
extern "C" inline PyObject *pybind11_object_new(PyTypeObject *type, PyObject *, PyObject *);
inline bool type_is_managed_by_our_internals(PyTypeObject *type_obj) {
#if defined(PYPY_VERSION)
auto &internals = get_internals();
return bool(internals.registered_types_py.find(type_obj)
!= internals.registered_types_py.end());
#else
return bool(type_obj->tp_new == pybind11_object_new);
#endif
}
inline bool is_instance_method_of_type(PyTypeObject *type_obj, PyObject *attr_name) {
PyObject *descr = _PyType_Lookup(type_obj, attr_name);
return bool((descr != nullptr) && PyInstanceMethod_Check(descr));
}
inline object try_get_cpp_conduit_method(PyObject *obj) {
if (PyType_Check(obj)) {
return object();
}
PyTypeObject *type_obj = Py_TYPE(obj);
str attr_name("_pybind11_conduit_v1_");
bool assumed_to_be_callable = false;
if (type_is_managed_by_our_internals(type_obj)) {
if (!is_instance_method_of_type(type_obj, attr_name.ptr())) {
return object();
}
assumed_to_be_callable = true;
}
PyObject *method = PyObject_GetAttr(obj, attr_name.ptr());
if (method == nullptr) {
PyErr_Clear();
return object();
}
if (!assumed_to_be_callable && PyCallable_Check(method) == 0) {
Py_DECREF(method);
return object();
}
return reinterpret_steal<object>(method);
}
inline void *try_raw_pointer_ephemeral_from_cpp_conduit(handle src,
const std::type_info *cpp_type_info) {
object method = try_get_cpp_conduit_method(src.ptr());
if (method) {
capsule cpp_type_info_capsule(const_cast<void *>(static_cast<const void *>(cpp_type_info)),
typeid(std::type_info).name());
object cpp_conduit = method(bytes(PYBIND11_PLATFORM_ABI_ID),
cpp_type_info_capsule,
bytes("raw_pointer_ephemeral"));
if (isinstance<capsule>(cpp_conduit)) {
return reinterpret_borrow<capsule>(cpp_conduit).get_pointer();
}
}
return nullptr;
}
#define PYBIND11_HAS_CPP_CONDUIT 1
PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -15,24 +15,26 @@ PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
#if !defined(_MSC_VER) #if !defined(_MSC_VER)
# define PYBIND11_DESCR_CONSTEXPR static constexpr # define PYBIND11_DESCR_CONSTEXPR static constexpr
#else #else
# define PYBIND11_DESCR_CONSTEXPR const # define PYBIND11_DESCR_CONSTEXPR const
#endif #endif
/* Concatenate type signatures at compile time */ /* Concatenate type signatures at compile time */
template <size_t N, typename... Ts> template <size_t N, typename... Ts>
struct descr { struct descr {
char text[N + 1]; char text[N + 1]{'\0'};
constexpr descr() : text{'\0'} { } constexpr descr() = default;
constexpr descr(char const (&s)[N+1]) : descr(s, make_index_sequence<N>()) { } // NOLINTNEXTLINE(google-explicit-constructor)
constexpr descr(char const (&s)[N + 1]) : descr(s, make_index_sequence<N>()) {}
template <size_t... Is> template <size_t... Is>
constexpr descr(char const (&s)[N+1], index_sequence<Is...>) : text{s[Is]..., '\0'} { } constexpr descr(char const (&s)[N + 1], index_sequence<Is...>) : text{s[Is]..., '\0'} {}
template <typename... Chars> template <typename... Chars>
constexpr descr(char c, Chars... cs) : text{c, static_cast<char>(cs)..., '\0'} { } // NOLINTNEXTLINE(google-explicit-constructor)
constexpr descr(char c, Chars... cs) : text{c, static_cast<char>(cs)..., '\0'} {}
static constexpr std::array<const std::type_info *, sizeof...(Ts) + 1> types() { static constexpr std::array<const std::type_info *, sizeof...(Ts) + 1> types() {
return {{&typeid(Ts)..., nullptr}}; return {{&typeid(Ts)..., nullptr}};
@ -40,60 +42,129 @@ struct descr {
}; };
template <size_t N1, size_t N2, typename... Ts1, typename... Ts2, size_t... Is1, size_t... Is2> template <size_t N1, size_t N2, typename... Ts1, typename... Ts2, size_t... Is1, size_t... Is2>
constexpr descr<N1 + N2, Ts1..., Ts2...> plus_impl(const descr<N1, Ts1...> &a, const descr<N2, Ts2...> &b, constexpr descr<N1 + N2, Ts1..., Ts2...> plus_impl(const descr<N1, Ts1...> &a,
index_sequence<Is1...>, index_sequence<Is2...>) { const descr<N2, Ts2...> &b,
index_sequence<Is1...>,
index_sequence<Is2...>) {
PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(b);
return {a.text[Is1]..., b.text[Is2]...}; return {a.text[Is1]..., b.text[Is2]...};
} }
template <size_t N1, size_t N2, typename... Ts1, typename... Ts2> template <size_t N1, size_t N2, typename... Ts1, typename... Ts2>
constexpr descr<N1 + N2, Ts1..., Ts2...> operator+(const descr<N1, Ts1...> &a, const descr<N2, Ts2...> &b) { constexpr descr<N1 + N2, Ts1..., Ts2...> operator+(const descr<N1, Ts1...> &a,
const descr<N2, Ts2...> &b) {
return plus_impl(a, b, make_index_sequence<N1>(), make_index_sequence<N2>()); return plus_impl(a, b, make_index_sequence<N1>(), make_index_sequence<N2>());
} }
template <size_t N> template <size_t N>
constexpr descr<N - 1> _(char const(&text)[N]) { return descr<N - 1>(text); } constexpr descr<N - 1> const_name(char const (&text)[N]) {
constexpr descr<0> _(char const(&)[1]) { return {}; } return descr<N - 1>(text);
}
constexpr descr<0> const_name(char const (&)[1]) { return {}; }
template <size_t Rem, size_t... Digits> struct int_to_str : int_to_str<Rem/10, Rem%10, Digits...> { }; template <size_t Rem, size_t... Digits>
template <size_t...Digits> struct int_to_str<0, Digits...> { struct int_to_str : int_to_str<Rem / 10, Rem % 10, Digits...> {};
template <size_t... Digits>
struct int_to_str<0, Digits...> {
// WARNING: This only works with C++17 or higher.
static constexpr auto digits = descr<sizeof...(Digits)>(('0' + Digits)...); static constexpr auto digits = descr<sizeof...(Digits)>(('0' + Digits)...);
}; };
// Ternary description (like std::conditional) // Ternary description (like std::conditional)
template <bool B, size_t N1, size_t N2> template <bool B, size_t N1, size_t N2>
constexpr enable_if_t<B, descr<N1 - 1>> _(char const(&text1)[N1], char const(&)[N2]) { constexpr enable_if_t<B, descr<N1 - 1>> const_name(char const (&text1)[N1], char const (&)[N2]) {
return _(text1); return const_name(text1);
} }
template <bool B, size_t N1, size_t N2> template <bool B, size_t N1, size_t N2>
constexpr enable_if_t<!B, descr<N2 - 1>> _(char const(&)[N1], char const(&text2)[N2]) { constexpr enable_if_t<!B, descr<N2 - 1>> const_name(char const (&)[N1], char const (&text2)[N2]) {
return _(text2); return const_name(text2);
} }
template <bool B, typename T1, typename T2> template <bool B, typename T1, typename T2>
constexpr enable_if_t<B, T1> _(const T1 &d, const T2 &) { return d; } constexpr enable_if_t<B, T1> const_name(const T1 &d, const T2 &) {
return d;
}
template <bool B, typename T1, typename T2> template <bool B, typename T1, typename T2>
constexpr enable_if_t<!B, T2> _(const T1 &, const T2 &d) { return d; } constexpr enable_if_t<!B, T2> const_name(const T1 &, const T2 &d) {
return d;
}
template <size_t Size> auto constexpr _() -> decltype(int_to_str<Size / 10, Size % 10>::digits) { template <size_t Size>
auto constexpr const_name() -> remove_cv_t<decltype(int_to_str<Size / 10, Size % 10>::digits)> {
return int_to_str<Size / 10, Size % 10>::digits; return int_to_str<Size / 10, Size % 10>::digits;
} }
template <typename Type> constexpr descr<1, Type> _() { return {'%'}; } template <typename Type>
constexpr descr<1, Type> const_name() {
return {'%'};
}
// If "_" is defined as a macro, py::detail::_ cannot be provided.
// It is therefore best to use py::detail::const_name universally.
// This block is for backward compatibility only.
// (The const_name code is repeated to avoid introducing a "_" #define ourselves.)
#ifndef _
# define PYBIND11_DETAIL_UNDERSCORE_BACKWARD_COMPATIBILITY
template <size_t N>
constexpr descr<N - 1> _(char const (&text)[N]) {
return const_name<N>(text);
}
template <bool B, size_t N1, size_t N2>
constexpr enable_if_t<B, descr<N1 - 1>> _(char const (&text1)[N1], char const (&text2)[N2]) {
return const_name<B, N1, N2>(text1, text2);
}
template <bool B, size_t N1, size_t N2>
constexpr enable_if_t<!B, descr<N2 - 1>> _(char const (&text1)[N1], char const (&text2)[N2]) {
return const_name<B, N1, N2>(text1, text2);
}
template <bool B, typename T1, typename T2>
constexpr enable_if_t<B, T1> _(const T1 &d1, const T2 &d2) {
return const_name<B, T1, T2>(d1, d2);
}
template <bool B, typename T1, typename T2>
constexpr enable_if_t<!B, T2> _(const T1 &d1, const T2 &d2) {
return const_name<B, T1, T2>(d1, d2);
}
template <size_t Size>
auto constexpr _() -> remove_cv_t<decltype(int_to_str<Size / 10, Size % 10>::digits)> {
return const_name<Size>();
}
template <typename Type>
constexpr descr<1, Type> _() {
return const_name<Type>();
}
#endif // #ifndef _
constexpr descr<0> concat() { return {}; } constexpr descr<0> concat() { return {}; }
template <size_t N, typename... Ts> template <size_t N, typename... Ts>
constexpr descr<N, Ts...> concat(const descr<N, Ts...> &descr) { return descr; } constexpr descr<N, Ts...> concat(const descr<N, Ts...> &descr) {
return descr;
}
#ifdef __cpp_fold_expressions
template <size_t N1, size_t N2, typename... Ts1, typename... Ts2>
constexpr descr<N1 + N2 + 2, Ts1..., Ts2...> operator,(const descr<N1, Ts1...> &a,
const descr<N2, Ts2...> &b) {
return a + const_name(", ") + b;
}
template <size_t N, typename... Ts, typename... Args>
constexpr auto concat(const descr<N, Ts...> &d, const Args &...args) {
return (d, ..., args);
}
#else
template <size_t N, typename... Ts, typename... Args> template <size_t N, typename... Ts, typename... Args>
constexpr auto concat(const descr<N, Ts...> &d, const Args &...args) constexpr auto concat(const descr<N, Ts...> &d, const Args &...args)
-> decltype(std::declval<descr<N + 2, Ts...>>() + concat(args...)) { -> decltype(std::declval<descr<N + 2, Ts...>>() + concat(args...)) {
return d + _(", ") + concat(args...); return d + const_name(", ") + concat(args...);
} }
#endif
template <size_t N, typename... Ts> template <size_t N, typename... Ts>
constexpr descr<N + 2, Ts...> type_descr(const descr<N, Ts...> &descr) { constexpr descr<N + 2, Ts...> type_descr(const descr<N, Ts...> &descr) {
return _("{") + descr + _("}"); return const_name("{") + descr + const_name("}");
} }
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)

View File

@ -12,6 +12,9 @@
#include "class.h" #include "class.h"
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_WARNING_DISABLE_MSVC(4127)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
template <> template <>
@ -22,9 +25,10 @@ public:
return true; return true;
} }
template <typename> using cast_op_type = value_and_holder &; template <typename>
operator value_and_holder &() { return *value; } using cast_op_type = value_and_holder &;
static constexpr auto name = _<value_and_holder>(); explicit operator value_and_holder &() { return *value; }
static constexpr auto name = const_name<value_and_holder>();
private: private:
value_and_holder *value = nullptr; value_and_holder *value = nullptr;
@ -33,15 +37,21 @@ private:
PYBIND11_NAMESPACE_BEGIN(initimpl) PYBIND11_NAMESPACE_BEGIN(initimpl)
inline void no_nullptr(void *ptr) { inline void no_nullptr(void *ptr) {
if (!ptr) throw type_error("pybind11::init(): factory function returned nullptr"); if (!ptr) {
throw type_error("pybind11::init(): factory function returned nullptr");
}
} }
// Implementing functions for all forms of py::init<...> and py::init(...) // Implementing functions for all forms of py::init<...> and py::init(...)
template <typename Class> using Cpp = typename Class::type; template <typename Class>
template <typename Class> using Alias = typename Class::type_alias; using Cpp = typename Class::type;
template <typename Class> using Holder = typename Class::holder_type; template <typename Class>
using Alias = typename Class::type_alias;
template <typename Class>
using Holder = typename Class::holder_type;
template <typename Class> using is_alias_constructible = std::is_constructible<Alias<Class>, Cpp<Class> &&>; template <typename Class>
using is_alias_constructible = std::is_constructible<Alias<Class>, Cpp<Class> &&>;
// Takes a Cpp pointer and returns true if it actually is a polymorphic Alias instance. // Takes a Cpp pointer and returns true if it actually is a polymorphic Alias instance.
template <typename Class, enable_if_t<Class::has_alias, int> = 0> template <typename Class, enable_if_t<Class::has_alias, int> = 0>
@ -50,17 +60,27 @@ bool is_alias(Cpp<Class> *ptr) {
} }
// Failing fallback version of the above for a no-alias class (always returns false) // Failing fallback version of the above for a no-alias class (always returns false)
template <typename /*Class*/> template <typename /*Class*/>
constexpr bool is_alias(void *) { return false; } constexpr bool is_alias(void *) {
return false;
}
// Constructs and returns a new object; if the given arguments don't map to a constructor, we fall // Constructs and returns a new object; if the given arguments don't map to a constructor, we fall
// back to brace aggregate initiailization so that for aggregate initialization can be used with // back to brace aggregate initialization so that for aggregate initialization can be used with
// py::init, e.g. `py::init<int, int>` to initialize a `struct T { int a; int b; }`. For // py::init, e.g. `py::init<int, int>` to initialize a `struct T { int a; int b; }`. For
// non-aggregate types, we need to use an ordinary T(...) constructor (invoking as `T{...}` usually // non-aggregate types, we need to use an ordinary T(...) constructor (invoking as `T{...}` usually
// works, but will not do the expected thing when `T` has an `initializer_list<T>` constructor). // works, but will not do the expected thing when `T` has an `initializer_list<T>` constructor).
template <typename Class, typename... Args, detail::enable_if_t<std::is_constructible<Class, Args...>::value, int> = 0> template <typename Class,
inline Class *construct_or_initialize(Args &&...args) { return new Class(std::forward<Args>(args)...); } typename... Args,
template <typename Class, typename... Args, detail::enable_if_t<!std::is_constructible<Class, Args...>::value, int> = 0> detail::enable_if_t<std::is_constructible<Class, Args...>::value, int> = 0>
inline Class *construct_or_initialize(Args &&...args) { return new Class{std::forward<Args>(args)...}; } inline Class *construct_or_initialize(Args &&...args) {
return new Class(std::forward<Args>(args)...);
}
template <typename Class,
typename... Args,
detail::enable_if_t<!std::is_constructible<Class, Args...>::value, int> = 0>
inline Class *construct_or_initialize(Args &&...args) {
return new Class{std::forward<Args>(args)...};
}
// Attempts to constructs an alias using a `Alias(Cpp &&)` constructor. This allows types with // Attempts to constructs an alias using a `Alias(Cpp &&)` constructor. This allows types with
// an alias to provide only a single Cpp factory function as long as the Alias can be // an alias to provide only a single Cpp factory function as long as the Alias can be
@ -69,12 +89,14 @@ inline Class *construct_or_initialize(Args &&...args) { return new Class{std::fo
// inherit all the base class constructors. // inherit all the base class constructors.
template <typename Class> template <typename Class>
void construct_alias_from_cpp(std::true_type /*is_alias_constructible*/, void construct_alias_from_cpp(std::true_type /*is_alias_constructible*/,
value_and_holder &v_h, Cpp<Class> &&base) { value_and_holder &v_h,
Cpp<Class> &&base) {
v_h.value_ptr() = new Alias<Class>(std::move(base)); v_h.value_ptr() = new Alias<Class>(std::move(base));
} }
template <typename Class> template <typename Class>
[[noreturn]] void construct_alias_from_cpp(std::false_type /*!is_alias_constructible*/, [[noreturn]] void construct_alias_from_cpp(std::false_type /*!is_alias_constructible*/,
value_and_holder &, Cpp<Class> &&) { value_and_holder &,
Cpp<Class> &&) {
throw type_error("pybind11::init(): unable to convert returned instance to required " throw type_error("pybind11::init(): unable to convert returned instance to required "
"alias class: no `Alias<Class>(Class &&)` constructor available"); "alias class: no `Alias<Class>(Class &&)` constructor available");
} }
@ -84,8 +106,8 @@ template <typename Class>
template <typename Class> template <typename Class>
void construct(...) { void construct(...) {
static_assert(!std::is_same<Class, Class>::value /* always false */, static_assert(!std::is_same<Class, Class>::value /* always false */,
"pybind11::init(): init function must return a compatible pointer, " "pybind11::init(): init function must return a compatible pointer, "
"holder, or value"); "holder, or value");
} }
// Pointer return v1: the factory function returns a class pointer for a registered class. // Pointer return v1: the factory function returns a class pointer for a registered class.
@ -94,6 +116,7 @@ void construct(...) {
// construct an Alias from the returned base instance. // construct an Alias from the returned base instance.
template <typename Class> template <typename Class>
void construct(value_and_holder &v_h, Cpp<Class> *ptr, bool need_alias) { void construct(value_and_holder &v_h, Cpp<Class> *ptr, bool need_alias) {
PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(need_alias);
no_nullptr(ptr); no_nullptr(ptr);
if (Class::has_alias && need_alias && !is_alias<Class>(ptr)) { if (Class::has_alias && need_alias && !is_alias<Class>(ptr)) {
// We're going to try to construct an alias by moving the cpp type. Whether or not // We're going to try to construct an alias by moving the cpp type. Whether or not
@ -105,7 +128,7 @@ void construct(value_and_holder &v_h, Cpp<Class> *ptr, bool need_alias) {
// the holder and destruction happens when we leave the C++ scope, and the holder // the holder and destruction happens when we leave the C++ scope, and the holder
// class gets to handle the destruction however it likes. // class gets to handle the destruction however it likes.
v_h.value_ptr() = ptr; v_h.value_ptr() = ptr;
v_h.set_instance_registered(true); // To prevent init_instance from registering it v_h.set_instance_registered(true); // To prevent init_instance from registering it
v_h.type->init_instance(v_h.inst, nullptr); // Set up the holder v_h.type->init_instance(v_h.inst, nullptr); // Set up the holder
Holder<Class> temp_holder(std::move(v_h.holder<Holder<Class>>())); // Steal the holder Holder<Class> temp_holder(std::move(v_h.holder<Holder<Class>>())); // Steal the holder
v_h.type->dealloc(v_h); // Destroys the moved-out holder remains, resets value ptr to null v_h.type->dealloc(v_h); // Destroys the moved-out holder remains, resets value ptr to null
@ -128,15 +151,18 @@ void construct(value_and_holder &v_h, Alias<Class> *alias_ptr, bool) {
// Holder return: copy its pointer, and move or copy the returned holder into the new instance's // Holder return: copy its pointer, and move or copy the returned holder into the new instance's
// holder. This also handles types like std::shared_ptr<T> and std::unique_ptr<T> where T is a // holder. This also handles types like std::shared_ptr<T> and std::unique_ptr<T> where T is a
// derived type (through those holder's implicit conversion from derived class holder constructors). // derived type (through those holder's implicit conversion from derived class holder
// constructors).
template <typename Class> template <typename Class>
void construct(value_and_holder &v_h, Holder<Class> holder, bool need_alias) { void construct(value_and_holder &v_h, Holder<Class> holder, bool need_alias) {
PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(need_alias);
auto *ptr = holder_helper<Holder<Class>>::get(holder); auto *ptr = holder_helper<Holder<Class>>::get(holder);
no_nullptr(ptr); no_nullptr(ptr);
// If we need an alias, check that the held pointer is actually an alias instance // If we need an alias, check that the held pointer is actually an alias instance
if (Class::has_alias && need_alias && !is_alias<Class>(ptr)) if (Class::has_alias && need_alias && !is_alias<Class>(ptr)) {
throw type_error("pybind11::init(): construction failed: returned holder-wrapped instance " throw type_error("pybind11::init(): construction failed: returned holder-wrapped instance "
"is not an alias instance"); "is not an alias instance");
}
v_h.value_ptr() = ptr; v_h.value_ptr() = ptr;
v_h.type->init_instance(v_h.inst, &holder); v_h.type->init_instance(v_h.inst, &holder);
@ -148,12 +174,14 @@ void construct(value_and_holder &v_h, Holder<Class> holder, bool need_alias) {
// need it, we simply move-construct the cpp value into a new instance. // need it, we simply move-construct the cpp value into a new instance.
template <typename Class> template <typename Class>
void construct(value_and_holder &v_h, Cpp<Class> &&result, bool need_alias) { void construct(value_and_holder &v_h, Cpp<Class> &&result, bool need_alias) {
static_assert(std::is_move_constructible<Cpp<Class>>::value, PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100(need_alias);
"pybind11::init() return-by-value factory function requires a movable class"); static_assert(is_move_constructible<Cpp<Class>>::value,
if (Class::has_alias && need_alias) "pybind11::init() return-by-value factory function requires a movable class");
if (Class::has_alias && need_alias) {
construct_alias_from_cpp<Class>(is_alias_constructible<Class>{}, v_h, std::move(result)); construct_alias_from_cpp<Class>(is_alias_constructible<Class>{}, v_h, std::move(result));
else } else {
v_h.value_ptr() = new Cpp<Class>(std::move(result)); v_h.value_ptr() = new Cpp<Class>(std::move(result));
}
} }
// return-by-value version 2: returning a value of the alias type itself. We move-construct an // return-by-value version 2: returning a value of the alias type itself. We move-construct an
@ -161,7 +189,8 @@ void construct(value_and_holder &v_h, Cpp<Class> &&result, bool need_alias) {
// cases where Alias initialization is always desired. // cases where Alias initialization is always desired.
template <typename Class> template <typename Class>
void construct(value_and_holder &v_h, Alias<Class> &&result, bool) { void construct(value_and_holder &v_h, Alias<Class> &&result, bool) {
static_assert(std::is_move_constructible<Alias<Class>>::value, static_assert(
is_move_constructible<Alias<Class>>::value,
"pybind11::init() return-by-alias-value factory function requires a movable alias class"); "pybind11::init() return-by-alias-value factory function requires a movable alias class");
v_h.value_ptr() = new Alias<Class>(std::move(result)); v_h.value_ptr() = new Alias<Class>(std::move(result));
} }
@ -170,48 +199,79 @@ void construct(value_and_holder &v_h, Alias<Class> &&result, bool) {
template <typename... Args> template <typename... Args>
struct constructor { struct constructor {
template <typename Class, typename... Extra, enable_if_t<!Class::has_alias, int> = 0> template <typename Class, typename... Extra, enable_if_t<!Class::has_alias, int> = 0>
static void execute(Class &cl, const Extra&... extra) { static void execute(Class &cl, const Extra &...extra) {
cl.def("__init__", [](value_and_holder &v_h, Args... args) { cl.def(
v_h.value_ptr() = construct_or_initialize<Cpp<Class>>(std::forward<Args>(args)...); "__init__",
}, is_new_style_constructor(), extra...); [](value_and_holder &v_h, Args... args) {
}
template <typename Class, typename... Extra,
enable_if_t<Class::has_alias &&
std::is_constructible<Cpp<Class>, Args...>::value, int> = 0>
static void execute(Class &cl, const Extra&... extra) {
cl.def("__init__", [](value_and_holder &v_h, Args... args) {
if (Py_TYPE(v_h.inst) == v_h.type->type)
v_h.value_ptr() = construct_or_initialize<Cpp<Class>>(std::forward<Args>(args)...); v_h.value_ptr() = construct_or_initialize<Cpp<Class>>(std::forward<Args>(args)...);
else },
v_h.value_ptr() = construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...); is_new_style_constructor(),
}, is_new_style_constructor(), extra...); extra...);
} }
template <typename Class, typename... Extra, template <
enable_if_t<Class::has_alias && typename Class,
!std::is_constructible<Cpp<Class>, Args...>::value, int> = 0> typename... Extra,
static void execute(Class &cl, const Extra&... extra) { enable_if_t<Class::has_alias && std::is_constructible<Cpp<Class>, Args...>::value, int>
cl.def("__init__", [](value_and_holder &v_h, Args... args) { = 0>
v_h.value_ptr() = construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...); static void execute(Class &cl, const Extra &...extra) {
}, is_new_style_constructor(), extra...); cl.def(
"__init__",
[](value_and_holder &v_h, Args... args) {
if (Py_TYPE(v_h.inst) == v_h.type->type) {
v_h.value_ptr()
= construct_or_initialize<Cpp<Class>>(std::forward<Args>(args)...);
} else {
v_h.value_ptr()
= construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...);
}
},
is_new_style_constructor(),
extra...);
}
template <
typename Class,
typename... Extra,
enable_if_t<Class::has_alias && !std::is_constructible<Cpp<Class>, Args...>::value, int>
= 0>
static void execute(Class &cl, const Extra &...extra) {
cl.def(
"__init__",
[](value_and_holder &v_h, Args... args) {
v_h.value_ptr()
= construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...);
},
is_new_style_constructor(),
extra...);
} }
}; };
// Implementing class for py::init_alias<...>() // Implementing class for py::init_alias<...>()
template <typename... Args> struct alias_constructor { template <typename... Args>
template <typename Class, typename... Extra, struct alias_constructor {
enable_if_t<Class::has_alias && std::is_constructible<Alias<Class>, Args...>::value, int> = 0> template <
static void execute(Class &cl, const Extra&... extra) { typename Class,
cl.def("__init__", [](value_and_holder &v_h, Args... args) { typename... Extra,
v_h.value_ptr() = construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...); enable_if_t<Class::has_alias && std::is_constructible<Alias<Class>, Args...>::value, int>
}, is_new_style_constructor(), extra...); = 0>
static void execute(Class &cl, const Extra &...extra) {
cl.def(
"__init__",
[](value_and_holder &v_h, Args... args) {
v_h.value_ptr()
= construct_or_initialize<Alias<Class>>(std::forward<Args>(args)...);
},
is_new_style_constructor(),
extra...);
} }
}; };
// Implementation class for py::init(Func) and py::init(Func, AliasFunc) // Implementation class for py::init(Func) and py::init(Func, AliasFunc)
template <typename CFunc, typename AFunc = void_type (*)(), template <typename CFunc,
typename = function_signature_t<CFunc>, typename = function_signature_t<AFunc>> typename AFunc = void_type (*)(),
typename = function_signature_t<CFunc>,
typename = function_signature_t<AFunc>>
struct factory; struct factory;
// Specialization for py::init(Func) // Specialization for py::init(Func)
@ -219,7 +279,8 @@ template <typename Func, typename Return, typename... Args>
struct factory<Func, void_type (*)(), Return(Args...)> { struct factory<Func, void_type (*)(), Return(Args...)> {
remove_reference_t<Func> class_factory; remove_reference_t<Func> class_factory;
factory(Func &&f) : class_factory(std::forward<Func>(f)) { } // NOLINTNEXTLINE(google-explicit-constructor)
factory(Func &&f) : class_factory(std::forward<Func>(f)) {}
// The given class either has no alias or has no separate alias factory; // The given class either has no alias or has no separate alias factory;
// this always constructs the class itself. If the class is registered with an alias // this always constructs the class itself. If the class is registered with an alias
@ -228,22 +289,32 @@ struct factory<Func, void_type (*)(), Return(Args...)> {
// instance, or the alias needs to be constructible from a `Class &&` argument. // instance, or the alias needs to be constructible from a `Class &&` argument.
template <typename Class, typename... Extra> template <typename Class, typename... Extra>
void execute(Class &cl, const Extra &...extra) && { void execute(Class &cl, const Extra &...extra) && {
#if defined(PYBIND11_CPP14) #if defined(PYBIND11_CPP14)
cl.def("__init__", [func = std::move(class_factory)] cl.def(
#else "__init__",
[func = std::move(class_factory)]
#else
auto &func = class_factory; auto &func = class_factory;
cl.def("__init__", [func] cl.def(
#endif "__init__",
(value_and_holder &v_h, Args... args) { [func]
construct<Class>(v_h, func(std::forward<Args>(args)...), #endif
Py_TYPE(v_h.inst) != v_h.type->type); (value_and_holder &v_h, Args... args) {
}, is_new_style_constructor(), extra...); construct<Class>(
v_h, func(std::forward<Args>(args)...), Py_TYPE(v_h.inst) != v_h.type->type);
},
is_new_style_constructor(),
extra...);
} }
}; };
// Specialization for py::init(Func, AliasFunc) // Specialization for py::init(Func, AliasFunc)
template <typename CFunc, typename AFunc, template <typename CFunc,
typename CReturn, typename... CArgs, typename AReturn, typename... AArgs> typename AFunc,
typename CReturn,
typename... CArgs,
typename AReturn,
typename... AArgs>
struct factory<CFunc, AFunc, CReturn(CArgs...), AReturn(AArgs...)> { struct factory<CFunc, AFunc, CReturn(CArgs...), AReturn(AArgs...)> {
static_assert(sizeof...(CArgs) == sizeof...(AArgs), static_assert(sizeof...(CArgs) == sizeof...(AArgs),
"pybind11::init(class_factory, alias_factory): class and alias factories " "pybind11::init(class_factory, alias_factory): class and alias factories "
@ -256,29 +327,37 @@ struct factory<CFunc, AFunc, CReturn(CArgs...), AReturn(AArgs...)> {
remove_reference_t<AFunc> alias_factory; remove_reference_t<AFunc> alias_factory;
factory(CFunc &&c, AFunc &&a) factory(CFunc &&c, AFunc &&a)
: class_factory(std::forward<CFunc>(c)), alias_factory(std::forward<AFunc>(a)) { } : class_factory(std::forward<CFunc>(c)), alias_factory(std::forward<AFunc>(a)) {}
// The class factory is called when the `self` type passed to `__init__` is the direct // The class factory is called when the `self` type passed to `__init__` is the direct
// class (i.e. not inherited), the alias factory when `self` is a Python-side subtype. // class (i.e. not inherited), the alias factory when `self` is a Python-side subtype.
template <typename Class, typename... Extra> template <typename Class, typename... Extra>
void execute(Class &cl, const Extra&... extra) && { void execute(Class &cl, const Extra &...extra) && {
static_assert(Class::has_alias, "The two-argument version of `py::init()` can " static_assert(Class::has_alias,
"only be used if the class has an alias"); "The two-argument version of `py::init()` can "
#if defined(PYBIND11_CPP14) "only be used if the class has an alias");
cl.def("__init__", [class_func = std::move(class_factory), alias_func = std::move(alias_factory)] #if defined(PYBIND11_CPP14)
#else cl.def(
"__init__",
[class_func = std::move(class_factory), alias_func = std::move(alias_factory)]
#else
auto &class_func = class_factory; auto &class_func = class_factory;
auto &alias_func = alias_factory; auto &alias_func = alias_factory;
cl.def("__init__", [class_func, alias_func] cl.def(
#endif "__init__",
(value_and_holder &v_h, CArgs... args) { [class_func, alias_func]
if (Py_TYPE(v_h.inst) == v_h.type->type) #endif
// If the instance type equals the registered type we don't have inheritance, so (value_and_holder &v_h, CArgs... args) {
// don't need the alias and can construct using the class function: if (Py_TYPE(v_h.inst) == v_h.type->type) {
construct<Class>(v_h, class_func(std::forward<CArgs>(args)...), false); // If the instance type equals the registered type we don't have inheritance,
else // so don't need the alias and can construct using the class function:
construct<Class>(v_h, alias_func(std::forward<CArgs>(args)...), true); construct<Class>(v_h, class_func(std::forward<CArgs>(args)...), false);
}, is_new_style_constructor(), extra...); } else {
construct<Class>(v_h, alias_func(std::forward<CArgs>(args)...), true);
}
},
is_new_style_constructor(),
extra...);
} }
}; };
@ -289,20 +368,34 @@ void setstate(value_and_holder &v_h, T &&result, bool need_alias) {
} }
/// Set both the C++ and Python states /// Set both the C++ and Python states
template <typename Class, typename T, typename O, template <typename Class,
typename T,
typename O,
enable_if_t<std::is_convertible<O, handle>::value, int> = 0> enable_if_t<std::is_convertible<O, handle>::value, int> = 0>
void setstate(value_and_holder &v_h, std::pair<T, O> &&result, bool need_alias) { void setstate(value_and_holder &v_h, std::pair<T, O> &&result, bool need_alias) {
construct<Class>(v_h, std::move(result.first), need_alias); construct<Class>(v_h, std::move(result.first), need_alias);
setattr((PyObject *) v_h.inst, "__dict__", result.second); auto d = handle(result.second);
if (PyDict_Check(d.ptr()) && PyDict_Size(d.ptr()) == 0) {
// Skipping setattr below, to not force use of py::dynamic_attr() for Class unnecessarily.
// See PR #2972 for details.
return;
}
setattr((PyObject *) v_h.inst, "__dict__", d);
} }
/// Implementation for py::pickle(GetState, SetState) /// Implementation for py::pickle(GetState, SetState)
template <typename Get, typename Set, template <typename Get,
typename = function_signature_t<Get>, typename = function_signature_t<Set>> typename Set,
typename = function_signature_t<Get>,
typename = function_signature_t<Set>>
struct pickle_factory; struct pickle_factory;
template <typename Get, typename Set, template <typename Get,
typename RetState, typename Self, typename NewInstance, typename ArgState> typename Set,
typename RetState,
typename Self,
typename NewInstance,
typename ArgState>
struct pickle_factory<Get, Set, RetState(Self), NewInstance(ArgState)> { struct pickle_factory<Get, Set, RetState(Self), NewInstance(ArgState)> {
static_assert(std::is_same<intrinsic_t<RetState>, intrinsic_t<ArgState>>::value, static_assert(std::is_same<intrinsic_t<RetState>, intrinsic_t<ArgState>>::value,
"The type returned by `__getstate__` must be the same " "The type returned by `__getstate__` must be the same "
@ -311,26 +404,31 @@ struct pickle_factory<Get, Set, RetState(Self), NewInstance(ArgState)> {
remove_reference_t<Get> get; remove_reference_t<Get> get;
remove_reference_t<Set> set; remove_reference_t<Set> set;
pickle_factory(Get get, Set set) pickle_factory(Get get, Set set) : get(std::forward<Get>(get)), set(std::forward<Set>(set)) {}
: get(std::forward<Get>(get)), set(std::forward<Set>(set)) { }
template <typename Class, typename... Extra> template <typename Class, typename... Extra>
void execute(Class &cl, const Extra &...extra) && { void execute(Class &cl, const Extra &...extra) && {
cl.def("__getstate__", std::move(get)); cl.def("__getstate__", std::move(get));
#if defined(PYBIND11_CPP14) #if defined(PYBIND11_CPP14)
cl.def("__setstate__", [func = std::move(set)] cl.def(
"__setstate__",
[func = std::move(set)]
#else #else
auto &func = set; auto &func = set;
cl.def("__setstate__", [func] cl.def(
"__setstate__",
[func]
#endif #endif
(value_and_holder &v_h, ArgState state) { (value_and_holder &v_h, ArgState state) {
setstate<Class>(v_h, func(std::forward<ArgState>(state)), setstate<Class>(
Py_TYPE(v_h.inst) != v_h.type->type); v_h, func(std::forward<ArgState>(state)), Py_TYPE(v_h.inst) != v_h.type->type);
}, is_new_style_constructor(), extra...); },
is_new_style_constructor(),
extra...);
} }
}; };
PYBIND11_NAMESPACE_END(initimpl) PYBIND11_NAMESPACE_END(initimpl)
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(pybind11) PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -9,10 +9,52 @@
#pragma once #pragma once
#include "common.h"
#if defined(WITH_THREAD) && defined(PYBIND11_SIMPLE_GIL_MANAGEMENT)
# include "../gil.h"
#endif
#include "../pytypes.h" #include "../pytypes.h"
#include <exception>
/// Tracks the `internals` and `type_info` ABI version independent of the main library version.
///
/// Some portions of the code use an ABI that is conditional depending on this
/// version number. That allows ABI-breaking changes to be "pre-implemented".
/// Once the default version number is incremented, the conditional logic that
/// no longer applies can be removed. Additionally, users that need not
/// maintain ABI compatibility can increase the version number in order to take
/// advantage of any functionality/efficiency improvements that depend on the
/// newer ABI.
///
/// WARNING: If you choose to manually increase the ABI version, note that
/// pybind11 may not be tested as thoroughly with a non-default ABI version, and
/// further ABI-incompatible changes may be made before the ABI is officially
/// changed to the new version.
#ifndef PYBIND11_INTERNALS_VERSION
# if PY_VERSION_HEX >= 0x030C0000 || defined(_MSC_VER)
// Version bump for Python 3.12+, before first 3.12 beta release.
// Version bump for MSVC piggy-backed on PR #4779. See comments there.
# define PYBIND11_INTERNALS_VERSION 5
# else
# define PYBIND11_INTERNALS_VERSION 4
# endif
#endif
// This requirement is mainly to reduce the support burden (see PR #4570).
static_assert(PY_VERSION_HEX < 0x030C0000 || PYBIND11_INTERNALS_VERSION >= 5,
"pybind11 ABI version 5 is the minimum for Python 3.12+");
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
using ExceptionTranslator = void (*)(std::exception_ptr);
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
constexpr const char *internals_function_record_capsule_name = "pybind11_function_record_capsule";
// Forward declarations // Forward declarations
inline PyTypeObject *make_static_property_type(); inline PyTypeObject *make_static_property_type();
inline PyTypeObject *make_default_metaclass(); inline PyTypeObject *make_default_metaclass();
@ -21,30 +63,64 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass);
// The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new // The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new
// Thread Specific Storage (TSS) API. // Thread Specific Storage (TSS) API.
#if PY_VERSION_HEX >= 0x03070000 #if PY_VERSION_HEX >= 0x03070000
# define PYBIND11_TLS_KEY_INIT(var) Py_tss_t *var = nullptr // Avoid unnecessary allocation of `Py_tss_t`, since we cannot use
# define PYBIND11_TLS_GET_VALUE(key) PyThread_tss_get((key)) // `Py_LIMITED_API` anyway.
# define PYBIND11_TLS_REPLACE_VALUE(key, value) PyThread_tss_set((key), (value)) # if PYBIND11_INTERNALS_VERSION > 4
# define PYBIND11_TLS_DELETE_VALUE(key) PyThread_tss_set((key), nullptr) # define PYBIND11_TLS_KEY_REF Py_tss_t &
# define PYBIND11_TLS_FREE(key) PyThread_tss_free(key) # if defined(__clang__)
#else # define PYBIND11_TLS_KEY_INIT(var) \
// Usually an int but a long on Cygwin64 with Python 3.x _Pragma("clang diagnostic push") /**/ \
# define PYBIND11_TLS_KEY_INIT(var) decltype(PyThread_create_key()) var = 0 _Pragma("clang diagnostic ignored \"-Wmissing-field-initializers\"") /**/ \
# define PYBIND11_TLS_GET_VALUE(key) PyThread_get_key_value((key)) Py_tss_t var \
# if PY_MAJOR_VERSION < 3 = Py_tss_NEEDS_INIT; \
# define PYBIND11_TLS_DELETE_VALUE(key) \ _Pragma("clang diagnostic pop")
PyThread_delete_key_value(key) # elif defined(__GNUC__) && !defined(__INTEL_COMPILER)
# define PYBIND11_TLS_REPLACE_VALUE(key, value) \ # define PYBIND11_TLS_KEY_INIT(var) \
do { \ _Pragma("GCC diagnostic push") /**/ \
PyThread_delete_key_value((key)); \ _Pragma("GCC diagnostic ignored \"-Wmissing-field-initializers\"") /**/ \
PyThread_set_key_value((key), (value)); \ Py_tss_t var \
} while (false) = Py_tss_NEEDS_INIT; \
_Pragma("GCC diagnostic pop")
# else
# define PYBIND11_TLS_KEY_INIT(var) Py_tss_t var = Py_tss_NEEDS_INIT;
# endif
# define PYBIND11_TLS_KEY_CREATE(var) (PyThread_tss_create(&(var)) == 0)
# define PYBIND11_TLS_GET_VALUE(key) PyThread_tss_get(&(key))
# define PYBIND11_TLS_REPLACE_VALUE(key, value) PyThread_tss_set(&(key), (value))
# define PYBIND11_TLS_DELETE_VALUE(key) PyThread_tss_set(&(key), nullptr)
# define PYBIND11_TLS_FREE(key) PyThread_tss_delete(&(key))
# else # else
# define PYBIND11_TLS_DELETE_VALUE(key) \ # define PYBIND11_TLS_KEY_REF Py_tss_t *
PyThread_set_key_value((key), nullptr) # define PYBIND11_TLS_KEY_INIT(var) Py_tss_t *var = nullptr;
# define PYBIND11_TLS_REPLACE_VALUE(key, value) \ # define PYBIND11_TLS_KEY_CREATE(var) \
PyThread_set_key_value((key), (value)) (((var) = PyThread_tss_alloc()) != nullptr && (PyThread_tss_create((var)) == 0))
# define PYBIND11_TLS_GET_VALUE(key) PyThread_tss_get((key))
# define PYBIND11_TLS_REPLACE_VALUE(key, value) PyThread_tss_set((key), (value))
# define PYBIND11_TLS_DELETE_VALUE(key) PyThread_tss_set((key), nullptr)
# define PYBIND11_TLS_FREE(key) PyThread_tss_free(key)
# endif # endif
# define PYBIND11_TLS_FREE(key) (void)key #else
// Usually an int but a long on Cygwin64 with Python 3.x
# define PYBIND11_TLS_KEY_REF decltype(PyThread_create_key())
# define PYBIND11_TLS_KEY_INIT(var) PYBIND11_TLS_KEY_REF var = 0;
# define PYBIND11_TLS_KEY_CREATE(var) (((var) = PyThread_create_key()) != -1)
# define PYBIND11_TLS_GET_VALUE(key) PyThread_get_key_value((key))
# if defined(PYPY_VERSION)
// On CPython < 3.4 and on PyPy, `PyThread_set_key_value` strangely does not set
// the value if it has already been set. Instead, it must first be deleted and
// then set again.
inline void tls_replace_value(PYBIND11_TLS_KEY_REF key, void *value) {
PyThread_delete_key_value(key);
PyThread_set_key_value(key, value);
}
# define PYBIND11_TLS_DELETE_VALUE(key) PyThread_delete_key_value(key)
# define PYBIND11_TLS_REPLACE_VALUE(key, value) \
::pybind11::detail::tls_replace_value((key), (value))
# else
# define PYBIND11_TLS_DELETE_VALUE(key) PyThread_set_key_value((key), nullptr)
# define PYBIND11_TLS_REPLACE_VALUE(key, value) PyThread_set_key_value((key), (value))
# endif
# define PYBIND11_TLS_FREE(key) (void) key
#endif #endif
// Python loads modules by default with dlopen with the RTLD_LOCAL flag; under libc++ and possibly // Python loads modules by default with dlopen with the RTLD_LOCAL flag; under libc++ and possibly
@ -53,7 +129,8 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass);
// libstdc++, this doesn't happen: equality and the type_index hash are based on the type name, // libstdc++, this doesn't happen: equality and the type_index hash are based on the type name,
// which works. If not under a known-good stl, provide our own name-based hash and equality // which works. If not under a known-good stl, provide our own name-based hash and equality
// functions that use the type name. // functions that use the type name.
#if defined(__GLIBCXX__) #if (PYBIND11_INTERNALS_VERSION <= 4 && defined(__GLIBCXX__)) \
|| (PYBIND11_INTERNALS_VERSION >= 5 && !defined(_LIBCPP_VERSION))
inline bool same_type(const std::type_info &lhs, const std::type_info &rhs) { return lhs == rhs; } inline bool same_type(const std::type_info &lhs, const std::type_info &rhs) { return lhs == rhs; }
using type_hash = std::hash<std::type_index>; using type_hash = std::hash<std::type_index>;
using type_equal_to = std::equal_to<std::type_index>; using type_equal_to = std::equal_to<std::type_index>;
@ -66,8 +143,9 @@ struct type_hash {
size_t operator()(const std::type_index &t) const { size_t operator()(const std::type_index &t) const {
size_t hash = 5381; size_t hash = 5381;
const char *ptr = t.name(); const char *ptr = t.name();
while (auto c = static_cast<unsigned char>(*ptr++)) while (auto c = static_cast<unsigned char>(*ptr++)) {
hash = (hash * 33) ^ c; hash = (hash * 33) ^ c;
}
return hash; return hash;
} }
}; };
@ -83,9 +161,9 @@ template <typename value_type>
using type_map = std::unordered_map<std::type_index, value_type, type_hash, type_equal_to>; using type_map = std::unordered_map<std::type_index, value_type, type_hash, type_equal_to>;
struct override_hash { struct override_hash {
inline size_t operator()(const std::pair<const PyObject *, const char *>& v) const { inline size_t operator()(const std::pair<const PyObject *, const char *> &v) const {
size_t value = std::hash<const void *>()(v.first); size_t value = std::hash<const void *>()(v.first);
value ^= std::hash<const void *>()(v.second) + 0x9e3779b9 + (value<<6) + (value>>2); value ^= std::hash<const void *>()(v.second) + 0x9e3779b9 + (value << 6) + (value >> 2);
return value; return value;
} }
}; };
@ -94,30 +172,56 @@ struct override_hash {
/// Whenever binary incompatible changes are made to this structure, /// Whenever binary incompatible changes are made to this structure,
/// `PYBIND11_INTERNALS_VERSION` must be incremented. /// `PYBIND11_INTERNALS_VERSION` must be incremented.
struct internals { struct internals {
type_map<type_info *> registered_types_cpp; // std::type_index -> pybind11's type information // std::type_index -> pybind11's type information
std::unordered_map<PyTypeObject *, std::vector<type_info *>> registered_types_py; // PyTypeObject* -> base type_info(s) type_map<type_info *> registered_types_cpp;
std::unordered_multimap<const void *, instance*> registered_instances; // void * -> instance* // PyTypeObject* -> base type_info(s)
std::unordered_set<std::pair<const PyObject *, const char *>, override_hash> inactive_override_cache; std::unordered_map<PyTypeObject *, std::vector<type_info *>> registered_types_py;
std::unordered_multimap<const void *, instance *> registered_instances; // void * -> instance*
std::unordered_set<std::pair<const PyObject *, const char *>, override_hash>
inactive_override_cache;
type_map<std::vector<bool (*)(PyObject *, void *&)>> direct_conversions; type_map<std::vector<bool (*)(PyObject *, void *&)>> direct_conversions;
std::unordered_map<const PyObject *, std::vector<PyObject *>> patients; std::unordered_map<const PyObject *, std::vector<PyObject *>> patients;
std::forward_list<void (*) (std::exception_ptr)> registered_exception_translators; std::forward_list<ExceptionTranslator> registered_exception_translators;
std::unordered_map<std::string, void *> shared_data; // Custom data to be shared across extensions std::unordered_map<std::string, void *> shared_data; // Custom data to be shared across
std::vector<PyObject *> loader_patient_stack; // Used by `loader_life_support` // extensions
std::forward_list<std::string> static_strings; // Stores the std::strings backing detail::c_str() #if PYBIND11_INTERNALS_VERSION == 4
std::vector<PyObject *> unused_loader_patient_stack_remove_at_v5;
#endif
std::forward_list<std::string> static_strings; // Stores the std::strings backing
// detail::c_str()
PyTypeObject *static_property_type; PyTypeObject *static_property_type;
PyTypeObject *default_metaclass; PyTypeObject *default_metaclass;
PyObject *instance_base; PyObject *instance_base;
#if defined(WITH_THREAD) #if defined(WITH_THREAD)
PYBIND11_TLS_KEY_INIT(tstate); // Unused if PYBIND11_SIMPLE_GIL_MANAGEMENT is defined:
PYBIND11_TLS_KEY_INIT(tstate)
# if PYBIND11_INTERNALS_VERSION > 4
PYBIND11_TLS_KEY_INIT(loader_life_support_tls_key)
# endif // PYBIND11_INTERNALS_VERSION > 4
// Unused if PYBIND11_SIMPLE_GIL_MANAGEMENT is defined:
PyInterpreterState *istate = nullptr; PyInterpreterState *istate = nullptr;
# if PYBIND11_INTERNALS_VERSION > 4
// Note that we have to use a std::string to allocate memory to ensure a unique address
// We want unique addresses since we use pointer equality to compare function records
std::string function_record_capsule_name = internals_function_record_capsule_name;
# endif
internals() = default;
internals(const internals &other) = delete;
internals &operator=(const internals &other) = delete;
~internals() { ~internals() {
# if PYBIND11_INTERNALS_VERSION > 4
PYBIND11_TLS_FREE(loader_life_support_tls_key);
# endif // PYBIND11_INTERNALS_VERSION > 4
// This destructor is called *after* Py_Finalize() in finalize_interpreter(). // This destructor is called *after* Py_Finalize() in finalize_interpreter().
// That *SHOULD BE* fine. The following details what happens whe PyThread_tss_free is called. // That *SHOULD BE* fine. The following details what happens when PyThread_tss_free is
// PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does nothing. // called. PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does
// PyThread_tss_free calls PyThread_tss_delete and PyMem_RawFree. // nothing. PyThread_tss_free calls PyThread_tss_delete and PyMem_RawFree.
// PyThread_tss_delete just calls TlsFree (on Windows) or pthread_key_delete (on *NIX). Neither // PyThread_tss_delete just calls TlsFree (on Windows) or pthread_key_delete (on *NIX).
// of those have anything to do with CPython internals. // Neither of those have anything to do with CPython internals. PyMem_RawFree *requires*
// PyMem_RawFree *requires* that the `tstate` be allocated with the CPython allocator. // that the `tstate` be allocated with the CPython allocator.
PYBIND11_TLS_FREE(tstate); PYBIND11_TLS_FREE(tstate);
} }
#endif #endif
@ -132,14 +236,16 @@ struct type_info {
void *(*operator_new)(size_t); void *(*operator_new)(size_t);
void (*init_instance)(instance *, const void *); void (*init_instance)(instance *, const void *);
void (*dealloc)(value_and_holder &v_h); void (*dealloc)(value_and_holder &v_h);
std::vector<PyObject *(*)(PyObject *, PyTypeObject *)> implicit_conversions; std::vector<PyObject *(*) (PyObject *, PyTypeObject *)> implicit_conversions;
std::vector<std::pair<const std::type_info *, void *(*)(void *)>> implicit_casts; std::vector<std::pair<const std::type_info *, void *(*) (void *)>> implicit_casts;
std::vector<bool (*)(PyObject *, void *&)> *direct_conversions; std::vector<bool (*)(PyObject *, void *&)> *direct_conversions;
buffer_info *(*get_buffer)(PyObject *, void *) = nullptr; buffer_info *(*get_buffer)(PyObject *, void *) = nullptr;
void *get_buffer_data = nullptr; void *get_buffer_data = nullptr;
void *(*module_local_load)(PyObject *, const type_info *) = nullptr; void *(*module_local_load)(PyObject *, const type_info *) = nullptr;
/* A simple type never occurs as a (direct or indirect) parent /* A simple type never occurs as a (direct or indirect) parent
* of a class that makes use of multiple inheritance */ * of a class that makes use of multiple inheritance.
* A type can be simple even if it has non-simple ancestors as long as it has no descendants.
*/
bool simple_type : 1; bool simple_type : 1;
/* True if there is no multiple inheritance in this type's inheritance tree */ /* True if there is no multiple inheritance in this type's inheritance tree */
bool simple_ancestors : 1; bool simple_ancestors : 1;
@ -149,72 +255,78 @@ struct type_info {
bool module_local : 1; bool module_local : 1;
}; };
/// Tracks the `internals` and `type_info` ABI version independent of the main library version
#define PYBIND11_INTERNALS_VERSION 4
/// On MSVC, debug and release builds are not ABI-compatible! /// On MSVC, debug and release builds are not ABI-compatible!
#if defined(_MSC_VER) && defined(_DEBUG) #if defined(_MSC_VER) && defined(_DEBUG)
# define PYBIND11_BUILD_TYPE "_debug" # define PYBIND11_BUILD_TYPE "_debug"
#else #else
# define PYBIND11_BUILD_TYPE "" # define PYBIND11_BUILD_TYPE ""
#endif #endif
/// Let's assume that different compilers are ABI-incompatible. /// Let's assume that different compilers are ABI-incompatible.
/// A user can manually set this string if they know their /// A user can manually set this string if they know their
/// compiler is compatible. /// compiler is compatible.
#ifndef PYBIND11_COMPILER_TYPE #ifndef PYBIND11_COMPILER_TYPE
# if defined(_MSC_VER) # if defined(_MSC_VER)
# define PYBIND11_COMPILER_TYPE "_msvc" # define PYBIND11_COMPILER_TYPE "_msvc"
# elif defined(__INTEL_COMPILER) # elif defined(__INTEL_COMPILER)
# define PYBIND11_COMPILER_TYPE "_icc" # define PYBIND11_COMPILER_TYPE "_icc"
# elif defined(__clang__) # elif defined(__clang__)
# define PYBIND11_COMPILER_TYPE "_clang" # define PYBIND11_COMPILER_TYPE "_clang"
# elif defined(__PGI) # elif defined(__PGI)
# define PYBIND11_COMPILER_TYPE "_pgi" # define PYBIND11_COMPILER_TYPE "_pgi"
# elif defined(__MINGW32__) # elif defined(__MINGW32__)
# define PYBIND11_COMPILER_TYPE "_mingw" # define PYBIND11_COMPILER_TYPE "_mingw"
# elif defined(__CYGWIN__) # elif defined(__CYGWIN__)
# define PYBIND11_COMPILER_TYPE "_gcc_cygwin" # define PYBIND11_COMPILER_TYPE "_gcc_cygwin"
# elif defined(__GNUC__) # elif defined(__GNUC__)
# define PYBIND11_COMPILER_TYPE "_gcc" # define PYBIND11_COMPILER_TYPE "_gcc"
# else # else
# define PYBIND11_COMPILER_TYPE "_unknown" # define PYBIND11_COMPILER_TYPE "_unknown"
# endif # endif
#endif #endif
/// Also standard libs /// Also standard libs
#ifndef PYBIND11_STDLIB #ifndef PYBIND11_STDLIB
# if defined(_LIBCPP_VERSION) # if defined(_LIBCPP_VERSION)
# define PYBIND11_STDLIB "_libcpp" # define PYBIND11_STDLIB "_libcpp"
# elif defined(__GLIBCXX__) || defined(__GLIBCPP__) # elif defined(__GLIBCXX__) || defined(__GLIBCPP__)
# define PYBIND11_STDLIB "_libstdcpp" # define PYBIND11_STDLIB "_libstdcpp"
# else # else
# define PYBIND11_STDLIB "" # define PYBIND11_STDLIB ""
# endif # endif
#endif #endif
/// On Linux/OSX, changes in __GXX_ABI_VERSION__ indicate ABI incompatibility. /// On Linux/OSX, changes in __GXX_ABI_VERSION__ indicate ABI incompatibility.
/// On MSVC, changes in _MSC_VER may indicate ABI incompatibility (#2898).
#ifndef PYBIND11_BUILD_ABI #ifndef PYBIND11_BUILD_ABI
# if defined(__GXX_ABI_VERSION) # if defined(__GXX_ABI_VERSION)
# define PYBIND11_BUILD_ABI "_cxxabi" PYBIND11_TOSTRING(__GXX_ABI_VERSION) # define PYBIND11_BUILD_ABI "_cxxabi" PYBIND11_TOSTRING(__GXX_ABI_VERSION)
# else # elif defined(_MSC_VER)
# define PYBIND11_BUILD_ABI "" # define PYBIND11_BUILD_ABI "_mscver" PYBIND11_TOSTRING(_MSC_VER)
# endif # else
# define PYBIND11_BUILD_ABI ""
# endif
#endif #endif
#ifndef PYBIND11_INTERNALS_KIND #ifndef PYBIND11_INTERNALS_KIND
# if defined(WITH_THREAD) # if defined(WITH_THREAD)
# define PYBIND11_INTERNALS_KIND "" # define PYBIND11_INTERNALS_KIND ""
# else # else
# define PYBIND11_INTERNALS_KIND "_without_thread" # define PYBIND11_INTERNALS_KIND "_without_thread"
# endif # endif
#endif #endif
#define PYBIND11_INTERNALS_ID "__pybind11_internals_v" \ #define PYBIND11_PLATFORM_ABI_ID \
PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) PYBIND11_INTERNALS_KIND PYBIND11_COMPILER_TYPE PYBIND11_STDLIB PYBIND11_BUILD_ABI PYBIND11_BUILD_TYPE "__" PYBIND11_INTERNALS_KIND PYBIND11_COMPILER_TYPE PYBIND11_STDLIB PYBIND11_BUILD_ABI \
PYBIND11_BUILD_TYPE
#define PYBIND11_MODULE_LOCAL_ID "__pybind11_module_local_v" \ #define PYBIND11_INTERNALS_ID \
PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) PYBIND11_INTERNALS_KIND PYBIND11_COMPILER_TYPE PYBIND11_STDLIB PYBIND11_BUILD_ABI PYBIND11_BUILD_TYPE "__" "__pybind11_internals_v" PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) \
PYBIND11_PLATFORM_ABI_ID "__"
#define PYBIND11_MODULE_LOCAL_ID \
"__pybind11_module_local_v" PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) \
PYBIND11_PLATFORM_ABI_ID "__"
/// Each module locally stores a pointer to the `internals` data. The data /// Each module locally stores a pointer to the `internals` data. The data
/// itself is shared among modules with the same `PYBIND11_INTERNALS_ID`. /// itself is shared among modules with the same `PYBIND11_INTERNALS_ID`.
@ -223,21 +335,93 @@ inline internals **&get_internals_pp() {
return internals_pp; return internals_pp;
} }
// forward decl
inline void translate_exception(std::exception_ptr);
template <class T,
enable_if_t<std::is_same<std::nested_exception, remove_cvref_t<T>>::value, int> = 0>
bool handle_nested_exception(const T &exc, const std::exception_ptr &p) {
std::exception_ptr nested = exc.nested_ptr();
if (nested != nullptr && nested != p) {
translate_exception(nested);
return true;
}
return false;
}
template <class T,
enable_if_t<!std::is_same<std::nested_exception, remove_cvref_t<T>>::value, int> = 0>
bool handle_nested_exception(const T &exc, const std::exception_ptr &p) {
if (const auto *nep = dynamic_cast<const std::nested_exception *>(std::addressof(exc))) {
return handle_nested_exception(*nep, p);
}
return false;
}
inline bool raise_err(PyObject *exc_type, const char *msg) {
if (PyErr_Occurred()) {
raise_from(exc_type, msg);
return true;
}
set_error(exc_type, msg);
return false;
}
inline void translate_exception(std::exception_ptr p) { inline void translate_exception(std::exception_ptr p) {
if (!p) {
return;
}
try { try {
if (p) std::rethrow_exception(p); std::rethrow_exception(p);
} catch (error_already_set &e) { e.restore(); return; } catch (error_already_set &e) {
} catch (const builtin_exception &e) { e.set_error(); return; handle_nested_exception(e, p);
} catch (const std::bad_alloc &e) { PyErr_SetString(PyExc_MemoryError, e.what()); return; e.restore();
} catch (const std::domain_error &e) { PyErr_SetString(PyExc_ValueError, e.what()); return; return;
} catch (const std::invalid_argument &e) { PyErr_SetString(PyExc_ValueError, e.what()); return; } catch (const builtin_exception &e) {
} catch (const std::length_error &e) { PyErr_SetString(PyExc_ValueError, e.what()); return; // Could not use template since it's an abstract class.
} catch (const std::out_of_range &e) { PyErr_SetString(PyExc_IndexError, e.what()); return; if (const auto *nep = dynamic_cast<const std::nested_exception *>(std::addressof(e))) {
} catch (const std::range_error &e) { PyErr_SetString(PyExc_ValueError, e.what()); return; handle_nested_exception(*nep, p);
} catch (const std::overflow_error &e) { PyErr_SetString(PyExc_OverflowError, e.what()); return; }
} catch (const std::exception &e) { PyErr_SetString(PyExc_RuntimeError, e.what()); return; e.set_error();
return;
} catch (const std::bad_alloc &e) {
handle_nested_exception(e, p);
raise_err(PyExc_MemoryError, e.what());
return;
} catch (const std::domain_error &e) {
handle_nested_exception(e, p);
raise_err(PyExc_ValueError, e.what());
return;
} catch (const std::invalid_argument &e) {
handle_nested_exception(e, p);
raise_err(PyExc_ValueError, e.what());
return;
} catch (const std::length_error &e) {
handle_nested_exception(e, p);
raise_err(PyExc_ValueError, e.what());
return;
} catch (const std::out_of_range &e) {
handle_nested_exception(e, p);
raise_err(PyExc_IndexError, e.what());
return;
} catch (const std::range_error &e) {
handle_nested_exception(e, p);
raise_err(PyExc_ValueError, e.what());
return;
} catch (const std::overflow_error &e) {
handle_nested_exception(e, p);
raise_err(PyExc_OverflowError, e.what());
return;
} catch (const std::exception &e) {
handle_nested_exception(e, p);
raise_err(PyExc_RuntimeError, e.what());
return;
} catch (const std::nested_exception &e) {
handle_nested_exception(e, p);
raise_err(PyExc_RuntimeError, "Caught an unknown nested exception!");
return;
} catch (...) { } catch (...) {
PyErr_SetString(PyExc_RuntimeError, "Caught an unknown exception!"); raise_err(PyExc_RuntimeError, "Caught an unknown exception!");
return; return;
} }
} }
@ -245,64 +429,117 @@ inline void translate_exception(std::exception_ptr p) {
#if !defined(__GLIBCXX__) #if !defined(__GLIBCXX__)
inline void translate_local_exception(std::exception_ptr p) { inline void translate_local_exception(std::exception_ptr p) {
try { try {
if (p) std::rethrow_exception(p); if (p) {
} catch (error_already_set &e) { e.restore(); return; std::rethrow_exception(p);
} catch (const builtin_exception &e) { e.set_error(); return; }
} catch (error_already_set &e) {
e.restore();
return;
} catch (const builtin_exception &e) {
e.set_error();
return;
} }
} }
#endif #endif
/// Return a reference to the current `internals` data inline object get_python_state_dict() {
PYBIND11_NOINLINE inline internals &get_internals() { object state_dict;
auto **&internals_pp = get_internals_pp(); #if PYBIND11_INTERNALS_VERSION <= 4 || PY_VERSION_HEX < 0x03080000 || defined(PYPY_VERSION)
if (internals_pp && *internals_pp) state_dict = reinterpret_borrow<object>(PyEval_GetBuiltins());
return **internals_pp; #else
# if PY_VERSION_HEX < 0x03090000
PyInterpreterState *istate = _PyInterpreterState_Get();
# else
PyInterpreterState *istate = PyInterpreterState_Get();
# endif
if (istate) {
state_dict = reinterpret_borrow<object>(PyInterpreterState_GetDict(istate));
}
#endif
if (!state_dict) {
raise_from(PyExc_SystemError, "pybind11::detail::get_python_state_dict() FAILED");
throw error_already_set();
}
return state_dict;
}
inline object get_internals_obj_from_state_dict(handle state_dict) {
return reinterpret_borrow<object>(dict_getitemstring(state_dict.ptr(), PYBIND11_INTERNALS_ID));
}
inline internals **get_internals_pp_from_capsule(handle obj) {
void *raw_ptr = PyCapsule_GetPointer(obj.ptr(), /*name=*/nullptr);
if (raw_ptr == nullptr) {
raise_from(PyExc_SystemError, "pybind11::detail::get_internals_pp_from_capsule() FAILED");
throw error_already_set();
}
return static_cast<internals **>(raw_ptr);
}
/// Return a reference to the current `internals` data
PYBIND11_NOINLINE internals &get_internals() {
auto **&internals_pp = get_internals_pp();
if (internals_pp && *internals_pp) {
return **internals_pp;
}
#if defined(WITH_THREAD)
# if defined(PYBIND11_SIMPLE_GIL_MANAGEMENT)
gil_scoped_acquire gil;
# else
// Ensure that the GIL is held since we will need to make Python calls. // Ensure that the GIL is held since we will need to make Python calls.
// Cannot use py::gil_scoped_acquire here since that constructor calls get_internals. // Cannot use py::gil_scoped_acquire here since that constructor calls get_internals.
struct gil_scoped_acquire_local { struct gil_scoped_acquire_local {
gil_scoped_acquire_local() : state (PyGILState_Ensure()) {} gil_scoped_acquire_local() : state(PyGILState_Ensure()) {}
gil_scoped_acquire_local(const gil_scoped_acquire_local &) = delete;
gil_scoped_acquire_local &operator=(const gil_scoped_acquire_local &) = delete;
~gil_scoped_acquire_local() { PyGILState_Release(state); } ~gil_scoped_acquire_local() { PyGILState_Release(state); }
const PyGILState_STATE state; const PyGILState_STATE state;
} gil; } gil;
# endif
#endif
error_scope err_scope;
constexpr auto *id = PYBIND11_INTERNALS_ID; dict state_dict = get_python_state_dict();
auto builtins = handle(PyEval_GetBuiltins()); if (object internals_obj = get_internals_obj_from_state_dict(state_dict)) {
if (builtins.contains(id) && isinstance<capsule>(builtins[id])) { internals_pp = get_internals_pp_from_capsule(internals_obj);
internals_pp = static_cast<internals **>(capsule(builtins[id])); }
if (internals_pp && *internals_pp) {
// We loaded builtins through python's builtins, which means that our `error_already_set` // We loaded the internals through `state_dict`, which means that our `error_already_set`
// and `builtin_exception` may be different local classes than the ones set up in the // and `builtin_exception` may be different local classes than the ones set up in the
// initial exception translator, below, so add another for our local exception classes. // initial exception translator, below, so add another for our local exception classes.
// //
// libstdc++ doesn't require this (types there are identified only by name) // libstdc++ doesn't require this (types there are identified only by name)
// libc++ with CPython doesn't require this (types are explicitly exported)
// libc++ with PyPy still need it, awaiting further investigation
#if !defined(__GLIBCXX__) #if !defined(__GLIBCXX__)
(*internals_pp)->registered_exception_translators.push_front(&translate_local_exception); (*internals_pp)->registered_exception_translators.push_front(&translate_local_exception);
#endif #endif
} else { } else {
if (!internals_pp) internals_pp = new internals*(); if (!internals_pp) {
internals_pp = new internals *();
}
auto *&internals_ptr = *internals_pp; auto *&internals_ptr = *internals_pp;
internals_ptr = new internals(); internals_ptr = new internals();
#if defined(WITH_THREAD) #if defined(WITH_THREAD)
#if PY_VERSION_HEX < 0x03090000
PyEval_InitThreads();
#endif
PyThreadState *tstate = PyThreadState_Get(); PyThreadState *tstate = PyThreadState_Get();
#if PY_VERSION_HEX >= 0x03070000 // NOLINTNEXTLINE(bugprone-assignment-in-if-condition)
internals_ptr->tstate = PyThread_tss_alloc(); if (!PYBIND11_TLS_KEY_CREATE(internals_ptr->tstate)) {
if (!internals_ptr->tstate || PyThread_tss_create(internals_ptr->tstate)) pybind11_fail("get_internals: could not successfully initialize the tstate TSS key!");
pybind11_fail("get_internals: could not successfully initialize the TSS key!"); }
PyThread_tss_set(internals_ptr->tstate, tstate); PYBIND11_TLS_REPLACE_VALUE(internals_ptr->tstate, tstate);
#else
internals_ptr->tstate = PyThread_create_key(); # if PYBIND11_INTERNALS_VERSION > 4
if (internals_ptr->tstate == -1) // NOLINTNEXTLINE(bugprone-assignment-in-if-condition)
pybind11_fail("get_internals: could not successfully initialize the TLS key!"); if (!PYBIND11_TLS_KEY_CREATE(internals_ptr->loader_life_support_tls_key)) {
PyThread_set_key_value(internals_ptr->tstate, tstate); pybind11_fail("get_internals: could not successfully initialize the "
#endif "loader_life_support TSS key!");
}
# endif
internals_ptr->istate = tstate->interp; internals_ptr->istate = tstate->interp;
#endif #endif
builtins[id] = capsule(internals_pp); state_dict[PYBIND11_INTERNALS_ID] = capsule(internals_pp);
internals_ptr->registered_exception_translators.push_front(&translate_exception); internals_ptr->registered_exception_translators.push_front(&translate_exception);
internals_ptr->static_property_type = make_static_property_type(); internals_ptr->static_property_type = make_static_property_type();
internals_ptr->default_metaclass = make_default_metaclass(); internals_ptr->default_metaclass = make_default_metaclass();
@ -311,10 +548,60 @@ PYBIND11_NOINLINE inline internals &get_internals() {
return **internals_pp; return **internals_pp;
} }
/// Works like `internals.registered_types_cpp`, but for module-local registered types: // the internals struct (above) is shared between all the modules. local_internals are only
inline type_map<type_info *> &registered_local_types_cpp() { // for a single module. Any changes made to internals may require an update to
static type_map<type_info *> locals{}; // PYBIND11_INTERNALS_VERSION, breaking backwards compatibility. local_internals is, by design,
return locals; // restricted to a single module. Whether a module has local internals or not should not
// impact any other modules, because the only things accessing the local internals is the
// module that contains them.
struct local_internals {
type_map<type_info *> registered_types_cpp;
std::forward_list<ExceptionTranslator> registered_exception_translators;
#if defined(WITH_THREAD) && PYBIND11_INTERNALS_VERSION == 4
// For ABI compatibility, we can't store the loader_life_support TLS key in
// the `internals` struct directly. Instead, we store it in `shared_data` and
// cache a copy in `local_internals`. If we allocated a separate TLS key for
// each instance of `local_internals`, we could end up allocating hundreds of
// TLS keys if hundreds of different pybind11 modules are loaded (which is a
// plausible number).
PYBIND11_TLS_KEY_INIT(loader_life_support_tls_key)
// Holds the shared TLS key for the loader_life_support stack.
struct shared_loader_life_support_data {
PYBIND11_TLS_KEY_INIT(loader_life_support_tls_key)
shared_loader_life_support_data() {
// NOLINTNEXTLINE(bugprone-assignment-in-if-condition)
if (!PYBIND11_TLS_KEY_CREATE(loader_life_support_tls_key)) {
pybind11_fail("local_internals: could not successfully initialize the "
"loader_life_support TLS key!");
}
}
// We can't help but leak the TLS key, because Python never unloads extension modules.
};
local_internals() {
auto &internals = get_internals();
// Get or create the `loader_life_support_stack_key`.
auto &ptr = internals.shared_data["_life_support"];
if (!ptr) {
ptr = new shared_loader_life_support_data;
}
loader_life_support_tls_key
= static_cast<shared_loader_life_support_data *>(ptr)->loader_life_support_tls_key;
}
#endif // defined(WITH_THREAD) && PYBIND11_INTERNALS_VERSION == 4
};
/// Works like `get_internals`, but for things which are locally registered.
inline local_internals &get_local_internals() {
// Current static can be created in the interpreter finalization routine. If the later will be
// destroyed in another static variable destructor, creation of this static there will cause
// static deinitialization fiasco. In order to avoid it we avoid destruction of the
// local_internals static. One can read more about the problem and current solution here:
// https://google.github.io/styleguide/cppguide.html#Static_and_Global_Variables
static auto *locals = new local_internals();
return *locals;
} }
/// Constructs a std::string with the given arguments, stores it in `internals`, and returns its /// Constructs a std::string with the given arguments, stores it in `internals`, and returns its
@ -328,19 +615,38 @@ const char *c_str(Args &&...args) {
return strings.front().c_str(); return strings.front().c_str();
} }
inline const char *get_function_record_capsule_name() {
#if PYBIND11_INTERNALS_VERSION > 4
return get_internals().function_record_capsule_name.c_str();
#else
return nullptr;
#endif
}
// Determine whether or not the following capsule contains a pybind11 function record.
// Note that we use `internals` to make sure that only ABI compatible records are touched.
//
// This check is currently used in two places:
// - An important optimization in functional.h to avoid overhead in C++ -> Python -> C++
// - The sibling feature of cpp_function to allow overloads
inline bool is_function_record_capsule(const capsule &cap) {
// Pointer equality as we rely on internals() to ensure unique pointers
return cap.name() == get_function_record_capsule_name();
}
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
/// Returns a named pointer that is shared among all extension modules (using the same /// Returns a named pointer that is shared among all extension modules (using the same
/// pybind11 version) running in the current interpreter. Names starting with underscores /// pybind11 version) running in the current interpreter. Names starting with underscores
/// are reserved for internal usage. Returns `nullptr` if no matching entry was found. /// are reserved for internal usage. Returns `nullptr` if no matching entry was found.
inline PYBIND11_NOINLINE void *get_shared_data(const std::string &name) { PYBIND11_NOINLINE void *get_shared_data(const std::string &name) {
auto &internals = detail::get_internals(); auto &internals = detail::get_internals();
auto it = internals.shared_data.find(name); auto it = internals.shared_data.find(name);
return it != internals.shared_data.end() ? it->second : nullptr; return it != internals.shared_data.end() ? it->second : nullptr;
} }
/// Set the shared data that can be later recovered by `get_shared_data()`. /// Set the shared data that can be later recovered by `get_shared_data()`.
inline PYBIND11_NOINLINE void *set_shared_data(const std::string &name, void *data) { PYBIND11_NOINLINE void *set_shared_data(const std::string &name, void *data) {
detail::get_internals().shared_data[name] = data; detail::get_internals().shared_data[name] = data;
return data; return data;
} }
@ -348,7 +654,7 @@ inline PYBIND11_NOINLINE void *set_shared_data(const std::string &name, void *da
/// Returns a typed reference to a shared data entry (by using `get_shared_data()`) if /// Returns a typed reference to a shared data entry (by using `get_shared_data()`) if
/// such entry exists. Otherwise, a new object of default-constructible type `T` is /// such entry exists. Otherwise, a new object of default-constructible type `T` is
/// added to the shared data under the given name and a reference to it is returned. /// added to the shared data under the given name and a reference to it is returned.
template<typename T> template <typename T>
T &get_or_create_shared_data(const std::string &name) { T &get_or_create_shared_data(const std::string &name) {
auto &internals = detail::get_internals(); auto &internals = detail::get_internals();
auto it = internals.shared_data.find(name); auto it = internals.shared_data.find(name);

File diff suppressed because it is too large Load Diff

View File

@ -13,29 +13,33 @@
#include <cstdlib> #include <cstdlib>
#if defined(__GNUG__) #if defined(__GNUG__)
#include <cxxabi.h> # include <cxxabi.h>
#endif #endif
#include "common.h" #include "common.h"
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
/// Erase all occurrences of a substring /// Erase all occurrences of a substring
inline void erase_all(std::string &string, const std::string &search) { inline void erase_all(std::string &string, const std::string &search) {
for (size_t pos = 0;;) { for (size_t pos = 0;;) {
pos = string.find(search, pos); pos = string.find(search, pos);
if (pos == std::string::npos) break; if (pos == std::string::npos) {
break;
}
string.erase(pos, search.length()); string.erase(pos, search.length());
} }
} }
PYBIND11_NOINLINE inline void clean_type_id(std::string &name) { PYBIND11_NOINLINE void clean_type_id(std::string &name) {
#if defined(__GNUG__) #if defined(__GNUG__)
int status = 0; int status = 0;
std::unique_ptr<char, void (*)(void *)> res { std::unique_ptr<char, void (*)(void *)> res{
abi::__cxa_demangle(name.c_str(), nullptr, nullptr, &status), std::free }; abi::__cxa_demangle(name.c_str(), nullptr, nullptr, &status), std::free};
if (status == 0) if (status == 0) {
name = res.get(); name = res.get();
}
#else #else
detail::erase_all(name, "class "); detail::erase_all(name, "class ");
detail::erase_all(name, "struct "); detail::erase_all(name, "struct ");
@ -43,13 +47,19 @@ PYBIND11_NOINLINE inline void clean_type_id(std::string &name) {
#endif #endif
detail::erase_all(name, "pybind11::"); detail::erase_all(name, "pybind11::");
} }
PYBIND11_NAMESPACE_END(detail)
/// Return a string representation of a C++ type inline std::string clean_type_id(const char *typeid_name) {
template <typename T> static std::string type_id() { std::string name(typeid_name);
std::string name(typeid(T).name());
detail::clean_type_id(name); detail::clean_type_id(name);
return name; return name;
} }
PYBIND11_NAMESPACE_END(detail)
/// Return a string representation of a C++ type
template <typename T>
static std::string type_id() {
return detail::clean_type_id(typeid(T).name());
}
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -9,599 +9,4 @@
#pragma once #pragma once
#include "numpy.h" #include "eigen/matrix.h"
#if defined(__INTEL_COMPILER)
# pragma warning(disable: 1682) // implicit conversion of a 64-bit integral type to a smaller integral type (potential portability problem)
#elif defined(__GNUG__) || defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wconversion"
# pragma GCC diagnostic ignored "-Wdeprecated-declarations"
# ifdef __clang__
// Eigen generates a bunch of implicit-copy-constructor-is-deprecated warnings with -Wdeprecated
// under Clang, so disable that warning here:
# pragma GCC diagnostic ignored "-Wdeprecated"
# endif
# if __GNUC__ >= 7
# pragma GCC diagnostic ignored "-Wint-in-bool-context"
# endif
#endif
#if defined(_MSC_VER)
# pragma warning(push)
# pragma warning(disable: 4127) // warning C4127: Conditional expression is constant
# pragma warning(disable: 4996) // warning C4996: std::unary_negate is deprecated in C++17
#endif
#include <Eigen/Core>
#include <Eigen/SparseCore>
// Eigen prior to 3.2.7 doesn't have proper move constructors--but worse, some classes get implicit
// move constructors that break things. We could detect this an explicitly copy, but an extra copy
// of matrices seems highly undesirable.
static_assert(EIGEN_VERSION_AT_LEAST(3,2,7), "Eigen support in pybind11 requires Eigen >= 3.2.7");
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
// Provide a convenience alias for easier pass-by-ref usage with fully dynamic strides:
using EigenDStride = Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic>;
template <typename MatrixType> using EigenDRef = Eigen::Ref<MatrixType, 0, EigenDStride>;
template <typename MatrixType> using EigenDMap = Eigen::Map<MatrixType, 0, EigenDStride>;
PYBIND11_NAMESPACE_BEGIN(detail)
#if EIGEN_VERSION_AT_LEAST(3,3,0)
using EigenIndex = Eigen::Index;
#else
using EigenIndex = EIGEN_DEFAULT_DENSE_INDEX_TYPE;
#endif
// Matches Eigen::Map, Eigen::Ref, blocks, etc:
template <typename T> using is_eigen_dense_map = all_of<is_template_base_of<Eigen::DenseBase, T>, std::is_base_of<Eigen::MapBase<T, Eigen::ReadOnlyAccessors>, T>>;
template <typename T> using is_eigen_mutable_map = std::is_base_of<Eigen::MapBase<T, Eigen::WriteAccessors>, T>;
template <typename T> using is_eigen_dense_plain = all_of<negation<is_eigen_dense_map<T>>, is_template_base_of<Eigen::PlainObjectBase, T>>;
template <typename T> using is_eigen_sparse = is_template_base_of<Eigen::SparseMatrixBase, T>;
// Test for objects inheriting from EigenBase<Derived> that aren't captured by the above. This
// basically covers anything that can be assigned to a dense matrix but that don't have a typical
// matrix data layout that can be copied from their .data(). For example, DiagonalMatrix and
// SelfAdjointView fall into this category.
template <typename T> using is_eigen_other = all_of<
is_template_base_of<Eigen::EigenBase, T>,
negation<any_of<is_eigen_dense_map<T>, is_eigen_dense_plain<T>, is_eigen_sparse<T>>>
>;
// Captures numpy/eigen conformability status (returned by EigenProps::conformable()):
template <bool EigenRowMajor> struct EigenConformable {
bool conformable = false;
EigenIndex rows = 0, cols = 0;
EigenDStride stride{0, 0}; // Only valid if negativestrides is false!
bool negativestrides = false; // If true, do not use stride!
EigenConformable(bool fits = false) : conformable{fits} {}
// Matrix type:
EigenConformable(EigenIndex r, EigenIndex c,
EigenIndex rstride, EigenIndex cstride) :
conformable{true}, rows{r}, cols{c} {
// TODO: when Eigen bug #747 is fixed, remove the tests for non-negativity. http://eigen.tuxfamily.org/bz/show_bug.cgi?id=747
if (rstride < 0 || cstride < 0) {
negativestrides = true;
} else {
stride = {EigenRowMajor ? rstride : cstride /* outer stride */,
EigenRowMajor ? cstride : rstride /* inner stride */ };
}
}
// Vector type:
EigenConformable(EigenIndex r, EigenIndex c, EigenIndex stride)
: EigenConformable(r, c, r == 1 ? c*stride : stride, c == 1 ? r : r*stride) {}
template <typename props> bool stride_compatible() const {
// To have compatible strides, we need (on both dimensions) one of fully dynamic strides,
// matching strides, or a dimension size of 1 (in which case the stride value is irrelevant)
return
!negativestrides &&
(props::inner_stride == Eigen::Dynamic || props::inner_stride == stride.inner() ||
(EigenRowMajor ? cols : rows) == 1) &&
(props::outer_stride == Eigen::Dynamic || props::outer_stride == stride.outer() ||
(EigenRowMajor ? rows : cols) == 1);
}
operator bool() const { return conformable; }
};
template <typename Type> struct eigen_extract_stride { using type = Type; };
template <typename PlainObjectType, int MapOptions, typename StrideType>
struct eigen_extract_stride<Eigen::Map<PlainObjectType, MapOptions, StrideType>> { using type = StrideType; };
template <typename PlainObjectType, int Options, typename StrideType>
struct eigen_extract_stride<Eigen::Ref<PlainObjectType, Options, StrideType>> { using type = StrideType; };
// Helper struct for extracting information from an Eigen type
template <typename Type_> struct EigenProps {
using Type = Type_;
using Scalar = typename Type::Scalar;
using StrideType = typename eigen_extract_stride<Type>::type;
static constexpr EigenIndex
rows = Type::RowsAtCompileTime,
cols = Type::ColsAtCompileTime,
size = Type::SizeAtCompileTime;
static constexpr bool
row_major = Type::IsRowMajor,
vector = Type::IsVectorAtCompileTime, // At least one dimension has fixed size 1
fixed_rows = rows != Eigen::Dynamic,
fixed_cols = cols != Eigen::Dynamic,
fixed = size != Eigen::Dynamic, // Fully-fixed size
dynamic = !fixed_rows && !fixed_cols; // Fully-dynamic size
template <EigenIndex i, EigenIndex ifzero> using if_zero = std::integral_constant<EigenIndex, i == 0 ? ifzero : i>;
static constexpr EigenIndex inner_stride = if_zero<StrideType::InnerStrideAtCompileTime, 1>::value,
outer_stride = if_zero<StrideType::OuterStrideAtCompileTime,
vector ? size : row_major ? cols : rows>::value;
static constexpr bool dynamic_stride = inner_stride == Eigen::Dynamic && outer_stride == Eigen::Dynamic;
static constexpr bool requires_row_major = !dynamic_stride && !vector && (row_major ? inner_stride : outer_stride) == 1;
static constexpr bool requires_col_major = !dynamic_stride && !vector && (row_major ? outer_stride : inner_stride) == 1;
// Takes an input array and determines whether we can make it fit into the Eigen type. If
// the array is a vector, we attempt to fit it into either an Eigen 1xN or Nx1 vector
// (preferring the latter if it will fit in either, i.e. for a fully dynamic matrix type).
static EigenConformable<row_major> conformable(const array &a) {
const auto dims = a.ndim();
if (dims < 1 || dims > 2)
return false;
if (dims == 2) { // Matrix type: require exact match (or dynamic)
EigenIndex
np_rows = a.shape(0),
np_cols = a.shape(1),
np_rstride = a.strides(0) / static_cast<ssize_t>(sizeof(Scalar)),
np_cstride = a.strides(1) / static_cast<ssize_t>(sizeof(Scalar));
if ((fixed_rows && np_rows != rows) || (fixed_cols && np_cols != cols))
return false;
return {np_rows, np_cols, np_rstride, np_cstride};
}
// Otherwise we're storing an n-vector. Only one of the strides will be used, but whichever
// is used, we want the (single) numpy stride value.
const EigenIndex n = a.shape(0),
stride = a.strides(0) / static_cast<ssize_t>(sizeof(Scalar));
if (vector) { // Eigen type is a compile-time vector
if (fixed && size != n)
return false; // Vector size mismatch
return {rows == 1 ? 1 : n, cols == 1 ? 1 : n, stride};
}
else if (fixed) {
// The type has a fixed size, but is not a vector: abort
return false;
}
else if (fixed_cols) {
// Since this isn't a vector, cols must be != 1. We allow this only if it exactly
// equals the number of elements (rows is Dynamic, and so 1 row is allowed).
if (cols != n) return false;
return {1, n, stride};
}
else {
// Otherwise it's either fully dynamic, or column dynamic; both become a column vector
if (fixed_rows && rows != n) return false;
return {n, 1, stride};
}
}
static constexpr bool show_writeable = is_eigen_dense_map<Type>::value && is_eigen_mutable_map<Type>::value;
static constexpr bool show_order = is_eigen_dense_map<Type>::value;
static constexpr bool show_c_contiguous = show_order && requires_row_major;
static constexpr bool show_f_contiguous = !show_c_contiguous && show_order && requires_col_major;
static constexpr auto descriptor =
_("numpy.ndarray[") + npy_format_descriptor<Scalar>::name +
_("[") + _<fixed_rows>(_<(size_t) rows>(), _("m")) +
_(", ") + _<fixed_cols>(_<(size_t) cols>(), _("n")) +
_("]") +
// For a reference type (e.g. Ref<MatrixXd>) we have other constraints that might need to be
// satisfied: writeable=True (for a mutable reference), and, depending on the map's stride
// options, possibly f_contiguous or c_contiguous. We include them in the descriptor output
// to provide some hint as to why a TypeError is occurring (otherwise it can be confusing to
// see that a function accepts a 'numpy.ndarray[float64[3,2]]' and an error message that you
// *gave* a numpy.ndarray of the right type and dimensions.
_<show_writeable>(", flags.writeable", "") +
_<show_c_contiguous>(", flags.c_contiguous", "") +
_<show_f_contiguous>(", flags.f_contiguous", "") +
_("]");
};
// Casts an Eigen type to numpy array. If given a base, the numpy array references the src data,
// otherwise it'll make a copy. writeable lets you turn off the writeable flag for the array.
template <typename props> handle eigen_array_cast(typename props::Type const &src, handle base = handle(), bool writeable = true) {
constexpr ssize_t elem_size = sizeof(typename props::Scalar);
array a;
if (props::vector)
a = array({ src.size() }, { elem_size * src.innerStride() }, src.data(), base);
else
a = array({ src.rows(), src.cols() }, { elem_size * src.rowStride(), elem_size * src.colStride() },
src.data(), base);
if (!writeable)
array_proxy(a.ptr())->flags &= ~detail::npy_api::NPY_ARRAY_WRITEABLE_;
return a.release();
}
// Takes an lvalue ref to some Eigen type and a (python) base object, creating a numpy array that
// reference the Eigen object's data with `base` as the python-registered base class (if omitted,
// the base will be set to None, and lifetime management is up to the caller). The numpy array is
// non-writeable if the given type is const.
template <typename props, typename Type>
handle eigen_ref_array(Type &src, handle parent = none()) {
// none here is to get past array's should-we-copy detection, which currently always
// copies when there is no base. Setting the base to None should be harmless.
return eigen_array_cast<props>(src, parent, !std::is_const<Type>::value);
}
// Takes a pointer to some dense, plain Eigen type, builds a capsule around it, then returns a numpy
// array that references the encapsulated data with a python-side reference to the capsule to tie
// its destruction to that of any dependent python objects. Const-ness is determined by whether or
// not the Type of the pointer given is const.
template <typename props, typename Type, typename = enable_if_t<is_eigen_dense_plain<Type>::value>>
handle eigen_encapsulate(Type *src) {
capsule base(src, [](void *o) { delete static_cast<Type *>(o); });
return eigen_ref_array<props>(*src, base);
}
// Type caster for regular, dense matrix types (e.g. MatrixXd), but not maps/refs/etc. of dense
// types.
template<typename Type>
struct type_caster<Type, enable_if_t<is_eigen_dense_plain<Type>::value>> {
using Scalar = typename Type::Scalar;
using props = EigenProps<Type>;
bool load(handle src, bool convert) {
// If we're in no-convert mode, only load if given an array of the correct type
if (!convert && !isinstance<array_t<Scalar>>(src))
return false;
// Coerce into an array, but don't do type conversion yet; the copy below handles it.
auto buf = array::ensure(src);
if (!buf)
return false;
auto dims = buf.ndim();
if (dims < 1 || dims > 2)
return false;
auto fits = props::conformable(buf);
if (!fits)
return false;
// Allocate the new type, then build a numpy reference into it
value = Type(fits.rows, fits.cols);
auto ref = reinterpret_steal<array>(eigen_ref_array<props>(value));
if (dims == 1) ref = ref.squeeze();
else if (ref.ndim() == 1) buf = buf.squeeze();
int result = detail::npy_api::get().PyArray_CopyInto_(ref.ptr(), buf.ptr());
if (result < 0) { // Copy failed!
PyErr_Clear();
return false;
}
return true;
}
private:
// Cast implementation
template <typename CType>
static handle cast_impl(CType *src, return_value_policy policy, handle parent) {
switch (policy) {
case return_value_policy::take_ownership:
case return_value_policy::automatic:
return eigen_encapsulate<props>(src);
case return_value_policy::move:
return eigen_encapsulate<props>(new CType(std::move(*src)));
case return_value_policy::copy:
return eigen_array_cast<props>(*src);
case return_value_policy::reference:
case return_value_policy::automatic_reference:
return eigen_ref_array<props>(*src);
case return_value_policy::reference_internal:
return eigen_ref_array<props>(*src, parent);
default:
throw cast_error("unhandled return_value_policy: should not happen!");
};
}
public:
// Normal returned non-reference, non-const value:
static handle cast(Type &&src, return_value_policy /* policy */, handle parent) {
return cast_impl(&src, return_value_policy::move, parent);
}
// If you return a non-reference const, we mark the numpy array readonly:
static handle cast(const Type &&src, return_value_policy /* policy */, handle parent) {
return cast_impl(&src, return_value_policy::move, parent);
}
// lvalue reference return; default (automatic) becomes copy
static handle cast(Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic || policy == return_value_policy::automatic_reference)
policy = return_value_policy::copy;
return cast_impl(&src, policy, parent);
}
// const lvalue reference return; default (automatic) becomes copy
static handle cast(const Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic || policy == return_value_policy::automatic_reference)
policy = return_value_policy::copy;
return cast(&src, policy, parent);
}
// non-const pointer return
static handle cast(Type *src, return_value_policy policy, handle parent) {
return cast_impl(src, policy, parent);
}
// const pointer return
static handle cast(const Type *src, return_value_policy policy, handle parent) {
return cast_impl(src, policy, parent);
}
static constexpr auto name = props::descriptor;
operator Type*() { return &value; }
operator Type&() { return value; }
operator Type&&() && { return std::move(value); }
template <typename T> using cast_op_type = movable_cast_op_type<T>;
private:
Type value;
};
// Base class for casting reference/map/block/etc. objects back to python.
template <typename MapType> struct eigen_map_caster {
private:
using props = EigenProps<MapType>;
public:
// Directly referencing a ref/map's data is a bit dangerous (whatever the map/ref points to has
// to stay around), but we'll allow it under the assumption that you know what you're doing (and
// have an appropriate keep_alive in place). We return a numpy array pointing directly at the
// ref's data (The numpy array ends up read-only if the ref was to a const matrix type.) Note
// that this means you need to ensure you don't destroy the object in some other way (e.g. with
// an appropriate keep_alive, or with a reference to a statically allocated matrix).
static handle cast(const MapType &src, return_value_policy policy, handle parent) {
switch (policy) {
case return_value_policy::copy:
return eigen_array_cast<props>(src);
case return_value_policy::reference_internal:
return eigen_array_cast<props>(src, parent, is_eigen_mutable_map<MapType>::value);
case return_value_policy::reference:
case return_value_policy::automatic:
case return_value_policy::automatic_reference:
return eigen_array_cast<props>(src, none(), is_eigen_mutable_map<MapType>::value);
default:
// move, take_ownership don't make any sense for a ref/map:
pybind11_fail("Invalid return_value_policy for Eigen Map/Ref/Block type");
}
}
static constexpr auto name = props::descriptor;
// Explicitly delete these: support python -> C++ conversion on these (i.e. these can be return
// types but not bound arguments). We still provide them (with an explicitly delete) so that
// you end up here if you try anyway.
bool load(handle, bool) = delete;
operator MapType() = delete;
template <typename> using cast_op_type = MapType;
};
// We can return any map-like object (but can only load Refs, specialized next):
template <typename Type> struct type_caster<Type, enable_if_t<is_eigen_dense_map<Type>::value>>
: eigen_map_caster<Type> {};
// Loader for Ref<...> arguments. See the documentation for info on how to make this work without
// copying (it requires some extra effort in many cases).
template <typename PlainObjectType, typename StrideType>
struct type_caster<
Eigen::Ref<PlainObjectType, 0, StrideType>,
enable_if_t<is_eigen_dense_map<Eigen::Ref<PlainObjectType, 0, StrideType>>::value>
> : public eigen_map_caster<Eigen::Ref<PlainObjectType, 0, StrideType>> {
private:
using Type = Eigen::Ref<PlainObjectType, 0, StrideType>;
using props = EigenProps<Type>;
using Scalar = typename props::Scalar;
using MapType = Eigen::Map<PlainObjectType, 0, StrideType>;
using Array = array_t<Scalar, array::forcecast |
((props::row_major ? props::inner_stride : props::outer_stride) == 1 ? array::c_style :
(props::row_major ? props::outer_stride : props::inner_stride) == 1 ? array::f_style : 0)>;
static constexpr bool need_writeable = is_eigen_mutable_map<Type>::value;
// Delay construction (these have no default constructor)
std::unique_ptr<MapType> map;
std::unique_ptr<Type> ref;
// Our array. When possible, this is just a numpy array pointing to the source data, but
// sometimes we can't avoid copying (e.g. input is not a numpy array at all, has an incompatible
// layout, or is an array of a type that needs to be converted). Using a numpy temporary
// (rather than an Eigen temporary) saves an extra copy when we need both type conversion and
// storage order conversion. (Note that we refuse to use this temporary copy when loading an
// argument for a Ref<M> with M non-const, i.e. a read-write reference).
Array copy_or_ref;
public:
bool load(handle src, bool convert) {
// First check whether what we have is already an array of the right type. If not, we can't
// avoid a copy (because the copy is also going to do type conversion).
bool need_copy = !isinstance<Array>(src);
EigenConformable<props::row_major> fits;
if (!need_copy) {
// We don't need a converting copy, but we also need to check whether the strides are
// compatible with the Ref's stride requirements
auto aref = reinterpret_borrow<Array>(src);
if (aref && (!need_writeable || aref.writeable())) {
fits = props::conformable(aref);
if (!fits) return false; // Incompatible dimensions
if (!fits.template stride_compatible<props>())
need_copy = true;
else
copy_or_ref = std::move(aref);
}
else {
need_copy = true;
}
}
if (need_copy) {
// We need to copy: If we need a mutable reference, or we're not supposed to convert
// (either because we're in the no-convert overload pass, or because we're explicitly
// instructed not to copy (via `py::arg().noconvert()`) we have to fail loading.
if (!convert || need_writeable) return false;
Array copy = Array::ensure(src);
if (!copy) return false;
fits = props::conformable(copy);
if (!fits || !fits.template stride_compatible<props>())
return false;
copy_or_ref = std::move(copy);
loader_life_support::add_patient(copy_or_ref);
}
ref.reset();
map.reset(new MapType(data(copy_or_ref), fits.rows, fits.cols, make_stride(fits.stride.outer(), fits.stride.inner())));
ref.reset(new Type(*map));
return true;
}
operator Type*() { return ref.get(); }
operator Type&() { return *ref; }
template <typename _T> using cast_op_type = pybind11::detail::cast_op_type<_T>;
private:
template <typename T = Type, enable_if_t<is_eigen_mutable_map<T>::value, int> = 0>
Scalar *data(Array &a) { return a.mutable_data(); }
template <typename T = Type, enable_if_t<!is_eigen_mutable_map<T>::value, int> = 0>
const Scalar *data(Array &a) { return a.data(); }
// Attempt to figure out a constructor of `Stride` that will work.
// If both strides are fixed, use a default constructor:
template <typename S> using stride_ctor_default = bool_constant<
S::InnerStrideAtCompileTime != Eigen::Dynamic && S::OuterStrideAtCompileTime != Eigen::Dynamic &&
std::is_default_constructible<S>::value>;
// Otherwise, if there is a two-index constructor, assume it is (outer,inner) like
// Eigen::Stride, and use it:
template <typename S> using stride_ctor_dual = bool_constant<
!stride_ctor_default<S>::value && std::is_constructible<S, EigenIndex, EigenIndex>::value>;
// Otherwise, if there is a one-index constructor, and just one of the strides is dynamic, use
// it (passing whichever stride is dynamic).
template <typename S> using stride_ctor_outer = bool_constant<
!any_of<stride_ctor_default<S>, stride_ctor_dual<S>>::value &&
S::OuterStrideAtCompileTime == Eigen::Dynamic && S::InnerStrideAtCompileTime != Eigen::Dynamic &&
std::is_constructible<S, EigenIndex>::value>;
template <typename S> using stride_ctor_inner = bool_constant<
!any_of<stride_ctor_default<S>, stride_ctor_dual<S>>::value &&
S::InnerStrideAtCompileTime == Eigen::Dynamic && S::OuterStrideAtCompileTime != Eigen::Dynamic &&
std::is_constructible<S, EigenIndex>::value>;
template <typename S = StrideType, enable_if_t<stride_ctor_default<S>::value, int> = 0>
static S make_stride(EigenIndex, EigenIndex) { return S(); }
template <typename S = StrideType, enable_if_t<stride_ctor_dual<S>::value, int> = 0>
static S make_stride(EigenIndex outer, EigenIndex inner) { return S(outer, inner); }
template <typename S = StrideType, enable_if_t<stride_ctor_outer<S>::value, int> = 0>
static S make_stride(EigenIndex outer, EigenIndex) { return S(outer); }
template <typename S = StrideType, enable_if_t<stride_ctor_inner<S>::value, int> = 0>
static S make_stride(EigenIndex, EigenIndex inner) { return S(inner); }
};
// type_caster for special matrix types (e.g. DiagonalMatrix), which are EigenBase, but not
// EigenDense (i.e. they don't have a data(), at least not with the usual matrix layout).
// load() is not supported, but we can cast them into the python domain by first copying to a
// regular Eigen::Matrix, then casting that.
template <typename Type>
struct type_caster<Type, enable_if_t<is_eigen_other<Type>::value>> {
protected:
using Matrix = Eigen::Matrix<typename Type::Scalar, Type::RowsAtCompileTime, Type::ColsAtCompileTime>;
using props = EigenProps<Matrix>;
public:
static handle cast(const Type &src, return_value_policy /* policy */, handle /* parent */) {
handle h = eigen_encapsulate<props>(new Matrix(src));
return h;
}
static handle cast(const Type *src, return_value_policy policy, handle parent) { return cast(*src, policy, parent); }
static constexpr auto name = props::descriptor;
// Explicitly delete these: support python -> C++ conversion on these (i.e. these can be return
// types but not bound arguments). We still provide them (with an explicitly delete) so that
// you end up here if you try anyway.
bool load(handle, bool) = delete;
operator Type() = delete;
template <typename> using cast_op_type = Type;
};
template<typename Type>
struct type_caster<Type, enable_if_t<is_eigen_sparse<Type>::value>> {
using Scalar = typename Type::Scalar;
using StorageIndex = remove_reference_t<decltype(*std::declval<Type>().outerIndexPtr())>;
using Index = typename Type::Index;
static constexpr bool rowMajor = Type::IsRowMajor;
bool load(handle src, bool) {
if (!src)
return false;
auto obj = reinterpret_borrow<object>(src);
object sparse_module = module_::import("scipy.sparse");
object matrix_type = sparse_module.attr(
rowMajor ? "csr_matrix" : "csc_matrix");
if (!type::handle_of(obj).is(matrix_type)) {
try {
obj = matrix_type(obj);
} catch (const error_already_set &) {
return false;
}
}
auto values = array_t<Scalar>((object) obj.attr("data"));
auto innerIndices = array_t<StorageIndex>((object) obj.attr("indices"));
auto outerIndices = array_t<StorageIndex>((object) obj.attr("indptr"));
auto shape = pybind11::tuple((pybind11::object) obj.attr("shape"));
auto nnz = obj.attr("nnz").cast<Index>();
if (!values || !innerIndices || !outerIndices)
return false;
value = Eigen::MappedSparseMatrix<Scalar, Type::Flags, StorageIndex>(
shape[0].cast<Index>(), shape[1].cast<Index>(), nnz,
outerIndices.mutable_data(), innerIndices.mutable_data(), values.mutable_data());
return true;
}
static handle cast(const Type &src, return_value_policy /* policy */, handle /* parent */) {
const_cast<Type&>(src).makeCompressed();
object matrix_type = module_::import("scipy.sparse").attr(
rowMajor ? "csr_matrix" : "csc_matrix");
array data(src.nonZeros(), src.valuePtr());
array outerIndices((rowMajor ? src.rows() : src.cols()) + 1, src.outerIndexPtr());
array innerIndices(src.nonZeros(), src.innerIndexPtr());
return matrix_type(
std::make_tuple(data, innerIndices, outerIndices),
std::make_pair(src.rows(), src.cols())
).release();
}
PYBIND11_TYPE_CASTER(Type, _<(Type::IsRowMajor) != 0>("scipy.sparse.csr_matrix[", "scipy.sparse.csc_matrix[")
+ npy_format_descriptor<Scalar>::name + _("]"));
};
PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)
#if defined(__GNUG__) || defined(__clang__)
# pragma GCC diagnostic pop
#elif defined(_MSC_VER)
# pragma warning(pop)
#endif

View File

@ -0,0 +1,9 @@
// Copyright (c) 2023 The pybind Community.
#pragma once
// Common message for `static_assert()`s, which are useful to easily
// preempt much less obvious errors.
#define PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED \
"Pointer types (in particular `PyObject *`) are not supported as scalar types for Eigen " \
"types."

View File

@ -0,0 +1,714 @@
/*
pybind11/eigen/matrix.h: Transparent conversion for dense and sparse Eigen matrices
Copyright (c) 2016 Wenzel Jakob <wenzel.jakob@epfl.ch>
All rights reserved. Use of this source code is governed by a
BSD-style license that can be found in the LICENSE file.
*/
#pragma once
#include "../numpy.h"
#include "common.h"
/* HINT: To suppress warnings originating from the Eigen headers, use -isystem.
See also:
https://stackoverflow.com/questions/2579576/i-dir-vs-isystem-dir
https://stackoverflow.com/questions/1741816/isystem-for-ms-visual-studio-c-compiler
*/
PYBIND11_WARNING_PUSH
PYBIND11_WARNING_DISABLE_MSVC(5054) // https://github.com/pybind/pybind11/pull/3741
// C5054: operator '&': deprecated between enumerations of different types
#if defined(__MINGW32__)
PYBIND11_WARNING_DISABLE_GCC("-Wmaybe-uninitialized")
#endif
#include <Eigen/Core>
#include <Eigen/SparseCore>
PYBIND11_WARNING_POP
// Eigen prior to 3.2.7 doesn't have proper move constructors--but worse, some classes get implicit
// move constructors that break things. We could detect this an explicitly copy, but an extra copy
// of matrices seems highly undesirable.
static_assert(EIGEN_VERSION_AT_LEAST(3, 2, 7),
"Eigen matrix support in pybind11 requires Eigen >= 3.2.7");
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_WARNING_DISABLE_MSVC(4127)
// Provide a convenience alias for easier pass-by-ref usage with fully dynamic strides:
using EigenDStride = Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic>;
template <typename MatrixType>
using EigenDRef = Eigen::Ref<MatrixType, 0, EigenDStride>;
template <typename MatrixType>
using EigenDMap = Eigen::Map<MatrixType, 0, EigenDStride>;
PYBIND11_NAMESPACE_BEGIN(detail)
#if EIGEN_VERSION_AT_LEAST(3, 3, 0)
using EigenIndex = Eigen::Index;
template <typename Scalar, int Flags, typename StorageIndex>
using EigenMapSparseMatrix = Eigen::Map<Eigen::SparseMatrix<Scalar, Flags, StorageIndex>>;
#else
using EigenIndex = EIGEN_DEFAULT_DENSE_INDEX_TYPE;
template <typename Scalar, int Flags, typename StorageIndex>
using EigenMapSparseMatrix = Eigen::MappedSparseMatrix<Scalar, Flags, StorageIndex>;
#endif
// Matches Eigen::Map, Eigen::Ref, blocks, etc:
template <typename T>
using is_eigen_dense_map = all_of<is_template_base_of<Eigen::DenseBase, T>,
std::is_base_of<Eigen::MapBase<T, Eigen::ReadOnlyAccessors>, T>>;
template <typename T>
using is_eigen_mutable_map = std::is_base_of<Eigen::MapBase<T, Eigen::WriteAccessors>, T>;
template <typename T>
using is_eigen_dense_plain
= all_of<negation<is_eigen_dense_map<T>>, is_template_base_of<Eigen::PlainObjectBase, T>>;
template <typename T>
using is_eigen_sparse = is_template_base_of<Eigen::SparseMatrixBase, T>;
// Test for objects inheriting from EigenBase<Derived> that aren't captured by the above. This
// basically covers anything that can be assigned to a dense matrix but that don't have a typical
// matrix data layout that can be copied from their .data(). For example, DiagonalMatrix and
// SelfAdjointView fall into this category.
template <typename T>
using is_eigen_other
= all_of<is_template_base_of<Eigen::EigenBase, T>,
negation<any_of<is_eigen_dense_map<T>, is_eigen_dense_plain<T>, is_eigen_sparse<T>>>>;
// Captures numpy/eigen conformability status (returned by EigenProps::conformable()):
template <bool EigenRowMajor>
struct EigenConformable {
bool conformable = false;
EigenIndex rows = 0, cols = 0;
EigenDStride stride{0, 0}; // Only valid if negativestrides is false!
bool negativestrides = false; // If true, do not use stride!
// NOLINTNEXTLINE(google-explicit-constructor)
EigenConformable(bool fits = false) : conformable{fits} {}
// Matrix type:
EigenConformable(EigenIndex r, EigenIndex c, EigenIndex rstride, EigenIndex cstride)
: conformable{true}, rows{r}, cols{c},
// TODO: when Eigen bug #747 is fixed, remove the tests for non-negativity.
// http://eigen.tuxfamily.org/bz/show_bug.cgi?id=747
stride{EigenRowMajor ? (rstride > 0 ? rstride : 0)
: (cstride > 0 ? cstride : 0) /* outer stride */,
EigenRowMajor ? (cstride > 0 ? cstride : 0)
: (rstride > 0 ? rstride : 0) /* inner stride */},
negativestrides{rstride < 0 || cstride < 0} {}
// Vector type:
EigenConformable(EigenIndex r, EigenIndex c, EigenIndex stride)
: EigenConformable(r, c, r == 1 ? c * stride : stride, c == 1 ? r : r * stride) {}
template <typename props>
bool stride_compatible() const {
// To have compatible strides, we need (on both dimensions) one of fully dynamic strides,
// matching strides, or a dimension size of 1 (in which case the stride value is
// irrelevant). Alternatively, if any dimension size is 0, the strides are not relevant
// (and numpy ≥ 1.23 sets the strides to 0 in that case, so we need to check explicitly).
if (negativestrides) {
return false;
}
if (rows == 0 || cols == 0) {
return true;
}
return (props::inner_stride == Eigen::Dynamic || props::inner_stride == stride.inner()
|| (EigenRowMajor ? cols : rows) == 1)
&& (props::outer_stride == Eigen::Dynamic || props::outer_stride == stride.outer()
|| (EigenRowMajor ? rows : cols) == 1);
}
// NOLINTNEXTLINE(google-explicit-constructor)
operator bool() const { return conformable; }
};
template <typename Type>
struct eigen_extract_stride {
using type = Type;
};
template <typename PlainObjectType, int MapOptions, typename StrideType>
struct eigen_extract_stride<Eigen::Map<PlainObjectType, MapOptions, StrideType>> {
using type = StrideType;
};
template <typename PlainObjectType, int Options, typename StrideType>
struct eigen_extract_stride<Eigen::Ref<PlainObjectType, Options, StrideType>> {
using type = StrideType;
};
// Helper struct for extracting information from an Eigen type
template <typename Type_>
struct EigenProps {
using Type = Type_;
using Scalar = typename Type::Scalar;
using StrideType = typename eigen_extract_stride<Type>::type;
static constexpr EigenIndex rows = Type::RowsAtCompileTime, cols = Type::ColsAtCompileTime,
size = Type::SizeAtCompileTime;
static constexpr bool row_major = Type::IsRowMajor,
vector
= Type::IsVectorAtCompileTime, // At least one dimension has fixed size 1
fixed_rows = rows != Eigen::Dynamic, fixed_cols = cols != Eigen::Dynamic,
fixed = size != Eigen::Dynamic, // Fully-fixed size
dynamic = !fixed_rows && !fixed_cols; // Fully-dynamic size
template <EigenIndex i, EigenIndex ifzero>
using if_zero = std::integral_constant<EigenIndex, i == 0 ? ifzero : i>;
static constexpr EigenIndex inner_stride
= if_zero<StrideType::InnerStrideAtCompileTime, 1>::value,
outer_stride = if_zero < StrideType::OuterStrideAtCompileTime,
vector ? size
: row_major ? cols
: rows > ::value;
static constexpr bool dynamic_stride
= inner_stride == Eigen::Dynamic && outer_stride == Eigen::Dynamic;
static constexpr bool requires_row_major
= !dynamic_stride && !vector && (row_major ? inner_stride : outer_stride) == 1;
static constexpr bool requires_col_major
= !dynamic_stride && !vector && (row_major ? outer_stride : inner_stride) == 1;
// Takes an input array and determines whether we can make it fit into the Eigen type. If
// the array is a vector, we attempt to fit it into either an Eigen 1xN or Nx1 vector
// (preferring the latter if it will fit in either, i.e. for a fully dynamic matrix type).
static EigenConformable<row_major> conformable(const array &a) {
const auto dims = a.ndim();
if (dims < 1 || dims > 2) {
return false;
}
if (dims == 2) { // Matrix type: require exact match (or dynamic)
EigenIndex np_rows = a.shape(0), np_cols = a.shape(1),
np_rstride = a.strides(0) / static_cast<ssize_t>(sizeof(Scalar)),
np_cstride = a.strides(1) / static_cast<ssize_t>(sizeof(Scalar));
if ((fixed_rows && np_rows != rows) || (fixed_cols && np_cols != cols)) {
return false;
}
return {np_rows, np_cols, np_rstride, np_cstride};
}
// Otherwise we're storing an n-vector. Only one of the strides will be used, but
// whichever is used, we want the (single) numpy stride value.
const EigenIndex n = a.shape(0),
stride = a.strides(0) / static_cast<ssize_t>(sizeof(Scalar));
if (vector) { // Eigen type is a compile-time vector
if (fixed && size != n) {
return false; // Vector size mismatch
}
return {rows == 1 ? 1 : n, cols == 1 ? 1 : n, stride};
}
if (fixed) {
// The type has a fixed size, but is not a vector: abort
return false;
}
if (fixed_cols) {
// Since this isn't a vector, cols must be != 1. We allow this only if it exactly
// equals the number of elements (rows is Dynamic, and so 1 row is allowed).
if (cols != n) {
return false;
}
return {1, n, stride};
} // Otherwise it's either fully dynamic, or column dynamic; both become a column vector
if (fixed_rows && rows != n) {
return false;
}
return {n, 1, stride};
}
static constexpr bool show_writeable
= is_eigen_dense_map<Type>::value && is_eigen_mutable_map<Type>::value;
static constexpr bool show_order = is_eigen_dense_map<Type>::value;
static constexpr bool show_c_contiguous = show_order && requires_row_major;
static constexpr bool show_f_contiguous
= !show_c_contiguous && show_order && requires_col_major;
static constexpr auto descriptor
= const_name("numpy.ndarray[") + npy_format_descriptor<Scalar>::name + const_name("[")
+ const_name<fixed_rows>(const_name<(size_t) rows>(), const_name("m")) + const_name(", ")
+ const_name<fixed_cols>(const_name<(size_t) cols>(), const_name("n")) + const_name("]")
+
// For a reference type (e.g. Ref<MatrixXd>) we have other constraints that might need to
// be satisfied: writeable=True (for a mutable reference), and, depending on the map's
// stride options, possibly f_contiguous or c_contiguous. We include them in the
// descriptor output to provide some hint as to why a TypeError is occurring (otherwise
// it can be confusing to see that a function accepts a 'numpy.ndarray[float64[3,2]]' and
// an error message that you *gave* a numpy.ndarray of the right type and dimensions.
const_name<show_writeable>(", flags.writeable", "")
+ const_name<show_c_contiguous>(", flags.c_contiguous", "")
+ const_name<show_f_contiguous>(", flags.f_contiguous", "") + const_name("]");
};
// Casts an Eigen type to numpy array. If given a base, the numpy array references the src data,
// otherwise it'll make a copy. writeable lets you turn off the writeable flag for the array.
template <typename props>
handle
eigen_array_cast(typename props::Type const &src, handle base = handle(), bool writeable = true) {
constexpr ssize_t elem_size = sizeof(typename props::Scalar);
array a;
if (props::vector) {
a = array({src.size()}, {elem_size * src.innerStride()}, src.data(), base);
} else {
a = array({src.rows(), src.cols()},
{elem_size * src.rowStride(), elem_size * src.colStride()},
src.data(),
base);
}
if (!writeable) {
array_proxy(a.ptr())->flags &= ~detail::npy_api::NPY_ARRAY_WRITEABLE_;
}
return a.release();
}
// Takes an lvalue ref to some Eigen type and a (python) base object, creating a numpy array that
// reference the Eigen object's data with `base` as the python-registered base class (if omitted,
// the base will be set to None, and lifetime management is up to the caller). The numpy array is
// non-writeable if the given type is const.
template <typename props, typename Type>
handle eigen_ref_array(Type &src, handle parent = none()) {
// none here is to get past array's should-we-copy detection, which currently always
// copies when there is no base. Setting the base to None should be harmless.
return eigen_array_cast<props>(src, parent, !std::is_const<Type>::value);
}
// Takes a pointer to some dense, plain Eigen type, builds a capsule around it, then returns a
// numpy array that references the encapsulated data with a python-side reference to the capsule to
// tie its destruction to that of any dependent python objects. Const-ness is determined by
// whether or not the Type of the pointer given is const.
template <typename props, typename Type, typename = enable_if_t<is_eigen_dense_plain<Type>::value>>
handle eigen_encapsulate(Type *src) {
capsule base(src, [](void *o) { delete static_cast<Type *>(o); });
return eigen_ref_array<props>(*src, base);
}
// Type caster for regular, dense matrix types (e.g. MatrixXd), but not maps/refs/etc. of dense
// types.
template <typename Type>
struct type_caster<Type, enable_if_t<is_eigen_dense_plain<Type>::value>> {
using Scalar = typename Type::Scalar;
static_assert(!std::is_pointer<Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
using props = EigenProps<Type>;
bool load(handle src, bool convert) {
// If we're in no-convert mode, only load if given an array of the correct type
if (!convert && !isinstance<array_t<Scalar>>(src)) {
return false;
}
// Coerce into an array, but don't do type conversion yet; the copy below handles it.
auto buf = array::ensure(src);
if (!buf) {
return false;
}
auto dims = buf.ndim();
if (dims < 1 || dims > 2) {
return false;
}
auto fits = props::conformable(buf);
if (!fits) {
return false;
}
// Allocate the new type, then build a numpy reference into it
value = Type(fits.rows, fits.cols);
auto ref = reinterpret_steal<array>(eigen_ref_array<props>(value));
if (dims == 1) {
ref = ref.squeeze();
} else if (ref.ndim() == 1) {
buf = buf.squeeze();
}
int result = detail::npy_api::get().PyArray_CopyInto_(ref.ptr(), buf.ptr());
if (result < 0) { // Copy failed!
PyErr_Clear();
return false;
}
return true;
}
private:
// Cast implementation
template <typename CType>
static handle cast_impl(CType *src, return_value_policy policy, handle parent) {
switch (policy) {
case return_value_policy::take_ownership:
case return_value_policy::automatic:
return eigen_encapsulate<props>(src);
case return_value_policy::move:
return eigen_encapsulate<props>(new CType(std::move(*src)));
case return_value_policy::copy:
return eigen_array_cast<props>(*src);
case return_value_policy::reference:
case return_value_policy::automatic_reference:
return eigen_ref_array<props>(*src);
case return_value_policy::reference_internal:
return eigen_ref_array<props>(*src, parent);
default:
throw cast_error("unhandled return_value_policy: should not happen!");
};
}
public:
// Normal returned non-reference, non-const value:
static handle cast(Type &&src, return_value_policy /* policy */, handle parent) {
return cast_impl(&src, return_value_policy::move, parent);
}
// If you return a non-reference const, we mark the numpy array readonly:
static handle cast(const Type &&src, return_value_policy /* policy */, handle parent) {
return cast_impl(&src, return_value_policy::move, parent);
}
// lvalue reference return; default (automatic) becomes copy
static handle cast(Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast_impl(&src, policy, parent);
}
// const lvalue reference return; default (automatic) becomes copy
static handle cast(const Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast(&src, policy, parent);
}
// non-const pointer return
static handle cast(Type *src, return_value_policy policy, handle parent) {
return cast_impl(src, policy, parent);
}
// const pointer return
static handle cast(const Type *src, return_value_policy policy, handle parent) {
return cast_impl(src, policy, parent);
}
static constexpr auto name = props::descriptor;
// NOLINTNEXTLINE(google-explicit-constructor)
operator Type *() { return &value; }
// NOLINTNEXTLINE(google-explicit-constructor)
operator Type &() { return value; }
// NOLINTNEXTLINE(google-explicit-constructor)
operator Type &&() && { return std::move(value); }
template <typename T>
using cast_op_type = movable_cast_op_type<T>;
private:
Type value;
};
// Base class for casting reference/map/block/etc. objects back to python.
template <typename MapType>
struct eigen_map_caster {
static_assert(!std::is_pointer<typename MapType::Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
private:
using props = EigenProps<MapType>;
public:
// Directly referencing a ref/map's data is a bit dangerous (whatever the map/ref points to has
// to stay around), but we'll allow it under the assumption that you know what you're doing
// (and have an appropriate keep_alive in place). We return a numpy array pointing directly at
// the ref's data (The numpy array ends up read-only if the ref was to a const matrix type.)
// Note that this means you need to ensure you don't destroy the object in some other way (e.g.
// with an appropriate keep_alive, or with a reference to a statically allocated matrix).
static handle cast(const MapType &src, return_value_policy policy, handle parent) {
switch (policy) {
case return_value_policy::copy:
return eigen_array_cast<props>(src);
case return_value_policy::reference_internal:
return eigen_array_cast<props>(src, parent, is_eigen_mutable_map<MapType>::value);
case return_value_policy::reference:
case return_value_policy::automatic:
case return_value_policy::automatic_reference:
return eigen_array_cast<props>(src, none(), is_eigen_mutable_map<MapType>::value);
default:
// move, take_ownership don't make any sense for a ref/map:
pybind11_fail("Invalid return_value_policy for Eigen Map/Ref/Block type");
}
}
static constexpr auto name = props::descriptor;
// Explicitly delete these: support python -> C++ conversion on these (i.e. these can be return
// types but not bound arguments). We still provide them (with an explicitly delete) so that
// you end up here if you try anyway.
bool load(handle, bool) = delete;
operator MapType() = delete;
template <typename>
using cast_op_type = MapType;
};
// We can return any map-like object (but can only load Refs, specialized next):
template <typename Type>
struct type_caster<Type, enable_if_t<is_eigen_dense_map<Type>::value>> : eigen_map_caster<Type> {};
// Loader for Ref<...> arguments. See the documentation for info on how to make this work without
// copying (it requires some extra effort in many cases).
template <typename PlainObjectType, typename StrideType>
struct type_caster<
Eigen::Ref<PlainObjectType, 0, StrideType>,
enable_if_t<is_eigen_dense_map<Eigen::Ref<PlainObjectType, 0, StrideType>>::value>>
: public eigen_map_caster<Eigen::Ref<PlainObjectType, 0, StrideType>> {
private:
using Type = Eigen::Ref<PlainObjectType, 0, StrideType>;
using props = EigenProps<Type>;
using Scalar = typename props::Scalar;
static_assert(!std::is_pointer<Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
using MapType = Eigen::Map<PlainObjectType, 0, StrideType>;
using Array
= array_t<Scalar,
array::forcecast
| ((props::row_major ? props::inner_stride : props::outer_stride) == 1
? array::c_style
: (props::row_major ? props::outer_stride : props::inner_stride) == 1
? array::f_style
: 0)>;
static constexpr bool need_writeable = is_eigen_mutable_map<Type>::value;
// Delay construction (these have no default constructor)
std::unique_ptr<MapType> map;
std::unique_ptr<Type> ref;
// Our array. When possible, this is just a numpy array pointing to the source data, but
// sometimes we can't avoid copying (e.g. input is not a numpy array at all, has an
// incompatible layout, or is an array of a type that needs to be converted). Using a numpy
// temporary (rather than an Eigen temporary) saves an extra copy when we need both type
// conversion and storage order conversion. (Note that we refuse to use this temporary copy
// when loading an argument for a Ref<M> with M non-const, i.e. a read-write reference).
Array copy_or_ref;
public:
bool load(handle src, bool convert) {
// First check whether what we have is already an array of the right type. If not, we
// can't avoid a copy (because the copy is also going to do type conversion).
bool need_copy = !isinstance<Array>(src);
EigenConformable<props::row_major> fits;
if (!need_copy) {
// We don't need a converting copy, but we also need to check whether the strides are
// compatible with the Ref's stride requirements
auto aref = reinterpret_borrow<Array>(src);
if (aref && (!need_writeable || aref.writeable())) {
fits = props::conformable(aref);
if (!fits) {
return false; // Incompatible dimensions
}
if (!fits.template stride_compatible<props>()) {
need_copy = true;
} else {
copy_or_ref = std::move(aref);
}
} else {
need_copy = true;
}
}
if (need_copy) {
// We need to copy: If we need a mutable reference, or we're not supposed to convert
// (either because we're in the no-convert overload pass, or because we're explicitly
// instructed not to copy (via `py::arg().noconvert()`) we have to fail loading.
if (!convert || need_writeable) {
return false;
}
Array copy = Array::ensure(src);
if (!copy) {
return false;
}
fits = props::conformable(copy);
if (!fits || !fits.template stride_compatible<props>()) {
return false;
}
copy_or_ref = std::move(copy);
loader_life_support::add_patient(copy_or_ref);
}
ref.reset();
map.reset(new MapType(data(copy_or_ref),
fits.rows,
fits.cols,
make_stride(fits.stride.outer(), fits.stride.inner())));
ref.reset(new Type(*map));
return true;
}
// NOLINTNEXTLINE(google-explicit-constructor)
operator Type *() { return ref.get(); }
// NOLINTNEXTLINE(google-explicit-constructor)
operator Type &() { return *ref; }
template <typename _T>
using cast_op_type = pybind11::detail::cast_op_type<_T>;
private:
template <typename T = Type, enable_if_t<is_eigen_mutable_map<T>::value, int> = 0>
Scalar *data(Array &a) {
return a.mutable_data();
}
template <typename T = Type, enable_if_t<!is_eigen_mutable_map<T>::value, int> = 0>
const Scalar *data(Array &a) {
return a.data();
}
// Attempt to figure out a constructor of `Stride` that will work.
// If both strides are fixed, use a default constructor:
template <typename S>
using stride_ctor_default = bool_constant<S::InnerStrideAtCompileTime != Eigen::Dynamic
&& S::OuterStrideAtCompileTime != Eigen::Dynamic
&& std::is_default_constructible<S>::value>;
// Otherwise, if there is a two-index constructor, assume it is (outer,inner) like
// Eigen::Stride, and use it:
template <typename S>
using stride_ctor_dual
= bool_constant<!stride_ctor_default<S>::value
&& std::is_constructible<S, EigenIndex, EigenIndex>::value>;
// Otherwise, if there is a one-index constructor, and just one of the strides is dynamic, use
// it (passing whichever stride is dynamic).
template <typename S>
using stride_ctor_outer
= bool_constant<!any_of<stride_ctor_default<S>, stride_ctor_dual<S>>::value
&& S::OuterStrideAtCompileTime == Eigen::Dynamic
&& S::InnerStrideAtCompileTime != Eigen::Dynamic
&& std::is_constructible<S, EigenIndex>::value>;
template <typename S>
using stride_ctor_inner
= bool_constant<!any_of<stride_ctor_default<S>, stride_ctor_dual<S>>::value
&& S::InnerStrideAtCompileTime == Eigen::Dynamic
&& S::OuterStrideAtCompileTime != Eigen::Dynamic
&& std::is_constructible<S, EigenIndex>::value>;
template <typename S = StrideType, enable_if_t<stride_ctor_default<S>::value, int> = 0>
static S make_stride(EigenIndex, EigenIndex) {
return S();
}
template <typename S = StrideType, enable_if_t<stride_ctor_dual<S>::value, int> = 0>
static S make_stride(EigenIndex outer, EigenIndex inner) {
return S(outer, inner);
}
template <typename S = StrideType, enable_if_t<stride_ctor_outer<S>::value, int> = 0>
static S make_stride(EigenIndex outer, EigenIndex) {
return S(outer);
}
template <typename S = StrideType, enable_if_t<stride_ctor_inner<S>::value, int> = 0>
static S make_stride(EigenIndex, EigenIndex inner) {
return S(inner);
}
};
// type_caster for special matrix types (e.g. DiagonalMatrix), which are EigenBase, but not
// EigenDense (i.e. they don't have a data(), at least not with the usual matrix layout).
// load() is not supported, but we can cast them into the python domain by first copying to a
// regular Eigen::Matrix, then casting that.
template <typename Type>
struct type_caster<Type, enable_if_t<is_eigen_other<Type>::value>> {
static_assert(!std::is_pointer<typename Type::Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
protected:
using Matrix
= Eigen::Matrix<typename Type::Scalar, Type::RowsAtCompileTime, Type::ColsAtCompileTime>;
using props = EigenProps<Matrix>;
public:
static handle cast(const Type &src, return_value_policy /* policy */, handle /* parent */) {
handle h = eigen_encapsulate<props>(new Matrix(src));
return h;
}
static handle cast(const Type *src, return_value_policy policy, handle parent) {
return cast(*src, policy, parent);
}
static constexpr auto name = props::descriptor;
// Explicitly delete these: support python -> C++ conversion on these (i.e. these can be return
// types but not bound arguments). We still provide them (with an explicitly delete) so that
// you end up here if you try anyway.
bool load(handle, bool) = delete;
operator Type() = delete;
template <typename>
using cast_op_type = Type;
};
template <typename Type>
struct type_caster<Type, enable_if_t<is_eigen_sparse<Type>::value>> {
using Scalar = typename Type::Scalar;
static_assert(!std::is_pointer<Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
using StorageIndex = remove_reference_t<decltype(*std::declval<Type>().outerIndexPtr())>;
using Index = typename Type::Index;
static constexpr bool rowMajor = Type::IsRowMajor;
bool load(handle src, bool) {
if (!src) {
return false;
}
auto obj = reinterpret_borrow<object>(src);
object sparse_module = module_::import("scipy.sparse");
object matrix_type = sparse_module.attr(rowMajor ? "csr_matrix" : "csc_matrix");
if (!type::handle_of(obj).is(matrix_type)) {
try {
obj = matrix_type(obj);
} catch (const error_already_set &) {
return false;
}
}
auto values = array_t<Scalar>((object) obj.attr("data"));
auto innerIndices = array_t<StorageIndex>((object) obj.attr("indices"));
auto outerIndices = array_t<StorageIndex>((object) obj.attr("indptr"));
auto shape = pybind11::tuple((pybind11::object) obj.attr("shape"));
auto nnz = obj.attr("nnz").cast<Index>();
if (!values || !innerIndices || !outerIndices) {
return false;
}
value = EigenMapSparseMatrix<Scalar,
Type::Flags &(Eigen::RowMajor | Eigen::ColMajor),
StorageIndex>(shape[0].cast<Index>(),
shape[1].cast<Index>(),
std::move(nnz),
outerIndices.mutable_data(),
innerIndices.mutable_data(),
values.mutable_data());
return true;
}
static handle cast(const Type &src, return_value_policy /* policy */, handle /* parent */) {
const_cast<Type &>(src).makeCompressed();
object matrix_type
= module_::import("scipy.sparse").attr(rowMajor ? "csr_matrix" : "csc_matrix");
array data(src.nonZeros(), src.valuePtr());
array outerIndices((rowMajor ? src.rows() : src.cols()) + 1, src.outerIndexPtr());
array innerIndices(src.nonZeros(), src.innerIndexPtr());
return matrix_type(pybind11::make_tuple(
std::move(data), std::move(innerIndices), std::move(outerIndices)),
pybind11::make_tuple(src.rows(), src.cols()))
.release();
}
PYBIND11_TYPE_CASTER(Type,
const_name<(Type::IsRowMajor) != 0>("scipy.sparse.csr_matrix[",
"scipy.sparse.csc_matrix[")
+ npy_format_descriptor<Scalar>::name + const_name("]"));
};
PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -0,0 +1,517 @@
/*
pybind11/eigen/tensor.h: Transparent conversion for Eigen tensors
All rights reserved. Use of this source code is governed by a
BSD-style license that can be found in the LICENSE file.
*/
#pragma once
#include "../numpy.h"
#include "common.h"
#if defined(__GNUC__) && !defined(__clang__) && !defined(__INTEL_COMPILER)
static_assert(__GNUC__ > 5, "Eigen Tensor support in pybind11 requires GCC > 5.0");
#endif
// Disable warnings for Eigen
PYBIND11_WARNING_PUSH
PYBIND11_WARNING_DISABLE_MSVC(4554)
PYBIND11_WARNING_DISABLE_MSVC(4127)
#if defined(__MINGW32__)
PYBIND11_WARNING_DISABLE_GCC("-Wmaybe-uninitialized")
#endif
#include <unsupported/Eigen/CXX11/Tensor>
PYBIND11_WARNING_POP
static_assert(EIGEN_VERSION_AT_LEAST(3, 3, 0),
"Eigen Tensor support in pybind11 requires Eigen >= 3.3.0");
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_WARNING_DISABLE_MSVC(4127)
PYBIND11_NAMESPACE_BEGIN(detail)
inline bool is_tensor_aligned(const void *data) {
return (reinterpret_cast<std::size_t>(data) % EIGEN_DEFAULT_ALIGN_BYTES) == 0;
}
template <typename T>
constexpr int compute_array_flag_from_tensor() {
static_assert((static_cast<int>(T::Layout) == static_cast<int>(Eigen::RowMajor))
|| (static_cast<int>(T::Layout) == static_cast<int>(Eigen::ColMajor)),
"Layout must be row or column major");
return (static_cast<int>(T::Layout) == static_cast<int>(Eigen::RowMajor)) ? array::c_style
: array::f_style;
}
template <typename T>
struct eigen_tensor_helper {};
template <typename Scalar_, int NumIndices_, int Options_, typename IndexType>
struct eigen_tensor_helper<Eigen::Tensor<Scalar_, NumIndices_, Options_, IndexType>> {
using Type = Eigen::Tensor<Scalar_, NumIndices_, Options_, IndexType>;
using ValidType = void;
static Eigen::DSizes<typename Type::Index, Type::NumIndices> get_shape(const Type &f) {
return f.dimensions();
}
static constexpr bool
is_correct_shape(const Eigen::DSizes<typename Type::Index, Type::NumIndices> & /*shape*/) {
return true;
}
template <typename T>
struct helper {};
template <size_t... Is>
struct helper<index_sequence<Is...>> {
static constexpr auto value = ::pybind11::detail::concat(const_name(((void) Is, "?"))...);
};
static constexpr auto dimensions_descriptor
= helper<decltype(make_index_sequence<Type::NumIndices>())>::value;
template <typename... Args>
static Type *alloc(Args &&...args) {
return new Type(std::forward<Args>(args)...);
}
static void free(Type *tensor) { delete tensor; }
};
template <typename Scalar_, typename std::ptrdiff_t... Indices, int Options_, typename IndexType>
struct eigen_tensor_helper<
Eigen::TensorFixedSize<Scalar_, Eigen::Sizes<Indices...>, Options_, IndexType>> {
using Type = Eigen::TensorFixedSize<Scalar_, Eigen::Sizes<Indices...>, Options_, IndexType>;
using ValidType = void;
static constexpr Eigen::DSizes<typename Type::Index, Type::NumIndices>
get_shape(const Type & /*f*/) {
return get_shape();
}
static constexpr Eigen::DSizes<typename Type::Index, Type::NumIndices> get_shape() {
return Eigen::DSizes<typename Type::Index, Type::NumIndices>(Indices...);
}
static bool
is_correct_shape(const Eigen::DSizes<typename Type::Index, Type::NumIndices> &shape) {
return get_shape() == shape;
}
static constexpr auto dimensions_descriptor
= ::pybind11::detail::concat(const_name<Indices>()...);
template <typename... Args>
static Type *alloc(Args &&...args) {
Eigen::aligned_allocator<Type> allocator;
return ::new (allocator.allocate(1)) Type(std::forward<Args>(args)...);
}
static void free(Type *tensor) {
Eigen::aligned_allocator<Type> allocator;
tensor->~Type();
allocator.deallocate(tensor, 1);
}
};
template <typename Type, bool ShowDetails, bool NeedsWriteable = false>
struct get_tensor_descriptor {
static constexpr auto details
= const_name<NeedsWriteable>(", flags.writeable", "")
+ const_name<static_cast<int>(Type::Layout) == static_cast<int>(Eigen::RowMajor)>(
", flags.c_contiguous", ", flags.f_contiguous");
static constexpr auto value
= const_name("numpy.ndarray[") + npy_format_descriptor<typename Type::Scalar>::name
+ const_name("[") + eigen_tensor_helper<remove_cv_t<Type>>::dimensions_descriptor
+ const_name("]") + const_name<ShowDetails>(details, const_name("")) + const_name("]");
};
// When EIGEN_AVOID_STL_ARRAY is defined, Eigen::DSizes<T, 0> does not have the begin() member
// function. Falling back to a simple loop works around this issue.
//
// We need to disable the type-limits warning for the inner loop when size = 0.
PYBIND11_WARNING_PUSH
PYBIND11_WARNING_DISABLE_GCC("-Wtype-limits")
template <typename T, int size>
std::vector<T> convert_dsizes_to_vector(const Eigen::DSizes<T, size> &arr) {
std::vector<T> result(size);
for (size_t i = 0; i < size; i++) {
result[i] = arr[i];
}
return result;
}
template <typename T, int size>
Eigen::DSizes<T, size> get_shape_for_array(const array &arr) {
Eigen::DSizes<T, size> result;
const T *shape = arr.shape();
for (size_t i = 0; i < size; i++) {
result[i] = shape[i];
}
return result;
}
PYBIND11_WARNING_POP
template <typename Type>
struct type_caster<Type, typename eigen_tensor_helper<Type>::ValidType> {
static_assert(!std::is_pointer<typename Type::Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
using Helper = eigen_tensor_helper<Type>;
static constexpr auto temp_name = get_tensor_descriptor<Type, false>::value;
PYBIND11_TYPE_CASTER(Type, temp_name);
bool load(handle src, bool convert) {
if (!convert) {
if (!isinstance<array>(src)) {
return false;
}
array temp = array::ensure(src);
if (!temp) {
return false;
}
if (!temp.dtype().is(dtype::of<typename Type::Scalar>())) {
return false;
}
}
array_t<typename Type::Scalar, compute_array_flag_from_tensor<Type>()> arr(
reinterpret_borrow<object>(src));
if (arr.ndim() != Type::NumIndices) {
return false;
}
auto shape = get_shape_for_array<typename Type::Index, Type::NumIndices>(arr);
if (!Helper::is_correct_shape(shape)) {
return false;
}
#if EIGEN_VERSION_AT_LEAST(3, 4, 0)
auto data_pointer = arr.data();
#else
// Handle Eigen bug
auto data_pointer = const_cast<typename Type::Scalar *>(arr.data());
#endif
if (is_tensor_aligned(arr.data())) {
value = Eigen::TensorMap<const Type, Eigen::Aligned>(data_pointer, shape);
} else {
value = Eigen::TensorMap<const Type>(data_pointer, shape);
}
return true;
}
static handle cast(Type &&src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::reference
|| policy == return_value_policy::reference_internal) {
pybind11_fail("Cannot use a reference return value policy for an rvalue");
}
return cast_impl(&src, return_value_policy::move, parent);
}
static handle cast(const Type &&src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::reference
|| policy == return_value_policy::reference_internal) {
pybind11_fail("Cannot use a reference return value policy for an rvalue");
}
return cast_impl(&src, return_value_policy::move, parent);
}
static handle cast(Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast_impl(&src, policy, parent);
}
static handle cast(const Type &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast(&src, policy, parent);
}
static handle cast(Type *src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic) {
policy = return_value_policy::take_ownership;
} else if (policy == return_value_policy::automatic_reference) {
policy = return_value_policy::reference;
}
return cast_impl(src, policy, parent);
}
static handle cast(const Type *src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic) {
policy = return_value_policy::take_ownership;
} else if (policy == return_value_policy::automatic_reference) {
policy = return_value_policy::reference;
}
return cast_impl(src, policy, parent);
}
template <typename C>
static handle cast_impl(C *src, return_value_policy policy, handle parent) {
object parent_object;
bool writeable = false;
switch (policy) {
case return_value_policy::move:
if (std::is_const<C>::value) {
pybind11_fail("Cannot move from a constant reference");
}
src = Helper::alloc(std::move(*src));
parent_object
= capsule(src, [](void *ptr) { Helper::free(reinterpret_cast<Type *>(ptr)); });
writeable = true;
break;
case return_value_policy::take_ownership:
if (std::is_const<C>::value) {
// This cast is ugly, and might be UB in some cases, but we don't have an
// alternative here as we must free that memory
Helper::free(const_cast<Type *>(src));
pybind11_fail("Cannot take ownership of a const reference");
}
parent_object
= capsule(src, [](void *ptr) { Helper::free(reinterpret_cast<Type *>(ptr)); });
writeable = true;
break;
case return_value_policy::copy:
writeable = true;
break;
case return_value_policy::reference:
parent_object = none();
writeable = !std::is_const<C>::value;
break;
case return_value_policy::reference_internal:
// Default should do the right thing
if (!parent) {
pybind11_fail("Cannot use reference internal when there is no parent");
}
parent_object = reinterpret_borrow<object>(parent);
writeable = !std::is_const<C>::value;
break;
default:
pybind11_fail("pybind11 bug in eigen.h, please file a bug report");
}
auto result = array_t<typename Type::Scalar, compute_array_flag_from_tensor<Type>()>(
convert_dsizes_to_vector(Helper::get_shape(*src)), src->data(), parent_object);
if (!writeable) {
array_proxy(result.ptr())->flags &= ~detail::npy_api::NPY_ARRAY_WRITEABLE_;
}
return result.release();
}
};
template <typename StoragePointerType,
bool needs_writeable,
enable_if_t<!needs_writeable, bool> = true>
StoragePointerType get_array_data_for_type(array &arr) {
#if EIGEN_VERSION_AT_LEAST(3, 4, 0)
return reinterpret_cast<StoragePointerType>(arr.data());
#else
// Handle Eigen bug
return reinterpret_cast<StoragePointerType>(const_cast<void *>(arr.data()));
#endif
}
template <typename StoragePointerType,
bool needs_writeable,
enable_if_t<needs_writeable, bool> = true>
StoragePointerType get_array_data_for_type(array &arr) {
return reinterpret_cast<StoragePointerType>(arr.mutable_data());
}
template <typename T, typename = void>
struct get_storage_pointer_type;
template <typename MapType>
struct get_storage_pointer_type<MapType, void_t<typename MapType::StoragePointerType>> {
using SPT = typename MapType::StoragePointerType;
};
template <typename MapType>
struct get_storage_pointer_type<MapType, void_t<typename MapType::PointerArgType>> {
using SPT = typename MapType::PointerArgType;
};
template <typename Type, int Options>
struct type_caster<Eigen::TensorMap<Type, Options>,
typename eigen_tensor_helper<remove_cv_t<Type>>::ValidType> {
static_assert(!std::is_pointer<typename Type::Scalar>::value,
PYBIND11_EIGEN_MESSAGE_POINTER_TYPES_ARE_NOT_SUPPORTED);
using MapType = Eigen::TensorMap<Type, Options>;
using Helper = eigen_tensor_helper<remove_cv_t<Type>>;
bool load(handle src, bool /*convert*/) {
// Note that we have a lot more checks here as we want to make sure to avoid copies
if (!isinstance<array>(src)) {
return false;
}
auto arr = reinterpret_borrow<array>(src);
if ((arr.flags() & compute_array_flag_from_tensor<Type>()) == 0) {
return false;
}
if (!arr.dtype().is(dtype::of<typename Type::Scalar>())) {
return false;
}
if (arr.ndim() != Type::NumIndices) {
return false;
}
constexpr bool is_aligned = (Options & Eigen::Aligned) != 0;
if (is_aligned && !is_tensor_aligned(arr.data())) {
return false;
}
auto shape = get_shape_for_array<typename Type::Index, Type::NumIndices>(arr);
if (!Helper::is_correct_shape(shape)) {
return false;
}
if (needs_writeable && !arr.writeable()) {
return false;
}
auto result = get_array_data_for_type<typename get_storage_pointer_type<MapType>::SPT,
needs_writeable>(arr);
value.reset(new MapType(std::move(result), std::move(shape)));
return true;
}
static handle cast(MapType &&src, return_value_policy policy, handle parent) {
return cast_impl(&src, policy, parent);
}
static handle cast(const MapType &&src, return_value_policy policy, handle parent) {
return cast_impl(&src, policy, parent);
}
static handle cast(MapType &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast_impl(&src, policy, parent);
}
static handle cast(const MapType &src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic
|| policy == return_value_policy::automatic_reference) {
policy = return_value_policy::copy;
}
return cast(&src, policy, parent);
}
static handle cast(MapType *src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic) {
policy = return_value_policy::take_ownership;
} else if (policy == return_value_policy::automatic_reference) {
policy = return_value_policy::reference;
}
return cast_impl(src, policy, parent);
}
static handle cast(const MapType *src, return_value_policy policy, handle parent) {
if (policy == return_value_policy::automatic) {
policy = return_value_policy::take_ownership;
} else if (policy == return_value_policy::automatic_reference) {
policy = return_value_policy::reference;
}
return cast_impl(src, policy, parent);
}
template <typename C>
static handle cast_impl(C *src, return_value_policy policy, handle parent) {
object parent_object;
constexpr bool writeable = !std::is_const<C>::value;
switch (policy) {
case return_value_policy::reference:
parent_object = none();
break;
case return_value_policy::reference_internal:
// Default should do the right thing
if (!parent) {
pybind11_fail("Cannot use reference internal when there is no parent");
}
parent_object = reinterpret_borrow<object>(parent);
break;
case return_value_policy::take_ownership:
delete src;
// fallthrough
default:
// move, take_ownership don't make any sense for a ref/map:
pybind11_fail("Invalid return_value_policy for Eigen Map type, must be either "
"reference or reference_internal");
}
auto result = array_t<typename Type::Scalar, compute_array_flag_from_tensor<Type>()>(
convert_dsizes_to_vector(Helper::get_shape(*src)),
src->data(),
std::move(parent_object));
if (!writeable) {
array_proxy(result.ptr())->flags &= ~detail::npy_api::NPY_ARRAY_WRITEABLE_;
}
return result.release();
}
#if EIGEN_VERSION_AT_LEAST(3, 4, 0)
static constexpr bool needs_writeable = !std::is_const<typename std::remove_pointer<
typename get_storage_pointer_type<MapType>::SPT>::type>::value;
#else
// Handle Eigen bug
static constexpr bool needs_writeable = !std::is_const<Type>::value;
#endif
protected:
// TODO: Move to std::optional once std::optional has more support
std::unique_ptr<MapType> value;
public:
static constexpr auto name = get_tensor_descriptor<Type, true, needs_writeable>::value;
explicit operator MapType *() { return value.get(); }
explicit operator MapType &() { return *value; }
explicit operator MapType &&() && { return std::move(*value); }
template <typename T_>
using cast_op_type = ::pybind11::detail::movable_cast_op_type<T_>;
};
PYBIND11_NAMESPACE_END(detail)
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

View File

@ -12,23 +12,16 @@
#include "pybind11.h" #include "pybind11.h"
#include "eval.h" #include "eval.h"
#include <memory>
#include <vector>
#if defined(PYPY_VERSION) #if defined(PYPY_VERSION)
# error Embedding the interpreter is not supported with PyPy # error Embedding the interpreter is not supported with PyPy
#endif #endif
#if PY_MAJOR_VERSION >= 3 #define PYBIND11_EMBEDDED_MODULE_IMPL(name) \
# define PYBIND11_EMBEDDED_MODULE_IMPL(name) \ extern "C" PyObject *pybind11_init_impl_##name(); \
extern "C" PyObject *pybind11_init_impl_##name(); \ extern "C" PyObject *pybind11_init_impl_##name() { return pybind11_init_wrapper_##name(); }
extern "C" PyObject *pybind11_init_impl_##name() { \
return pybind11_init_wrapper_##name(); \
}
#else
# define PYBIND11_EMBEDDED_MODULE_IMPL(name) \
extern "C" void pybind11_init_impl_##name(); \
extern "C" void pybind11_init_impl_##name() { \
pybind11_init_wrapper_##name(); \
}
#endif
/** \rst /** \rst
Add a new module to the table of builtins for the interpreter. Must be Add a new module to the table of builtins for the interpreter. Must be
@ -45,69 +38,173 @@
}); });
} }
\endrst */ \endrst */
#define PYBIND11_EMBEDDED_MODULE(name, variable) \ #define PYBIND11_EMBEDDED_MODULE(name, variable) \
static ::pybind11::module_::module_def \ static ::pybind11::module_::module_def PYBIND11_CONCAT(pybind11_module_def_, name); \
PYBIND11_CONCAT(pybind11_module_def_, name); \ static void PYBIND11_CONCAT(pybind11_init_, name)(::pybind11::module_ &); \
static void PYBIND11_CONCAT(pybind11_init_, name)(::pybind11::module_ &); \ static PyObject PYBIND11_CONCAT(*pybind11_init_wrapper_, name)() { \
static PyObject PYBIND11_CONCAT(*pybind11_init_wrapper_, name)() { \ auto m = ::pybind11::module_::create_extension_module( \
auto m = ::pybind11::module_::create_extension_module( \ PYBIND11_TOSTRING(name), nullptr, &PYBIND11_CONCAT(pybind11_module_def_, name)); \
PYBIND11_TOSTRING(name), nullptr, \ try { \
&PYBIND11_CONCAT(pybind11_module_def_, name)); \ PYBIND11_CONCAT(pybind11_init_, name)(m); \
try { \ return m.ptr(); \
PYBIND11_CONCAT(pybind11_init_, name)(m); \ } \
return m.ptr(); \ PYBIND11_CATCH_INIT_EXCEPTIONS \
} PYBIND11_CATCH_INIT_EXCEPTIONS \ } \
} \ PYBIND11_EMBEDDED_MODULE_IMPL(name) \
PYBIND11_EMBEDDED_MODULE_IMPL(name) \ ::pybind11::detail::embedded_module PYBIND11_CONCAT(pybind11_module_, name)( \
::pybind11::detail::embedded_module PYBIND11_CONCAT(pybind11_module_, name) \ PYBIND11_TOSTRING(name), PYBIND11_CONCAT(pybind11_init_impl_, name)); \
(PYBIND11_TOSTRING(name), \ void PYBIND11_CONCAT(pybind11_init_, name)(::pybind11::module_ \
PYBIND11_CONCAT(pybind11_init_impl_, name)); \ & variable) // NOLINT(bugprone-macro-parentheses)
void PYBIND11_CONCAT(pybind11_init_, name)(::pybind11::module_ &variable)
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
PYBIND11_NAMESPACE_BEGIN(detail) PYBIND11_NAMESPACE_BEGIN(detail)
/// Python 2.7/3.x compatible version of `PyImport_AppendInittab` and error checks. /// Python 2.7/3.x compatible version of `PyImport_AppendInittab` and error checks.
struct embedded_module { struct embedded_module {
#if PY_MAJOR_VERSION >= 3 using init_t = PyObject *(*) ();
using init_t = PyObject *(*)();
#else
using init_t = void (*)();
#endif
embedded_module(const char *name, init_t init) { embedded_module(const char *name, init_t init) {
if (Py_IsInitialized()) if (Py_IsInitialized() != 0) {
pybind11_fail("Can't add new modules after the interpreter has been initialized"); pybind11_fail("Can't add new modules after the interpreter has been initialized");
}
auto result = PyImport_AppendInittab(name, init); auto result = PyImport_AppendInittab(name, init);
if (result == -1) if (result == -1) {
pybind11_fail("Insufficient memory to add a new module"); pybind11_fail("Insufficient memory to add a new module");
}
} }
}; };
struct wide_char_arg_deleter {
void operator()(wchar_t *ptr) const {
// API docs: https://docs.python.org/3/c-api/sys.html#c.Py_DecodeLocale
PyMem_RawFree(ptr);
}
};
inline wchar_t *widen_chars(const char *safe_arg) {
wchar_t *widened_arg = Py_DecodeLocale(safe_arg, nullptr);
return widened_arg;
}
inline void precheck_interpreter() {
if (Py_IsInitialized() != 0) {
pybind11_fail("The interpreter is already running");
}
}
#if !defined(PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX)
# define PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX (0x03080000)
#endif
#if PY_VERSION_HEX < PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX
inline void initialize_interpreter_pre_pyconfig(bool init_signal_handlers,
int argc,
const char *const *argv,
bool add_program_dir_to_path) {
detail::precheck_interpreter();
Py_InitializeEx(init_signal_handlers ? 1 : 0);
# if defined(WITH_THREAD) && PY_VERSION_HEX < 0x03070000
PyEval_InitThreads();
# endif
// Before it was special-cased in python 3.8, passing an empty or null argv
// caused a segfault, so we have to reimplement the special case ourselves.
bool special_case = (argv == nullptr || argc <= 0);
const char *const empty_argv[]{"\0"};
const char *const *safe_argv = special_case ? empty_argv : argv;
if (special_case) {
argc = 1;
}
auto argv_size = static_cast<size_t>(argc);
// SetArgv* on python 3 takes wchar_t, so we have to convert.
std::unique_ptr<wchar_t *[]> widened_argv(new wchar_t *[argv_size]);
std::vector<std::unique_ptr<wchar_t[], detail::wide_char_arg_deleter>> widened_argv_entries;
widened_argv_entries.reserve(argv_size);
for (size_t ii = 0; ii < argv_size; ++ii) {
widened_argv_entries.emplace_back(detail::widen_chars(safe_argv[ii]));
if (!widened_argv_entries.back()) {
// A null here indicates a character-encoding failure or the python
// interpreter out of memory. Give up.
return;
}
widened_argv[ii] = widened_argv_entries.back().get();
}
auto *pysys_argv = widened_argv.get();
PySys_SetArgvEx(argc, pysys_argv, static_cast<int>(add_program_dir_to_path));
}
#endif
PYBIND11_NAMESPACE_END(detail) PYBIND11_NAMESPACE_END(detail)
#if PY_VERSION_HEX >= PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX
inline void initialize_interpreter(PyConfig *config,
int argc = 0,
const char *const *argv = nullptr,
bool add_program_dir_to_path = true) {
detail::precheck_interpreter();
PyStatus status = PyConfig_SetBytesArgv(config, argc, const_cast<char *const *>(argv));
if (PyStatus_Exception(status) != 0) {
// A failure here indicates a character-encoding failure or the python
// interpreter out of memory. Give up.
PyConfig_Clear(config);
throw std::runtime_error(PyStatus_IsError(status) != 0 ? status.err_msg
: "Failed to prepare CPython");
}
status = Py_InitializeFromConfig(config);
if (PyStatus_Exception(status) != 0) {
PyConfig_Clear(config);
throw std::runtime_error(PyStatus_IsError(status) != 0 ? status.err_msg
: "Failed to init CPython");
}
if (add_program_dir_to_path) {
PyRun_SimpleString("import sys, os.path; "
"sys.path.insert(0, "
"os.path.abspath(os.path.dirname(sys.argv[0])) "
"if sys.argv and os.path.exists(sys.argv[0]) else '')");
}
PyConfig_Clear(config);
}
#endif
/** \rst /** \rst
Initialize the Python interpreter. No other pybind11 or CPython API functions can be Initialize the Python interpreter. No other pybind11 or CPython API functions can be
called before this is done; with the exception of `PYBIND11_EMBEDDED_MODULE`. The called before this is done; with the exception of `PYBIND11_EMBEDDED_MODULE`. The
optional parameter can be used to skip the registration of signal handlers (see the optional `init_signal_handlers` parameter can be used to skip the registration of
`Python documentation`_ for details). Calling this function again after the interpreter signal handlers (see the `Python documentation`_ for details). Calling this function
has already been initialized is a fatal error. again after the interpreter has already been initialized is a fatal error.
If initializing the Python interpreter fails, then the program is terminated. (This If initializing the Python interpreter fails, then the program is terminated. (This
is controlled by the CPython runtime and is an exception to pybind11's normal behavior is controlled by the CPython runtime and is an exception to pybind11's normal behavior
of throwing exceptions on errors.) of throwing exceptions on errors.)
The remaining optional parameters, `argc`, `argv`, and `add_program_dir_to_path` are
used to populate ``sys.argv`` and ``sys.path``.
See the |PySys_SetArgvEx documentation|_ for details.
.. _Python documentation: https://docs.python.org/3/c-api/init.html#c.Py_InitializeEx .. _Python documentation: https://docs.python.org/3/c-api/init.html#c.Py_InitializeEx
.. |PySys_SetArgvEx documentation| replace:: ``PySys_SetArgvEx`` documentation
.. _PySys_SetArgvEx documentation: https://docs.python.org/3/c-api/init.html#c.PySys_SetArgvEx
\endrst */ \endrst */
inline void initialize_interpreter(bool init_signal_handlers = true) { inline void initialize_interpreter(bool init_signal_handlers = true,
if (Py_IsInitialized()) int argc = 0,
pybind11_fail("The interpreter is already running"); const char *const *argv = nullptr,
bool add_program_dir_to_path = true) {
#if PY_VERSION_HEX < PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX
detail::initialize_interpreter_pre_pyconfig(
init_signal_handlers, argc, argv, add_program_dir_to_path);
#else
PyConfig config;
PyConfig_InitPythonConfig(&config);
// See PR #4473 for background
config.parse_argv = 0;
Py_InitializeEx(init_signal_handlers ? 1 : 0); config.install_signal_handlers = init_signal_handlers ? 1 : 0;
initialize_interpreter(&config, argc, argv, add_program_dir_to_path);
// Make .py files in the working directory available by default #endif
module_::import("sys").attr("path").cast<list>().append(".");
} }
/** \rst /** \rst
@ -146,16 +243,19 @@ inline void initialize_interpreter(bool init_signal_handlers = true) {
\endrst */ \endrst */
inline void finalize_interpreter() { inline void finalize_interpreter() {
handle builtins(PyEval_GetBuiltins());
const char *id = PYBIND11_INTERNALS_ID;
// Get the internals pointer (without creating it if it doesn't exist). It's possible for the // Get the internals pointer (without creating it if it doesn't exist). It's possible for the
// internals to be created during Py_Finalize() (e.g. if a py::capsule calls `get_internals()` // internals to be created during Py_Finalize() (e.g. if a py::capsule calls `get_internals()`
// during destruction), so we get the pointer-pointer here and check it after Py_Finalize(). // during destruction), so we get the pointer-pointer here and check it after Py_Finalize().
detail::internals **internals_ptr_ptr = detail::get_internals_pp(); detail::internals **internals_ptr_ptr = detail::get_internals_pp();
// It could also be stashed in builtins, so look there too: // It could also be stashed in state_dict, so look there too:
if (builtins.contains(id) && isinstance<capsule>(builtins[id])) if (object internals_obj
internals_ptr_ptr = capsule(builtins[id]); = get_internals_obj_from_state_dict(detail::get_python_state_dict())) {
internals_ptr_ptr = detail::get_internals_pp_from_capsule(internals_obj);
}
// Local internals contains data managed by the current interpreter, so we must clear them to
// avoid undefined behaviors when initializing another interpreter
detail::get_local_internals().registered_types_cpp.clear();
detail::get_local_internals().registered_exception_translators.clear();
Py_Finalize(); Py_Finalize();
@ -169,6 +269,8 @@ inline void finalize_interpreter() {
Scope guard version of `initialize_interpreter` and `finalize_interpreter`. Scope guard version of `initialize_interpreter` and `finalize_interpreter`.
This a move-only guard and only a single instance can exist. This a move-only guard and only a single instance can exist.
See `initialize_interpreter` for a discussion of its constructor arguments.
.. code-block:: cpp .. code-block:: cpp
#include <pybind11/embed.h> #include <pybind11/embed.h>
@ -180,18 +282,31 @@ inline void finalize_interpreter() {
\endrst */ \endrst */
class scoped_interpreter { class scoped_interpreter {
public: public:
scoped_interpreter(bool init_signal_handlers = true) { explicit scoped_interpreter(bool init_signal_handlers = true,
initialize_interpreter(init_signal_handlers); int argc = 0,
const char *const *argv = nullptr,
bool add_program_dir_to_path = true) {
initialize_interpreter(init_signal_handlers, argc, argv, add_program_dir_to_path);
} }
#if PY_VERSION_HEX >= PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX
explicit scoped_interpreter(PyConfig *config,
int argc = 0,
const char *const *argv = nullptr,
bool add_program_dir_to_path = true) {
initialize_interpreter(config, argc, argv, add_program_dir_to_path);
}
#endif
scoped_interpreter(const scoped_interpreter &) = delete; scoped_interpreter(const scoped_interpreter &) = delete;
scoped_interpreter(scoped_interpreter &&other) noexcept { other.is_valid = false; } scoped_interpreter(scoped_interpreter &&other) noexcept { other.is_valid = false; }
scoped_interpreter &operator=(const scoped_interpreter &) = delete; scoped_interpreter &operator=(const scoped_interpreter &) = delete;
scoped_interpreter &operator=(scoped_interpreter &&) = delete; scoped_interpreter &operator=(scoped_interpreter &&) = delete;
~scoped_interpreter() { ~scoped_interpreter() {
if (is_valid) if (is_valid) {
finalize_interpreter(); finalize_interpreter();
}
} }
private: private:

Some files were not shown because too many files have changed in this diff Show More