-
-
Notifications
You must be signed in to change notification settings - Fork 156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
implement speedups with rust v2 #438
Open
carsonburr
wants to merge
16
commits into
pallets:main
Choose a base branch
from
carsonburr:rust_v2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 15 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
233a133
rust attempt v2. needs build setup
carsonburr 420abeb
docs tweaks
carsonburr ad8caf1
comment tweaks
carsonburr 62c4087
add tests
carsonburr ec70e30
added benchmark against naive implementation
carsonburr 2869935
added benchmark to compare native python vs naive rust speedups vs op…
carsonburr e48ca4b
mask operates on 4 vectors at a time to try to squeeze some ilp perf
carsonburr 1199641
Merge branch 'main' into rust_v2
carsonburr a21d5a1
fix rust_speedups export and add it to bench.py
carsonburr 4a3e904
remove naive rust impl signature
carsonburr 97ee7e8
fix _escape_inner signature
carsonburr 7755c7e
fix _escape_inner signature attempt 2
carsonburr adcba6f
rust speedups: simplified to lookup table instead of simd
carsonburr ca768ad
rust speedups: instruction-level-parallelsim for short circuit check
carsonburr b14f35d
rust speedups: bug fix
carsonburr e431070
rust speedups: changed table representation
carsonburr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,3 +11,4 @@ dist/ | |
htmlcov/ | ||
.tox/ | ||
docs/_build/ | ||
/src/rust/target/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,84 +1,30 @@ | ||
import os | ||
import platform | ||
import sys | ||
|
||
from setuptools import Extension | ||
from setuptools import setup | ||
from setuptools.command.build_ext import build_ext | ||
from setuptools.errors import CCompilerError | ||
from setuptools.errors import ExecError | ||
from setuptools.errors import PlatformError | ||
from setuptools_rust import RustExtension | ||
|
||
ext_modules = [Extension("markupsafe._speedups", ["src/markupsafe/_speedups.c"])] | ||
|
||
|
||
class BuildFailed(Exception): | ||
pass | ||
|
||
|
||
class ve_build_ext(build_ext): | ||
"""This class allows C extension building to fail.""" | ||
|
||
def run(self): | ||
try: | ||
super().run() | ||
except PlatformError as e: | ||
raise BuildFailed() from e | ||
|
||
def build_extension(self, ext): | ||
try: | ||
super().build_extension(ext) | ||
except (CCompilerError, ExecError, PlatformError) as e: | ||
raise BuildFailed() from e | ||
except ValueError as e: | ||
# this can happen on Windows 64 bit, see Python issue 7511 | ||
if "'path'" in str(sys.exc_info()[1]): # works with Python 2 and 3 | ||
raise BuildFailed() from e | ||
raise | ||
|
||
|
||
def run_setup(with_binary): | ||
setup( | ||
cmdclass={"build_ext": ve_build_ext}, | ||
ext_modules=ext_modules if with_binary else [], | ||
) | ||
|
||
|
||
def show_message(*lines): | ||
print("=" * 74) | ||
for line in lines: | ||
print(line) | ||
print("=" * 74) | ||
|
||
|
||
supports_speedups = platform.python_implementation() not in { | ||
if platform.python_implementation() not in { | ||
"PyPy", | ||
"Jython", | ||
"GraalVM", | ||
} | ||
|
||
if os.environ.get("CIBUILDWHEEL", "0") == "1" and supports_speedups: | ||
run_setup(True) | ||
elif supports_speedups: | ||
try: | ||
run_setup(True) | ||
except BuildFailed: | ||
show_message( | ||
"WARNING: The C extension could not be compiled, speedups" | ||
" are not enabled.", | ||
"Failure information, if any, is above.", | ||
"Retrying the build without the C extension now.", | ||
) | ||
run_setup(False) | ||
show_message( | ||
"WARNING: The C extension could not be compiled, speedups" | ||
" are not enabled.", | ||
"Plain-Python build succeeded.", | ||
) | ||
else: | ||
run_setup(False) | ||
show_message( | ||
"WARNING: C extensions are not supported on this Python" | ||
" platform, speedups are not enabled.", | ||
"Plain-Python build succeeded.", | ||
}: | ||
local = os.environ.get("CIBUILDWHEEL", "0") != "1" | ||
setup( | ||
ext_modules=[ | ||
Extension( | ||
"markupsafe._speedups", ["src/markupsafe/_speedups.c"], optional=local | ||
) | ||
], | ||
rust_extensions=[ | ||
RustExtension( | ||
"markupsafe._rust_speedups", | ||
"src/rust/Cargo.toml", | ||
optional=local, | ||
debug=False, | ||
) | ||
], | ||
) | ||
else: | ||
setup() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
def _escape_inner(s: str, /) -> str: ... |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
[package] | ||
name = "markupsafe-rust" | ||
version = "0.1.0" | ||
edition = "2021" | ||
publish = false | ||
|
||
[profile.release] | ||
debug = true | ||
|
||
[dependencies] | ||
pyo3 = "0.22.2" | ||
|
||
[lib] | ||
name = "_rust_speedups" | ||
crate-type = ["cdylib"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@davidism I guess if you want abi3, can just enable the feature: