Skip to content

Commit 268f0a3

Browse files
committed
Add property-based tests using Hypothesis
property_tests.py checks general properties of jsonpatch against generated documents and patches: - make_patch: diffing equal documents gives an empty patch, applying make_patch(src, dst) to src gives dst, inputs are not modified, the result shares no objects with dst - apply_patch: the document and the patch are not modified, only JsonPatchException / JsonPointerException are raised (also for malformed patches), in_place and string patches give the same result - single operations: whole-document targets and sources, moving into own children, replace of any existing location, 'test' using JSON equality - JsonPatch: string roundtrip, equal patches have equal hashes - a state machine applying random valid operations and comparing each result with a direct implementation of RFC 6902 Properties that do not hold yet are expected failures, each pinned to the minimal counterexample Hypothesis found. Among them are known issues (#124, #137, #138, #179, #180, #190) and some that appear to be unreported: make_patch mishandling object keys that int() accepts, replace rejecting the object member '-', hash() failing for patches with array/object values, 'test' treating 1 and true as equal, copy from the whole document failing, and moves into own children being allowed for array members. The tests run in CI and via "make test". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqGXptMsaz9bfrMoaMpZ1A
1 parent 4f39f79 commit 268f0a3

6 files changed

Lines changed: 618 additions & 1 deletion

File tree

‎.github/workflows/test.yaml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
- name: Install dependencies
2121
run: |
2222
python -m pip install --upgrade pip
23-
pip install flake8 pytest
23+
pip install flake8 pytest hypothesis
2424
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
2525
pip install coveralls
2626
# - name: Lint with flake8
@@ -32,3 +32,6 @@ jobs:
3232
- name: Test
3333
run: |
3434
coverage run --source=jsonpointer tests.py
35+
- name: Property-based tests
36+
run: |
37+
python property_tests.py

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ build
44
dist
55
*.egg-info/
66
doc/_build
7+
.hypothesis

‎README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,14 @@ Running external tests
2626
To run external tests (such as those from https://github.com/json-patch/json-patch-tests) use ext_test.py
2727

2828
./ext_tests.py ../json-patch-tests/tests.json
29+
30+
Running property-based tests
31+
----------------------------
32+
Property-based tests using [Hypothesis](https://hypothesis.readthedocs.io/)
33+
are in property_tests.py. Properties that the implementation does not satisfy
34+
yet are marked as expected failures.
35+
36+
pip install hypothesis
37+
python property_tests.py
38+
39+
Set `HYPOTHESIS_PROFILE=thorough` to run many more examples.

‎makefile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ help:
1111

1212
test:
1313
python -Wd -m coverage run --branch --source=jsonpatch tests.py
14+
python -Wd property_tests.py
1415
coverage report --show-missing
1516

1617
coverage:

0 commit comments

Comments
 (0)