Skip to content

Commit af5f3e4

Browse files
author
David Ledvinka
committed
Add differential privacy theory and Gaussian rdo programs
1 parent 1ef24d6 commit af5f3e4

11 files changed

Lines changed: 1374 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 109 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,109 @@
1-
# Random-do notation
1+
# Random-do notation
2+
3+
RandomDo writes probabilistic algorithms in `rdo`. The same program can be interpreted as a
4+
mathematical measure for proofs or instantiated with an executable sampling backend.
5+
6+
## Differential privacy
7+
8+
Import `RandomDo` (or `RandomDo.DifferentialPrivacy`) and open `DifferentialPrivacy`.
9+
10+
### The Gaussian `rdo` samplers
11+
12+
The algorithms are in [`RandomDo/DifferentialPrivacy/Programs.lean`](RandomDo/DifferentialPrivacy/Programs.lean):
13+
14+
- `gaussianProgram center σ` draws zero-mean Gaussian noise and returns `center + noise`.
15+
- `gaussianCoordinates center σ` does this independently for each coordinate.
16+
- `gaussianMeanRealProgram f D c n xs` computes the scalar empirical mean and adds noise.
17+
- `gaussianMeanProgram f D c n xs` computes the vector empirical mean and adds independent noise
18+
in each coordinate.
19+
20+
The full scalar mean algorithm is:
21+
22+
```lean
23+
def gaussianMeanRealProgram {α : Type*} {R V : Type} [Scalar R] [Scalar V]
24+
[HasGaussian m R V R] (f : α → R) (D c : V) (n : ℕ) (xs : Dataset α) : m R := rdo
25+
let center := meanValue f xs
26+
let σ := c * D / (n : V)
27+
let z ← gaussianProgram (m := m) center σ
28+
return z
29+
```
30+
31+
Here `m` is an arbitrary `MeasurableSpaceMonad`. `meanValue` is pure arithmetic, and
32+
`gaussianProgram` samples through `HasGaussian` in `rdo`. The vector version uses the same
33+
computation coordinatewise. There is one implementation of each algorithm.
34+
35+
`D` is a public upper bound on the domain's ℓ₂ diameter, `n > 0` is a public lower bound on
36+
sample size, and `c` is a public noise multiplier. The standard deviation is `c * D / n`;
37+
`HasGaussian` takes **variance**, so the sampler passes its square.
38+
39+
At `m := Measure`, `R := ℝ`, and `V := ℝ≥0`, the programs denote exact real distributions.
40+
The theorems `gaussianMeanRealProgram_eq_mechanism` and `gaussianMeanProgram_eq_mechanism`
41+
identify these complete programs with the Gaussian laws used in the privacy proofs.
42+
Choosing another interpretation runs these same definitions. The existing floating-point
43+
backend is exercised in `Test.DifferentialPrivacy`; its arithmetic and generator are not part
44+
of the exact-real privacy guarantee.
45+
46+
### Privacy of the programs
47+
48+
The following proves privacy of the actual vector `rdo` sampler under adding or removing one
49+
record. `Mechanism.ofMeasure` only pairs the program with its proof of total probability one.
50+
51+
```lean
52+
import RandomDo
53+
54+
open DifferentialPrivacy MeasureTheory
55+
open scoped NNReal
56+
57+
example {α : Type*} {d : ℕ} (f : α → EuclideanSpace ℝ (Fin d))
58+
{D ε δ : ℝ≥0} (hD : HasDiameter f D)
59+
(hε : 0 < ε) (hδ : 0 < δ) (hδ1 : δ ≤ 1) :
60+
ApproxDP (Dataset.AtLeast 10 Dataset.AddRemove) ε δ
61+
(Mechanism.ofMeasure
62+
(gaussianMeanProgram (m := Measure) (fun a ↦ ⇑(f a)) D
63+
(gaussianNoiseMultiplier ε δ) 10) (fun _ ↦ inferInstance)) :=
64+
gaussianMeanProgram_approxDP_addRemove hD (by norm_num) hε hδ hδ1
65+
```
66+
67+
`gaussianMeanProgram_approxDP_replaceOne` covers replacement. The corresponding scalar theorems
68+
are prefixed `gaussianMeanRealProgram_approxDP`. Zero diameter is supported and requires no noise.
69+
70+
For `ε > 0` and `0 < δ ≤ 1`, `gaussianNoiseMultiplier ε δ` supplies the multiplier:
71+
72+
```text
73+
t = sqrt(2 log(1/δ))
74+
c = (t + sqrt(t² + 2ε)) / (2ε)
75+
σ = c * D / n
76+
```
77+
78+
This sufficient calibration follows from a Gaussian Chernoff bound and works for every positive
79+
`ε`. The proof establishes the joint density ratio, a dimension-independent directional tail
80+
bound, and the DP event inequality. See
81+
[Dwork and Roth, Appendix A](https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf) for the
82+
Gaussian privacy-loss argument; this calibration solves the Chernoff inequality directly.
83+
84+
### Theory and assumptions
85+
86+
- `Dataset α` is `List α`: finite sequences of arbitrary length, allowing repeated records.
87+
- `Dataset.AddRemove` inserts or removes one record at any position; `Dataset.ReplaceOne`
88+
replaces one record while preserving length.
89+
- `Mechanism X Y` maps inputs to `ProbabilityMeasure Y`.
90+
- `PureDP adj ε M` means `P(M x ∈ A) ≤ exp(ε) P(M x' ∈ A)` for adjacent inputs and
91+
measurable output events.
92+
- `ApproxDP adj ε δ M` adds `δ` to the right-hand side. Nonconstant Gaussian releases generally
93+
require this approximate notion.
94+
95+
The theory includes measurable deterministic postprocessing, restriction of adjacency,
96+
adjacency-preserving input transformations, and monotonicity in privacy budgets.
97+
98+
`HasDiameter f D` bounds all pairs of records in the public domain. For vectors, use an embedding
99+
into `EuclideanSpace ℝ (Fin d)` to express the ℓ₂ norm. A bound on each coordinate alone is not an
100+
ℓ₂ diameter bound. If both datasets have at least `n > 0` records, the empirical mean has
101+
sensitivity at most `D / n` under either adjacency relation. Inserting into a nonempty dataset
102+
has the sharper bound `D / (length + 1)`.
103+
104+
The programs accept their parameters directly; the privacy theorems state the conditions on
105+
those parameters and the inputs. `n` and `D` must be public. Choosing the variance from the
106+
private dataset size or its observed diameter is not justified by these theorems. The real mean
107+
of an empty list is defined as zero, but the mean privacy theorems require nonempty datasets.
108+
109+
Run the library, proofs, and execution checks with `lake build RandomDo Test`.

‎RandomDo.lean‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
module -- shake: keep-all --deprecated_module: ignore
22

3+
public import RandomDo.DifferentialPrivacy
34
public import RandomDo.ForMathlib.MeasureTheory.MeasurableSpace.Embedding
45
public import RandomDo.Measurable
56
public import RandomDo.Monad.ForInInstances

‎RandomDo/DifferentialPrivacy.lean‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
/-
2+
Copyright (c) 2026 David Ledvinka. All rights reserved.
3+
Released under Apache 2.0 license as described in the file LICENSE.
4+
Authors: David Ledvinka
5+
-/
6+
module
7+
8+
public import RandomDo.DifferentialPrivacy.Basic
9+
public import RandomDo.DifferentialPrivacy.Dataset
10+
public import RandomDo.DifferentialPrivacy.Mean
11+
public import RandomDo.DifferentialPrivacy.Gaussian
12+
public import RandomDo.DifferentialPrivacy.Calibration
13+
public import RandomDo.DifferentialPrivacy.Programs
Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
/-
2+
Copyright (c) 2026 David Ledvinka. All rights reserved.
3+
Released under Apache 2.0 license as described in the file LICENSE.
4+
Authors: David Ledvinka
5+
-/
6+
module
7+
8+
public import Mathlib.MeasureTheory.Measure.ProbabilityMeasure
9+
public import Mathlib.Analysis.SpecialFunctions.Exp
10+
11+
/-!
12+
# Differential privacy
13+
14+
A `Mechanism X Y` assigns a probability distribution on `Y` to each input in `X`.
15+
There is no measurable-space requirement on inputs: privacy compares individual inputs.
16+
`ApproxDP` quantifies over measurable output events and an explicit adjacency relation.
17+
`PureDP` is its specialization to `δ = 0`.
18+
19+
Privacy parameters are nonnegative reals. Allowing `δ ≥ 1` is convenient for bounds, although
20+
only `δ < 1` is informative. No finite-precision sampler is identified with an exact real law.
21+
-/
22+
23+
@[expose] public section
24+
25+
open MeasureTheory
26+
open scoped ENNReal NNReal
27+
28+
namespace DifferentialPrivacy
29+
30+
universe u v w
31+
32+
/-- A probabilistic algorithm, specified by its output law at each input. -/
33+
abbrev Mechanism (X : Type u) (Y : Type v) [MeasurableSpace Y] := X → ProbabilityMeasure Y
34+
35+
namespace Mechanism
36+
37+
variable {X : Type u} {Y : Type v} {Z : Type w}
38+
[MeasurableSpace Y] [MeasurableSpace Z]
39+
40+
/-- Package a measure-valued algorithm with proofs that all its laws have mass one. -/
41+
def ofMeasure (M : X → Measure Y) (hM : ∀ x, IsProbabilityMeasure (M x)) : Mechanism X Y :=
42+
fun x ↦ ⟨M x, hM x⟩
43+
44+
/-- Measurable deterministic postprocessing of a mechanism. -/
45+
noncomputable def postprocess (M : Mechanism X Y) (f : Y → Z) (hf : Measurable f) :
46+
Mechanism X Z := fun x ↦ (M x).map hf.aemeasurable
47+
48+
end Mechanism
49+
50+
variable {X : Type u} {Y : Type v} {Z : Type w}
51+
[MeasurableSpace Y] [MeasurableSpace Z]
52+
53+
/-- The directed event inequality underlying approximate differential privacy. Symmetry is
54+
supplied by a symmetric adjacency relation when using `ApproxDP`. -/
55+
def DistributionLE (ε δ : ℝ≥0) (μ ν : Measure Y) : Prop :=
56+
∀ A, MeasurableSet A → μ A ≤ ENNReal.ofReal (Real.exp ε) * ν A + δ
57+
58+
/-- `(ε, δ)` differential privacy relative to `adj`. -/
59+
def ApproxDP (adj : X → X → Prop) (ε δ : ℝ≥0) (M : Mechanism X Y) : Prop :=
60+
∀ ⦃x x'⦄, adj x x' → DistributionLE ε δ (M x : Measure Y) (M x' : Measure Y)
61+
62+
/-- Pure `ε` differential privacy: `P(M x ∈ A) ≤ exp ε * P(M x' ∈ A)`. -/
63+
def PureDP (adj : X → X → Prop) (ε : ℝ≥0) (M : Mechanism X Y) : Prop :=
64+
ApproxDP adj ε 0 M
65+
66+
namespace DistributionLE
67+
68+
variable {ε ε' δ δ' : ℝ≥0} {μ ν : Measure Y}
69+
70+
theorem refl (μ : Measure Y) : DistributionLE 0 0 μ μ := by
71+
intro A _
72+
simp
73+
74+
/-- A larger privacy budget preserves the event inequality. -/
75+
theorem mono (h : DistributionLE ε δ μ ν) (hε : ε ≤ ε') (hδ : δ ≤ δ') :
76+
DistributionLE ε' δ' μ ν := by
77+
intro A hA
78+
exact (h A hA).trans (add_le_add
79+
(mul_le_mul' (ENNReal.ofReal_le_ofReal (Real.exp_le_exp.mpr hε)) le_rfl)
80+
(ENNReal.coe_le_coe.mpr hδ))
81+
82+
theorem map (h : DistributionLE ε δ μ ν) {f : Y → Z} (hf : Measurable f) :
83+
DistributionLE ε δ (μ.map f) (ν.map f) := by
84+
intro A hA
85+
simpa only [Measure.map_apply hf hA] using h (f ⁻¹' A) (hf hA)
86+
87+
/-- It suffices to dominate the measure on a good event and bound its complement. -/
88+
theorem of_goodSet {G : Set Y} (hG : MeasurableSet G)
89+
(hgood : ∀ A, MeasurableSet A → μ (A ∩ G) ≤ ENNReal.ofReal (Real.exp ε) * ν A)
90+
(hbad : μ Gᶜ ≤ δ) : DistributionLE ε δ μ ν := by
91+
intro A hA
92+
calc
93+
μ A = μ (A ∩ G) + μ (A \ G) := (measure_inter_add_sdiff A hG).symm
94+
_ ≤ ENNReal.ofReal (Real.exp ε) * ν A + δ :=
95+
add_le_add (hgood A hA) ((measure_mono (Set.sdiff_subset_compl A G)).trans hbad)
96+
97+
end DistributionLE
98+
99+
namespace ApproxDP
100+
101+
variable {adj adj' : X → X → Prop} {ε ε' δ δ' : ℝ≥0} {M : Mechanism X Y}
102+
103+
theorem mono (h : ApproxDP adj ε δ M) (hε : ε ≤ ε') (hδ : δ ≤ δ') :
104+
ApproxDP adj ε' δ' M := fun _ _ hxx' ↦ (h hxx').mono hε hδ
105+
106+
/-- Privacy for a relation also holds for any subrelation. -/
107+
theorem of_subrelation (h : ApproxDP adj ε δ M) (hadj : ∀ x x', adj' x x' → adj x x') :
108+
ApproxDP adj' ε δ M := fun _ _ hxx' ↦ h (hadj _ _ hxx')
109+
110+
theorem postprocess (h : ApproxDP adj ε δ M) {f : Y → Z} (hf : Measurable f) :
111+
ApproxDP adj ε δ (M.postprocess f hf) := fun _ _ hxx' ↦ (h hxx').map hf
112+
113+
/-- Preprocessing is allowed when it preserves adjacency. In particular this restricts a
114+
mechanism to a public input domain, such as datasets of size at least `n`. -/
115+
theorem precompose {W : Type*} {rel : W → W → Prop} (h : ApproxDP adj ε δ M)
116+
(f : W → X) (hf : ∀ x x', rel x x' → adj (f x) (f x')) :
117+
ApproxDP rel ε δ (M ∘ f) := fun _ _ hxx' ↦ h (hf _ _ hxx')
118+
119+
theorem const (adj : X → X → Prop) (μ : ProbabilityMeasure Y) :
120+
ApproxDP adj 0 0 (fun _ : X ↦ μ) := fun _ _ _ ↦ DistributionLE.refl (μ : Measure Y)
121+
122+
end ApproxDP
123+
124+
theorem pureDP_iff {adj : X → X → Prop} {ε : ℝ≥0} {M : Mechanism X Y} :
125+
PureDP adj ε M ↔ ∀ ⦃x x'⦄, adj x x' → ∀ A, MeasurableSet A →
126+
(M x : Measure Y) A ≤ ENNReal.ofReal (Real.exp ε) * (M x' : Measure Y) A := by
127+
simp [PureDP, ApproxDP, DistributionLE]
128+
129+
end DifferentialPrivacy

0 commit comments

Comments
 (0)