atom.xml

<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.2.2">Jekyll</generator><link href="https://programelot.github.io/atom.xml" rel="self" type="application/atom+xml" /><link href="https://programelot.github.io/" rel="alternate" type="text/html" /><updated>2026-06-28T01:01:00-05:00</updated><id>https://programelot.github.io/atom.xml</id><title type="html">REAL</title><subtitle>I am Programelot who is researching about optimization.</subtitle><author><name>Programelot</name></author><entry><title type="html">Efficient Maximum Flow Algorithms and Bipartite Matching</title><link href="https://programelot.github.io/algorithms/2026/06/03/bipartite-matching/" rel="alternate" type="text/html" title="Efficient Maximum Flow Algorithms and Bipartite Matching" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/bipartite-matching</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/bipartite-matching/"><![CDATA[<h1 id="improving-ford-fulkerson">Improving Ford-Fulkerson</h1>

<h2 id="a-bad-example">A Bad Example</h2>

<p>Consider the following network:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>s ──100── x
│         │
1         1
│         │
s ──100── y ──100── t
</code></pre></div></div>

<p>If the algorithm alternates between $s$-$x$-$y$-$t$ (bottleneck 1) and $s$-$y$-$x$-$t$ (bottleneck 1), up to 200 iterations may be needed, confirming the $O(mC)$ pseudo-polynomial bound is tight.</p>

<hr />

<h2 id="capacity-scaling">Capacity Scaling</h2>

<p><strong>Idea:</strong> Prioritize paths with larger residual capacities.</p>

<blockquote>
  <p><strong>Definition 1.</strong> $G_f(\Delta)$ = the subgraph of $G_f$ containing only edges with residual capacity $\geq \Delta$.</p>
</blockquote>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>f(e) ← 0  for all e
Δ ← largest power of 2 ≤ max capacity of edges leaving s
while Δ ≥ 1 do
    while Gf(Δ) has an s-t path do
        P ← simple s-t path in Gf(Δ)
        f ← Augment(f, P)
    Δ ← Δ/2
return f
</code></pre></div></div>

<p>This is a variant of Ford-Fulkerson with a specific path-selection rule, so correctness is immediate.</p>

<p><strong>Lemma 1.</strong> The algorithm performs at most $2m\lceil 1 + \log_2 C \rceil$ augmentations.</p>

<p><strong>Theorem 1.</strong> Time complexity: $O(m^2 \log_2 C)$ — <strong>weakly polynomial</strong>.</p>

<hr />

<h2 id="edmonds-karp-algorithm">Edmonds-Karp Algorithm</h2>

<p>Select augmenting paths with the <strong>fewest edges</strong> (BFS shortest path).</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>f(e) ← 0  for all e
while Gf has an s-t path do
    P ← s-t path with minimum number of edges in Gf (BFS)
    f ← Augment(f, P)
return f
</code></pre></div></div>

<p><strong>Lemma 2.</strong> The Edmonds-Karp algorithm performs $O(mn)$ augmentations.</p>

<p><strong>Theorem 2.</strong> Time complexity: $O(m^2 n)$ — <strong>strongly polynomial</strong> (bounded only by the number of input entries, not their values).</p>

<blockquote>
  <p><strong>Note:</strong> Orlin (2012) gave an $O(mn)$-time algorithm, which is optimal but beyond our scope.</p>
</blockquote>

<hr />

<h1 id="bipartite-matching">Bipartite Matching</h1>

<h2 id="problem-definition">Problem Definition</h2>

<blockquote>
  <p><strong>Definition 2.</strong> A graph $G = (V, E)$ is <strong>bipartite</strong> if $V = X \cup Y$ ($X \cap Y = \emptyset$) and every edge has one endpoint in $X$ and the other in $Y$.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 3.</strong> A <strong>matching</strong> $M \subseteq E$: every vertex appears in $M$ at most once.</p>
</blockquote>

<blockquote>
  <p><strong>Problem (Maximum Bipartite Matching).</strong> Find a matching of maximum cardinality in bipartite graph $G = (X \cup Y, E)$.</p>
</blockquote>

<hr />

<h2 id="reduction-to-maximum-flow">Reduction to Maximum Flow</h2>

<ol>
  <li>Orient all edges in $E$ from $X$ to $Y$ with capacity $\infty$.</li>
  <li>Add source $s$ with edges $\langle s, x \rangle$ of capacity 1 for all $x \in X$.</li>
  <li>Add sink $t$ with edges $\langle y, t \rangle$ of capacity 1 for all $y \in Y$.</li>
  <li>Compute maximum integer flow $f^*$.</li>
  <li>Output $M = \lbrace (x,y) : f^*(x,y) \neq 0\rbrace$.</li>
</ol>

<hr />

<h2 id="correctness-proof">Correctness Proof</h2>

<p><strong>Lemma 3.</strong> If $(x,y) \in E$ and $f^<em>(x,y) \neq 0$, then $f^</em>(x,y) = 1$.</p>

<p><strong>Proof.</strong> The only incoming edge of $x$ is $\langle s,x \rangle$ with capacity 1, so $f^<em>(x,y) \leq 1$ by flow conservation. Since $f^</em>$ is integer, $f^*(x,y) = 1$. ∎</p>

<p><strong>Lemma 4.</strong> $\lvert M\rvert \geq \text{OPT}$.</p>

<p><strong>Proof.</strong> Fix maximum matching $O$. For each $(x,y) \in O$, send unit flow along $s$-$x$-$y$-$t$. Since $O$ is a matching, each edge is used at most once → valid flow of value $\lvert O\rvert$. Therefore $v(f^<em>) \geq \lvert O\rvert = \text{OPT}$. With cut $S := \lbrace s\rbrace \cup X$, $v(f^</em>) = \lvert M\rvert$ (by Lemma 3, no edges enter $S$). ∎</p>

<p><strong>Lemma 5.</strong> Each $X$ node is incident to at most one edge of $M$.</p>

<p><strong>Proof.</strong> If $x \in X$ had $\geq 2$ edges in $M$, then $f^{<em>out}(x) \geq 2$ (by Lemma 3). But $c(s,x) = 1$ means $f^{</em>in}(x) \leq 1$, contradicting flow conservation. ∎</p>

<p><strong>Lemma 6.</strong> Each $Y$ node is incident to at most one edge of $M$. (Same argument as Lemma 5.)</p>

<p><strong>Theorem 3.</strong> The algorithm returns a maximum bipartite matching. (Follows from Lemmas 4, 5, and 6.)</p>

<hr />

<h2 id="time-complexity">Time Complexity</h2>

<p><strong>Theorem 4.</strong> The algorithm runs in $O((m+n)n)$ time.</p>

<p><strong>Proof.</strong> The constructed network has $n+2$ vertices and $m+n$ edges. Ford-Fulkerson computes integer flow in $O(m’ \cdot \lvert X\rvert) = O((m+n)n)$ time. ∎</p>

<hr />

<h2 id="alternating-paths">Alternating Paths</h2>

<p>Viewed from the bipartite matching perspective, each Ford-Fulkerson iteration finds an <strong>alternating path</strong>:</p>

<ul>
  <li>Alternates between $X$ and $Y$, using matched ($f \neq 0$) and unmatched edges alternately.</li>
  <li>First and last edges are both unmatched.</li>
  <li>Flipping matched/unmatched status along the path increases the matching size by 1.</li>
</ul>

<p>An algorithm can directly search for alternating paths without explicitly constructing the flow network.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="bipartite-matching" /><category term="network-flow" /><category term="edmonds-karp" /><category term="capacity-scaling" /><category term="alternating-path" /><summary type="html"><![CDATA[Improving Ford-Fulkerson]]></summary></entry><entry><title type="html">Maximum Subarray Sum and Closest Pair of Points</title><link href="https://programelot.github.io/algorithms/2026/06/03/maximum-subarray-and-closest-pair/" rel="alternate" type="text/html" title="Maximum Subarray Sum and Closest Pair of Points" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/maximum-subarray-and-closest-pair</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/maximum-subarray-and-closest-pair/"><![CDATA[<h1 id="maximum-subarray-sum">Maximum Subarray Sum</h1>

<h2 id="problem-definition">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 1.</strong> Given a sequence of integers $a_1, \ldots, a_n$, find the maximum sum of a contiguous subsequence. The empty set is also a valid subsequence, with sum 0.</p>
</blockquote>

<p><strong>Example:</strong> For input <code class="language-plaintext highlighter-rouge">0 100 -50 -60 30 -10 40 5 45 -2 1</code>, the answer is $110 = 30 + (-10) + 40 + 5 + 45$.</p>

<hr />

<h2 id="on3-algorithm">$O(n^3)$ Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sol ← 0
for each i, j such that 1 ≤ i ≤ j ≤ n do
    if Σ(k=i to j) ak &gt; sol then
        sol ← Σ(k=i to j) ak
return sol
</code></pre></div></div>

<p><strong>Theorem 1.</strong> This algorithm runs in $O(n^3)$ time.</p>

<p><strong>Proof.</strong> The number of basic operations performed is proportional to:</p>

\[\sum_{i=1}^{n} \sum_{j=i}^{n} (j - i + 1) = \frac{n(n+1)(n+2)}{6} = O(n^3)\]

<hr />

<h2 id="on2-algorithm-eliminating-redundant-computation">$O(n^2)$ Algorithm (Eliminating Redundant Computation)</h2>

<p><strong>Key idea:</strong> Reuse $a_2+a_3+a_4+a_5$ computed for $i=2, j=5$ when computing $i=2, j=6$.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sol ← 0
for i ← 1, ..., n do
    sum(i, i-1) ← 0
    for j ← i, ..., n do
        sum(i,j) ← sum(i,j-1) + aj
        if sum(i,j) &gt; sol then
            sol ← sum(i,j)
return sol
</code></pre></div></div>

<p><strong>Theorem 2.</strong> This algorithm runs in $O(n^2)$ time.</p>

<p><strong>Theorem 3 (Correctness).</strong> We prove $\text{sum}<em>{i,j} = \sum</em>{k=i}^{j} a_k$ by induction on $j-i$.</p>

<ul>
  <li><strong>Base case:</strong> If $j - i = 0$, then $\text{sum}<em>{i,i} = \text{sum}</em>{i,i-1} + a_i = \sum_{k=i}^{i} a_k$.</li>
  <li><strong>Inductive step:</strong> For $j - i = d_0 + 1$, $\text{sum}<em>{i,j} = \text{sum}</em>{i,j-1} + a_j = \sum_{k=i}^{j-1} a_k + a_j = \sum_{k=i}^{j} a_k$.</li>
</ul>

<hr />

<h2 id="on-log-n-algorithm-divide-and-conquer">$O(n \log n)$ Algorithm (Divide and Conquer)</h2>

<p><strong>Key idea:</strong> Like mergesort, split the sequence left and right, then separately compute the maximum subarray that crosses the boundary.</p>

<p>The optimal subsequence must be one of:</p>
<ol>
  <li>Entirely in the left half</li>
  <li>Entirely in the right half</li>
  <li>Containing both $a_m$ and $a_{m+1}$ at the boundary</li>
</ol>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function MaxSumSubseq(s, t):
    if s = t: return max(as, 0)
    m ← ⌊(s+t)/2⌋
    soll ← MaxSumSubseq(s, m)
    solr ← MaxSumSubseq(m+1, t)
    subsoll ← (max subarray sum going left from m)
    subsolr ← (max subarray sum going right from m+1)
    return max(soll, solr, subsoll + subsolr)
</code></pre></div></div>

<p><strong>Observation 1.</strong> For $m &gt; 1$: $\text{subsol}^l_m = \max(\text{subsol}^l_{m-1} + a_m,\ 0)$</p>

<p><strong>Proof.</strong> When $\text{subsol}^l_m$ is nonzero, the contiguous subsequence achieving it is the maximum subsequence ending at $a_{m-1}$, extended by $a_m$.</p>

<hr />

<h2 id="on-algorithm">$O(n)$ Algorithm</h2>

<p>We ask “where does the optimal subsequence <strong>end</strong>?” Without computing $\text{subsolr}$, we only track the maximum subarray sum ending at each position $m$.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>MaxEndingAt1 = max(a1, 0)
for m = 2, ..., n do
    MaxEndingAtm = max(MaxEndingAt(m-1) + am, 0)
sol ← 0
for m = 1, ..., n do
    if MaxEndingAtm &gt; sol then sol ← MaxEndingAtm
return sol
</code></pre></div></div>

<hr />

<h2 id="lower-bound">Lower Bound</h2>

<p><strong>Theorem 4.</strong> There is no deterministic algorithm that solves the maximum subarray problem in sublinear time.</p>

<p><strong>Proof sketch.</strong> If such an algorithm existed, for sufficiently large $n_0$ there would be an input where the value at some position $k$ is never read.</p>
<ul>
  <li>If every maximum-sum subsequence contains $a_k$: replacing $a_k$ with $-\infty$ yields a wrong answer.</li>
  <li>Otherwise: replacing $a_k$ with $+\infty$ yields a wrong answer.</li>
</ul>

<p>Therefore no sublinear algorithm can exist. ∎</p>

<hr />

<h2 id="definition-of-efficient-algorithm">Definition of Efficient Algorithm</h2>

<blockquote>
  <p><strong>Definition 1.</strong> An algorithm is <strong>efficient</strong> if its running time is bounded by a polynomial in the input size.</p>
</blockquote>

<p>The maximum subarray problem is easy under this criterion. However, some problems are believed to have no efficient algorithms; the theory that studies this is <strong>NP-completeness</strong>.</p>

<hr />

<h1 id="closest-pair-of-points">Closest Pair of Points</h1>

<h2 id="problem-definition-1">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 2.</strong> Given $n$ points $P = \lbrace p_1, \ldots, p_n\rbrace$ in the 2D Euclidean plane, find the closest pair of points.</p>
</blockquote>

<p>$(x_i, y_i)$ are the coordinates of $p_i$, and $d(p_i, p_j)$ is the Euclidean distance between two points.</p>

<ul>
  <li>Naive algorithm: $O(n^2)$</li>
  <li>1D case: $O(n \log n)$ by sorting then scanning</li>
  <li>Goal: Design an algorithm faster than $O(n^2)$ in 2D</li>
</ul>

<hr />

<h2 id="key-idea">Key Idea</h2>

<p>Let $\delta$ be the minimum distance found so far:</p>

<ol>
  <li>Points farther than $\delta$ from the boundary need not be considered.</li>
  <li>Only check points inside the <strong>center band</strong> of width $2\delta$.</li>
  <li>If points inside the band are sorted by $y$-coordinate, any pair closer than $\delta$ must be within <strong>11 positions</strong> of each other.</li>
</ol>

<hr />

<h2 id="intermediate-algorithm">Intermediate Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function ClosestPair(Px):
    if |Px| ≤ 3: enumerate
    Lx ← first ⌊|Px|/2⌋ points,  Rx ← rest
    x* ← max x-coordinate in Lx
    (pl1, pl2) ← ClosestPair(Lx)
    (pr1, pr2) ← ClosestPair(Rx)
    δ ← min{d(pl1,pl2), d(pr1,pr2)}
    S ← points with x-coordinate in (x*-δ, x*+δ)
    Sy ← S sorted by y-coordinate
    for each p ∈ Sy:
        compute distances to next 11 points in Sy, update closest (pm1, pm2)
    return minimum of three results
</code></pre></div></div>

<p>If $S_y$ is sorted via mergesort in each call:</p>

\[T(n) \leq 2T(n/2) + c_2 n \log n \implies T(n) = O(n \log^2 n)\]

<hr />

<h2 id="master-theorem">Master Theorem</h2>

<blockquote>
  <p><strong>Theorem 5 (Master Theorem).</strong> For constants $a \geq 1$, $b &gt; 1$ and $T(n) = aT(n/b) + f(n)$:</p>

  <ol>
    <li>$f(n) = O(n^{(\log_b a) - \epsilon})$ $\implies T(n) = \Theta(n^{\log_b a})$</li>
    <li>$f(n) = \Theta(n^{\log_b a} \log^k n)$ $\implies T(n) = \Theta(n^{\log_b a} \log^{k+1} n)$</li>
    <li>$f(n) = \Omega(n^{(\log_b a) + \epsilon})$ and $a \cdot f(n/b) \leq cf(n)$ $\implies T(n) = \Theta(f(n))$</li>
  </ol>
</blockquote>

<p><strong>Reducing to $O(n \log n)$:</strong> Precompute $P_y$ (all points sorted by $y$-coordinate) before recursion and pass it along. In each recursive call, scan $P_y$ to build $S_y$ in $O(n)$.</p>

\[T(n) = 2T(n/2) + \Theta(n) \implies T(n) = O(n \log n)\]

<hr />

<h2 id="final-algorithm">Final Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sort P by x-coords → Px
let Py = P sorted by y-coords
return ClosestPair(Px, Py)

function ClosestPair(Px, Py):
    if |Px| ≤ 3: enumerate all pairs
    Lx ← first ⌊|Px|/2⌋ points,  Rx ← rest
    Ly, Ry ← lists of Lx, Rx points sorted by y-coordinate
    x* ← max x-coordinate in Lx
    (pl1, pl2) ← ClosestPair(Lx, Ly)
    (pr1, pr2) ← ClosestPair(Rx, Ry)
    δ ← min{d(pl1,pl2), d(pr1,pr2)}
    S ← points with x-coordinate in (x*-δ, x*+δ)
    Sy ← extract points of S by scanning Py (O(n))
    for each p ∈ Sy:
        compute distances to next 11 points in Sy, update closest (pm1, pm2)
    return minimum of three results
</code></pre></div></div>

<hr />

<h2 id="correctness-proof">Correctness Proof</h2>

<h3 id="lemma-1">Lemma 1</h3>

<p>If $p_i \in L_x$, $p_j \in R_x$, and $d(p_i, p_j) &lt; \delta$, then $x_i, x_j \in (x^* - \delta,\ x^* + \delta)$.</p>

<p><strong>Proof.</strong> Since $x_i \leq x^* \leq x_j$:</p>

<p>\(x^* - x_i \leq x_j - x_i \leq d(p_i, p_j) &lt; \delta\)
\(x_j - x^* \leq x_j - x_i \leq d(p_i, p_j) &lt; \delta \qquad \square\)</p>

<h3 id="lemma-2">Lemma 2</h3>

<p>If $p, q \in S_y$ and $d(p,q) &lt; \delta$, then $p$ and $q$ are within 11 positions of each other in $S_y$.</p>

<p><strong>Proof.</strong> Partition the strip into <strong>boxes</strong> of side length $\delta/2$. Each row consists of 4 boxes across. Each box contains <strong>at most 1 point</strong> (two points in the same half cannot be closer than $\delta$). If $p$ and $q$ are 12 or more positions apart, by the pigeonhole principle they cannot both fit within the strip at distance $&lt; \delta$, giving a contradiction. ∎</p>

<h3 id="theorem-6">Theorem 6</h3>

<p>The algorithm correctly finds the closest pair of points in $P$.</p>

<p><strong>Proof.</strong> By induction on $\lvert P_x\rvert$. The case $\lvert P_x\rvert \leq 3$ is trivial. For $\lvert P_x\rvert \geq 4$, by the inductive hypothesis $(p_{l1}, p_{l2})$ and $(p_{r1}, p_{r2})$ are the closest pairs in each half.</p>

<p><strong>Case 1.</strong> $\delta &gt; d(p_{o1}, p_{o2})$: The two closest points must be in different halves. By Lemmas 1 and 2, both are in $S_y$ within 11 positions, so the algorithm finds them.</p>

<p><strong>Case 2.</strong> $\delta = d(p_{o1}, p_{o2})$: The minimum of the three results equals $\delta$, the closest pair distance. ∎</p>

<hr />

<h2 id="time-complexity">Time Complexity</h2>

<p><strong>Theorem 7.</strong> The algorithm runs in $O(n \log n)$ time.</p>

<p><strong>Proof.</strong> Separating $L_y, R_y$ and constructing $S_y$ each take $O(n)$ by scanning $P_y$ once. Therefore $T(n) = 2T(n/2) + \Theta(n)$, which by the Master Theorem gives $T(n) = O(n \log n)$. Preprocessing (sorting) also takes $O(n \log n)$. ∎</p>

<p><strong>Bottom-up approach:</strong> Since the structure mirrors mergesort, one can include the $y$-sorted list in the return tuple of each recursive call and build it during the merge step.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="divide-and-conquer" /><category term="maximum-subarray" /><category term="closest-pair" /><category term="kadane" /><category term="master-theorem" /><summary type="html"><![CDATA[Maximum Subarray Sum]]></summary></entry><entry><title type="html">NP-Completeness: Nondeterminism and 3-Dimensional Matching</title><link href="https://programelot.github.io/algorithms/2026/06/03/np-completeness-3dm/" rel="alternate" type="text/html" title="NP-Completeness: Nondeterminism and 3-Dimensional Matching" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/np-completeness-3dm</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/np-completeness-3dm/"><![CDATA[<h1 id="np-hard-and-np-complete">NP-Hard and NP-Complete</h1>

<h2 id="alternative-view-nondeterministic-programming">Alternative View: Nondeterministic Programming</h2>

<p>Consider extending C++ with the following construct:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>either {
    // statements 1
} or {
    // statements 2
}
</code></pre></div></div>

<p>When this executes, <strong>two parallel universes</strong> are created. If any one returns <code class="language-plaintext highlighter-rouge">yes</code>, the program outputs <code class="language-plaintext highlighter-rouge">yes</code>; otherwise <code class="language-plaintext highlighter-rouge">no</code>. Running time is the time of the longest-running universe.</p>

<blockquote>
  <p><strong>NP</strong> = the set of problems solvable in polynomial time using this nondeterministic C++.</p>
</blockquote>

<p>(NP stands for <strong>Nondeterministic Polynomial</strong>; this definition is equivalent to the certifier-based one.)</p>

<hr />

<h2 id="np-hard-and-np-complete-definitions">NP-Hard and NP-Complete Definitions</h2>

<blockquote>
  <p><strong>Definition 1.</strong> Problem $X$ is <strong>NP-hard</strong> if every $Y \in \text{NP}$ satisfies $Y \leq_P X$.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 2.</strong> Problem $X$ is <strong>NP-complete</strong> if $X \in \text{NP}$ and $X$ is NP-hard.</p>
</blockquote>

<p>NP-complete problems are the “hardest” problems within NP.</p>

<p><strong>Theorem 1.</strong> If $X$ is NP-complete, then $X$ can be solved in polynomial time if and only if $\text{P} = \text{NP}$.</p>

<p><strong>Proof.</strong></p>
<ul>
  <li>If $\text{P} = \text{NP}$: $X \in \text{NP} = \text{P}$, so a polynomial-time algorithm exists.</li>
  <li>If $X$ is solvable in polynomial time: for any $Y \in \text{NP}$, $Y \leq_P X$, so $Y$ is also solvable in polynomial time. ∎</li>
</ul>

<hr />

<h2 id="cook-levin-theorem">Cook-Levin Theorem</h2>

<blockquote>
  <p><strong>Theorem 2 (Cook-Levin).</strong> 3-SAT is NP-complete.</p>
</blockquote>

<p>This is the first NP-completeness proof and is the starting point for all subsequent ones.</p>

<p><strong>Recipe for proving NP-completeness of a new problem $X$:</strong></p>
<ol>
  <li>Show $X \in \text{NP}$. (Construct an efficient certifier)</li>
  <li>Show $Y \leq_P X$ for some already-known NP-complete problem $Y$.</li>
</ol>

<hr />

<h1 id="3-dimensional-matching-is-np-complete">3-Dimensional Matching is NP-Complete</h1>

<h2 id="problem-definition">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 3 (3-Dimensional Matching).</strong> Given three disjoint sets $X, Y, Z$ of equal size ($\lvert X\rvert=\lvert Y\rvert=\lvert Z\rvert=n$) and $T \subseteq X \times Y \times Z$: does $T$ contain $n$ triples that together include each element of $X \cup Y \cup Z$ exactly once?</p>
</blockquote>

<p><strong>Theorem 3.</strong> 3-Dimensional Matching is NP-complete.</p>

<hr />

<h2 id="part-1-in-np">Part 1: $\in$ NP</h2>

<p>For a yes-instance, present as hint a subset $T’ \subseteq T$ with $\lvert T’\rvert=n$. We verify in polynomial time that $\lvert T’\rvert=n$, $T’ \subseteq T$, and each element of $X \cup Y \cup Z$ appears exactly once.</p>

<hr />

<h2 id="part-2-3-sat-leq_p-3-dimensional-matching">Part 2: 3-SAT $\leq_P$ 3-Dimensional Matching</h2>

<p>From a 3-SAT instance with $n$ variables and $k$ clauses, construct three gadgets.</p>

<h3 id="variable-gadget">Variable Gadget</h3>

<p>For each variable $x_i$:</p>
<ul>
  <li>$A_i := \lbrace a_{i,1}, \ldots, a_{i,2k}\rbrace$, $B_i := \lbrace b_{i,1}, \ldots, b_{i,2k}\rbrace$</li>
  <li><strong>Variable triples</strong> $t_{i,\ell} = (a_{i,\ell},\ a_{i,\ell+1},\ b_{i,\ell})$ for $\ell = 1, \ldots, 2k$ (with $a_{i,2k+1} := a_{i,1}$)</li>
</ul>

<p>These are the <strong>only</strong> triples containing elements of $A_i$.</p>

<p><strong>Interpretation:</strong> If $x_i$ is true, select triples with odd $\ell$ → $b_{i,j}$ covered for odd $j$. If false, select even.</p>

<h3 id="clause-gadget">Clause Gadget</h3>

<p>For each clause $C_j$, define $P_j := \lbrace p_j, p’_j\rbrace$ and add three <strong>clause triples</strong>:</p>
<ul>
  <li>If $C_j$ contains $x_i$: add $(p_j, p’<em>j, b</em>{i,2j})$</li>
  <li>If $C_j$ contains $\bar{x}<em>i$: add $(p_j, p’_j, b</em>{i,2j-1})$</li>
</ul>

<p>The only triples containing $p_j$ or $p’_j$ are these three.</p>

<h3 id="cleanup-gadget">Cleanup Gadget</h3>

<p>$(n-1)k$ cleanup gadgets: for $m = 1, \ldots, (n-1)k$, define $Q_m := \lbrace q_m, q’<em>m\rbrace$ and add <strong>cleanup triples</strong> $(q_m, q’_m, b</em>{i,\ell})$ for all $i, \ell$.</p>

<hr />

<h2 id="set-partition">Set Partition</h2>

<p>\(X := \{a_{i,j}\ \mid\ j \text{ even}\} \cup \{p_j\} \cup \{q_m\}\)
\(Y := \{a_{i,j}\ \mid\ j \text{ odd}\} \cup \{p'_j\} \cup \{q'_m\}\)
\(Z := \{b_{i,j}\ \mid\ 1 \leq i \leq n,\ 1 \leq j \leq 2k\}\)</p>

<p>$\lvert X\rvert = \lvert Y\rvert = \lvert Z\rvert = 2nk$, $\lvert T\rvert = O(n^2k^2)$ — the construction is completed in polynomial time.</p>

<hr />

<h2 id="equivalence">Equivalence</h2>

<p><strong>3-SAT yes $\Rightarrow$ 3DM yes:</strong></p>
<ol>
  <li>Fix a satisfying assignment. If $x_i$ is true, select variable triples with odd $\ell$; if false, even.</li>
  <li>For each clause $C_j$, select the clause triple corresponding to a true literal (the corresponding $b_{i,\ell}$ is not yet covered).</li>
  <li>Cover remaining $B_i$ elements and $\cup_m Q_m$ using cleanup triples.</li>
</ol>

<p><strong>3DM yes $\Rightarrow$ 3-SAT yes:</strong></p>
<ol>
  <li>Elements of $A_i$ can only be covered by variable triples → the matching selects either all odd or all even $\ell$ triples.</li>
  <li>Assign $x_i$ = true if odd triples selected, false if even.</li>
  <li>For each clause $C_j$, the selected clause triple’s $b_{i,\ell}$ reveals the corresponding variable is true → the clause is satisfied. ∎</li>
</ol>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="NP-complete" /><category term="3DM" /><category term="reduction" /><category term="NP-hard" /><category term="cook-levin" /><category term="nondeterminism" /><summary type="html"><![CDATA[NP-Hard and NP-Complete]]></summary></entry><entry><title type="html">Dynamic Programming: RNA Secondary Structure and Bellman-Ford</title><link href="https://programelot.github.io/algorithms/2026/06/03/rna-secondary-structure-and-bellman-ford/" rel="alternate" type="text/html" title="Dynamic Programming: RNA Secondary Structure and Bellman-Ford" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/rna-secondary-structure-and-bellman-ford</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/rna-secondary-structure-and-bellman-ford/"><![CDATA[<h1 id="rna-secondary-structure-prediction">RNA Secondary Structure Prediction</h1>

<h2 id="problem-definition">Problem Definition</h2>

<blockquote>
  <p><strong>Definition 1.</strong> The <strong>primary structure</strong> of an RNA molecule $B = b_1 \ldots b_n$ is a sequence of symbols drawn from $\lbrace A, C, G, U\rbrace$.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 2.</strong> A <strong>secondary structure</strong> $S \subseteq \lbrace 1,\ldots,n\rbrace \times \lbrace 1,\ldots,n\rbrace$ over primary structure $B$ satisfies:</p>

  <ol>
    <li><strong>(No sharp turns)</strong> $(i,j) \in S \implies i &lt; j - 4$</li>
    <li><strong>(Complementary base pairs)</strong> $\lbrace b_i, b_j\rbrace = \lbrace A,U\rbrace$ or $\lbrace C,G\rbrace$</li>
    <li><strong>(Matching)</strong> Each index appears at most once</li>
    <li><strong>(Noncrossing)</strong> For $(i,j),(k,l) \in S$, $i &lt; k &lt; j &lt; l$ is forbidden</li>
  </ol>
</blockquote>

<blockquote>
  <p><strong>Problem 1.</strong> Given an RNA primary structure $B$, find the secondary structure that maximizes the number of complementary base pairs.</p>
</blockquote>

<hr />

<h2 id="subproblem-definition">Subproblem Definition</h2>

<p>Defining $\text{OPT}(j)$ as the maximum base pairs over the first $j$ bases fails: pairing $b_j$ with $b_k$ ($k &lt; j-4$) leaves a subproblem that is not a prefix. Instead, we use:</p>

<blockquote>
  <p><strong>Definition 3.</strong> $\text{OPT}(i,j)$ = the maximum secondary structure size over $b_i \ldots b_j$</p>
</blockquote>

<hr />

<h2 id="recurrence">Recurrence</h2>

<p><strong>Lemma 1.</strong> For $1 \leq i \leq j \leq n$:</p>

\[\text{OPT}(i,j) = \max \begin{cases} \text{OPT}(i, j-1) \\ \displaystyle\max_{\substack{k:\ i \leq k &lt; j-4 \\ b_j \text{ and } b_k \text{ complementary}}} \bigl[\text{OPT}(i, k-1) + \text{OPT}(k+1, j-1) + 1\bigr] \end{cases}\]

<p>where $\text{OPT}(i, i-1) := 0$.</p>

<p><strong>Proof.</strong></p>

<ul>
  <li>Case $j$ is unpaired: $\text{OPT}(i,j) = \text{OPT}(i, j-1)$.</li>
  <li>Case $(k, j) \in S$: $b_k$ and $b_j$ must be complementary and $i \leq k &lt; j-4$. By the <strong>noncrossing condition</strong>, the remaining structure decomposes into $b_i \ldots b_{k-1}$ and $b_{k+1} \ldots b_{j-1}$. Defining $\text{OPT}(i, i-1) = 0$ handles boundary cases $k = i$ and $k = j-1$. ∎</li>
</ul>

<hr />

<h2 id="evaluation-order-and-algorithm">Evaluation Order and Algorithm</h2>

<p>The right-hand side always has smaller size ($= j - i$) than the left-hand side, so fill the table in order of increasing size:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[i, i-1] ← 0  for all 1 ≤ i ≤ n
for ℓ ← 0, ..., n-1 do
    for i ← 1, ..., n-ℓ do
        j ← i + ℓ
        M[i,j] ← M[i, j-1]
        k[i,j] ← 0
        if M[i,j] &lt; max{ M[i,k-1]+M[k+1,j-1]+1 | i≤k&lt;j-4, bj,bk comp. } then
            M[i,j] ← (above maximum)
            k[i,j] ← corresponding k (argmax)
sol ← M[1,n]
OutputSecondaryStructure(1, n)

function OutputSecondaryStructure(i, j):
    if i ≤ j then
        if k[i,j] = 0 then
            OutputSecondaryStructure(i, j-1)
        else
            OutputSecondaryStructure(i, k[i,j]-1)
            print (k[i,j], j)
            OutputSecondaryStructure(k[i,j]+1, j-1)
</code></pre></div></div>

<p><strong>Theorem 1.</strong> The algorithm is correct ($M[i,j] = \text{OPT}(i,j)$ by induction on $\ell$) and runs in $O(n^3)$ time.</p>

<hr />

<h1 id="shortest-path-with-negative-edges-bellman-ford">Shortest Path with Negative Edges (Bellman-Ford)</h1>

<h2 id="problem-definition-1">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 2.</strong> Given a weighted directed graph $G = (V, A)$, a cost function $c: A \to \mathbb{Q}$, a source $s$, and a destination $t$, find the shortest path from $s$ to $t$.</p>
</blockquote>

<ul>
  <li><strong>Difference from Dijkstra:</strong> Negative-cost edges are allowed.</li>
  <li><strong>Assumption:</strong> No negative cycles (otherwise shortest path length is undefined).</li>
</ul>

<hr />

<h2 id="key-lemma">Key Lemma</h2>

<p><strong>Lemma 2.</strong> If $G$ has no negative cycles, there exists a simple path among the shortest $s$-$t$ paths, with at most $n-1$ edges.</p>

<p><strong>Proof.</strong> A non-simple path has a repeated vertex. Removing the cycle reduces cost by a nonnegative amount. Repeating this yields a simple path. ∎</p>

<hr />

<h2 id="subproblem-and-recurrence">Subproblem and Recurrence</h2>

<blockquote>
  <p><strong>Definition 4.</strong> $\text{OPT}(i, v)$ = the length of the shortest path from $v$ to $t$ using at most $i$ edges. $+\infty$ if no such path exists.</p>
</blockquote>

<p><strong>Boundary conditions:</strong> $\text{OPT}(0, t) = 0$, $\text{OPT}(0, v) = +\infty$ for $v \neq t$</p>

<p><strong>Lemma 3.</strong> For all $i \geq 1$:</p>

\[\text{OPT}(i, v) = \min \begin{cases} \text{OPT}(i-1, v) \\ \min_{\langle v,w \rangle \in A} \bigl[c(v,w) + \text{OPT}(i-1, w)\bigr] \end{cases}\]

<p><strong>Proof.</strong> Consider the shortest $v$-$t$ path using at most $i$ edges. If it uses $\leq i-1$ edges, the first case applies; if it uses the first edge $\langle v,w \rangle$, the second case applies. ∎</p>

<hr />

<h2 id="bellman-ford-algorithm">Bellman-Ford Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[0, t] ← 0
M[0, v] ← +∞  for all v ≠ t
for i ← 1, ..., n-1 do
    for each v ∈ V do
        M[i, v] ← min(
            M[i-1, v],
            min{ c(v,w) + M[i-1, w] | ⟨v,w⟩ ∈ A }
        )
return M[n-1, s]
</code></pre></div></div>

<p><strong>Theorem 2.</strong> The algorithm returns the shortest path length from $s$ to $t$.</p>

<p><strong>Theorem 3.</strong> Time complexity: $O(n(m+n))$ (outer loop $n-1$ iterations, each processing all edges $O(m+n)$).</p>

<hr />

<h2 id="space-optimized-version">Space-Optimized Version</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[t] ← 0
M[v] ← +∞  for all v ≠ t
for i ← 1, ..., n-1 do
    for each v ∈ V do
        M[v] ← min(
            M[v],
            min{ c(v,w) + M[w] | ⟨v,w⟩ ∈ A }
        )
return M[s]
</code></pre></div></div>

<p>This version using only a 1D array is also correct.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="dynamic-programming" /><category term="RNA-folding" /><category term="bellman-ford" /><category term="shortest-path" /><category term="negative-edges" /><summary type="html"><![CDATA[RNA Secondary Structure Prediction]]></summary></entry><entry><title type="html">Complexity Theory: Polynomial-Time Reductions</title><link href="https://programelot.github.io/algorithms/2026/06/03/polynomial-time-reductions/" rel="alternate" type="text/html" title="Complexity Theory: Polynomial-Time Reductions" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/polynomial-time-reductions</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/polynomial-time-reductions/"><![CDATA[<h1 id="complexity-theory-polynomial-time-reductions">Complexity Theory: Polynomial-Time Reductions</h1>

<h2 id="introduction">Introduction</h2>

<p>So far, our goal has been to design efficient algorithms. In Complexity Theory, the goal is the opposite: to identify <strong>hard problems for which no efficient algorithm exists</strong>.</p>

<p><strong>Key idea:</strong> Compare the difficulty of different problems. If we prove that some problem is “harder” than a famous open problem that has remained unsolved for decades, this provides strong evidence that the problem is also hard.</p>

<p>We only consider <strong>decision problems</strong> with yes/no answers.</p>

<hr />

<h2 id="polynomial-time-reduction">Polynomial-Time Reduction</h2>

<blockquote>
  <p><strong>Definition 1.</strong> $Y$ is <strong>polynomial-time reducible</strong> to $X$, written $Y \leq_P X$, if given a black box that solves $X$, any instance of $Y$ can be solved using a polynomial number of basic operations and a polynomial number of black-box calls.</p>
</blockquote>

<p>Intuitively, $Y \leq_P X$ means “$X$ is at least as hard as $Y$.”</p>

<p><strong>Theorem 1.</strong> If $Y \leq_P X$ and $X$ can be solved in polynomial time, then $Y$ can also be solved in polynomial time.</p>

<p><strong>Proof.</strong> Let $A$ be an algorithm for $Y$ and $B$ be a polynomial-time algorithm for $X$. Run $A$ but replace black-box calls with $B$. The $X$ instances written by $A$ have polynomial size, and polynomials are closed under composition, so the total running time is polynomial. ∎</p>

<p><strong>Corollary 1.</strong> If $Y \leq_P X$ and $Y$ cannot be solved in polynomial time, then $X$ cannot be solved in polynomial time.</p>

<p><strong>Lemma 2 (Transitivity).</strong> If $Z \leq_P Y$ and $Y \leq_P X$, then $Z \leq_P X$.</p>

<hr />

<h1 id="independent-set--vertex-cover">Independent Set ↔ Vertex Cover</h1>

<h2 id="problem-definitions">Problem Definitions</h2>

<blockquote>
  <p><strong>Problem 1 (Independent Set).</strong> Given $G = (V, E)$ and integer $k$: does an independent set of size $\geq k$ exist?</p>
</blockquote>

<p>An independent set $S$: no two nodes in $S$ are adjacent.</p>

<blockquote>
  <p><strong>Problem 2 (Vertex Cover).</strong> Given $G = (V, E)$ and integer $k$: does a vertex cover of size $\leq k$ exist?</p>
</blockquote>

<p>A vertex cover $S$: every edge $e \in E$ has at least one endpoint in $S$.</p>

<hr />

<h2 id="reduction">Reduction</h2>

<p><strong>Lemma 3.</strong> $S \subseteq V$ is an independent set $\iff$ $V \setminus S$ is a vertex cover.</p>

<p><strong>Proof.</strong></p>
<ul>
  <li>$(\Rightarrow)$: If $S$ is independent, for every edge $(u,v)$, at least one endpoint is in $V \setminus S$.</li>
  <li>$(\Leftarrow)$: If $V \setminus S$ is a vertex cover, any two vertices $u, v$ in $S$ have $(u,v) \notin E$. ∎</li>
</ul>

<p><strong>Theorem 2.</strong> Independent Set $\leq_P$ Vertex Cover (and vice versa — they are equally hard).</p>

<p><strong>Proof.</strong> Given $G = (V,E)$ and $k$, ask the Vertex Cover oracle “Does $G$ have a vertex cover of size $\leq \lvert V\rvert-k$?” By Lemma 3, this answer is the answer to the original problem. ∎</p>

<hr />

<h1 id="vertex-cover--set-cover">Vertex Cover → Set Cover</h1>

<blockquote>
  <p><strong>Problem 3 (Set Cover).</strong> Given a base set $U$, subsets $S_1, \ldots, S_m \subseteq U$, and integer $k$: do at most $k$ of the $S_i$’s have union equal to $U$?</p>
</blockquote>

<p>Vertex Cover is a special case of Set Cover (edges = elements to cover, vertices = covering sets).</p>

<p><strong>Theorem 3.</strong> Vertex Cover $\leq_P$ Set Cover.</p>

<p><strong>Proof.</strong> Given Vertex Cover instance $G = (V,E)$, $k$, construct:</p>
<ul>
  <li>Base set $U = E$</li>
  <li>For each vertex $i \in V$: $S_i = \lbrace\text{all edges incident to }i\rbrace$</li>
  <li>$k’ = k$</li>
</ul>

<p>The Vertex Cover instance is yes $\iff$ the Set Cover instance is yes. ∎</p>

<p>The chain of reductions established so far:</p>

\[\text{Independent Set} \leq_P \text{Vertex Cover} \leq_P \text{Set Cover}\]

<hr />

<h1 id="equivalence-of-optimization-and-decision-versions">Equivalence of Optimization and Decision Versions</h1>

<p><strong>Remark.</strong> The optimization version of Independent Set (computing the maximum independent set size) and the decision version are equally hard.</p>

<ul>
  <li>Optimization → decision: trivial</li>
  <li>Decision → optimization: run the decision version for $k = 0, 1, \ldots, \lvert V\rvert$ (binary search suffices)</li>
</ul>

<p>This equivalence justifies focusing only on yes/no decision problems throughout complexity theory.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="complexity-theory" /><category term="polynomial-reduction" /><category term="independent-set" /><category term="vertex-cover" /><category term="set-cover" /><summary type="html"><![CDATA[Complexity Theory: Polynomial-Time Reductions]]></summary></entry><entry><title type="html">Maximum Flow: Ford-Fulkerson and the Max-Flow Min-Cut Theorem</title><link href="https://programelot.github.io/algorithms/2026/06/03/network-flow-ford-fulkerson/" rel="alternate" type="text/html" title="Maximum Flow: Ford-Fulkerson and the Max-Flow Min-Cut Theorem" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/network-flow-ford-fulkerson</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/network-flow-ford-fulkerson/"><![CDATA[<h1 id="maximum-flow">Maximum Flow</h1>

<h2 id="motivation-and-definitions">Motivation and Definitions</h2>

<p>The problem of sending as much flow (e.g., oil, data) as possible from source $s$ to sink $t$ through a network with capacity constraints.</p>

<blockquote>
  <p><strong>Definition 1.</strong> A <strong>flow network</strong> is a directed graph $G = (V, E)$ where each edge $e$ has a nonnegative capacity $c_e$ and two special nodes (source $s$, sink $t$).</p>
</blockquote>

<p><strong>Simplifying assumptions:</strong> No edges enter $s$ or leave $t$; every node has at least one incident edge; all capacities are integers.</p>

<blockquote>
  <p><strong>Definition 2.</strong> An <strong>$s$-$t$ flow</strong> $f: E \to \mathbb{R}_+$ satisfies:</p>
  <ul>
    <li><strong>Capacity condition:</strong> $0 \leq f(e) \leq c_e$ for all $e$</li>
    <li><strong>Flow conservation:</strong> $f^{in}(v) = f^{out}(v)$ for all $v \in V \setminus \lbrace s,t\rbrace$</li>
  </ul>
</blockquote>

<blockquote>
  <p><strong>Definition 3.</strong> The value of a flow: $v(f) := \sum_{e \text{ out of } s} f(e)$</p>
</blockquote>

<blockquote>
  <p><strong>Problem (Maximum Flow).</strong> Find an $s$-$t$ flow of maximum value.</p>
</blockquote>

<hr />

<h2 id="residual-graph">Residual Graph</h2>

<p>A greedy approach (always augmenting along any path) is insufficient: once flow is sent, it cannot be redirected. We fix this by introducing the <strong>residual graph</strong>, which allows canceling flow.</p>

<blockquote>
  <p><strong>Definition 4.</strong> The <strong>residual graph</strong> $G_f$ for flow $f$:</p>
  <ul>
    <li>Node set: same as $G$</li>
    <li><strong>Forward edge:</strong> for $e = \langle u,v \rangle$ with $f(e) &lt; c_e$, add $\langle u,v \rangle$ with capacity $c_e - f(e)$</li>
    <li><strong>Backward edge:</strong> for $e = \langle u,v \rangle$ with $f(e) &gt; 0$, add $\langle v,u \rangle$ with capacity $f(e)$</li>
  </ul>
</blockquote>

<p>An $s$-$t$ path in $G_f$ is called an <strong>augmenting path</strong>.</p>

<hr />

<h2 id="augment-function">Augment Function</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function Augment(f, P):
    b ← minimum residual capacity over all edges in P
    for each e = ⟨u,v⟩ ∈ P do
        if e is forward edge: f(e) ← f(e) + b
        else:
            e' := ⟨v,u⟩
            f(e') ← f(e') - b
    return f
</code></pre></div></div>

<p><strong>Lemma 1.</strong> If $f$ is a valid $s$-$t$ flow and $P$ is a simple $s$-$t$ path in $G_f$, then <code class="language-plaintext highlighter-rouge">Augment(f, P)</code> returns a valid $s$-$t$ flow.</p>

<p><strong>Proof sketch.</strong> For capacity: forward edges $f’(e) = f(e)+b \leq c_e$; backward edges $f’(e’) = f(e’)-b \geq 0$. For flow conservation: at each internal node $v$ of $P$, exactly one edge enters and one leaves, so the net change is zero. ∎</p>

<hr />

<h2 id="ford-fulkerson-algorithm">Ford-Fulkerson Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>f(e) ← 0  for all e ∈ E
while there exists an s-t path in Gf do
    P ← simple s-t path in Gf
    f ← Augment(f, P)
return f
</code></pre></div></div>

<hr />

<h2 id="termination-and-complexity">Termination and Complexity</h2>

<p><strong>Lemma 2.</strong> Throughout the algorithm, all edge flows and residual capacities are always integers.</p>

<p><strong>Proof.</strong> By induction on iterations; the bottleneck $b$ is always an integer. ∎</p>

<p><strong>Lemma 3.</strong> Each iteration strictly increases $v(f)$.</p>

<p><strong>Proof.</strong> All residual capacities are positive, so $b &gt; 0$. The first edge of $P$ is a forward edge leaving $s$ (not revisited), so $f^{out}(s)$ increases by $b &gt; 0$. ∎</p>

<p>Define $C := \sum_{e \text{ out of } s} c_e$.</p>

<p><strong>Lemma 4.</strong> The algorithm terminates in at most $C$ iterations.</p>

<p><strong>Proof.</strong> $v(f) \leq C$ always. Each iteration increases $v(f)$ by at least 1 (integer, by Lemma 2). Starting from 0, at most $C$ iterations follow. ∎</p>

<p><strong>Theorem 1.</strong> Ford-Fulkerson runs in $O(mC)$ time.</p>

<blockquote>
  <p><strong>Note:</strong> This is <strong>not</strong> a polynomial-time algorithm. $C$ can be exponential in the number of input bits, making this a <strong>pseudo-polynomial</strong> algorithm. A strongly polynomial algorithm is bounded only by the number of input entries.</p>
</blockquote>

<hr />

<h2 id="s-t-cut">$s$-$t$ Cut</h2>

<blockquote>
  <p><strong>Definition 5.</strong> $(S, \bar{S})$ is an <strong>$s$-$t$ cut</strong> if $s \in S$, $t \in \bar{S}$, $S \cap \bar{S} = \emptyset$, $S \cup \bar{S} = V$.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 6.</strong> Capacity of cut $(S, \bar{S})$: $c(S, \bar{S}) := \sum_{e \text{ out of } S} c_e$</p>
</blockquote>

<p><strong>Lemma 5.</strong> For any $s$-$t$ flow $f$ and $s$-$t$ cut $(S, \bar{S})$: $v(f) = f^{out}(S) - f^{in}(S)$.</p>

<p><strong>Proof.</strong> Sum $[f^{out}(v) - f^{in}(v)]$ over all $v \in S$: internal edges cancel; flow conservation makes all $v \neq s$ contribute 0. ∎</p>

<p><strong>Corollary 1.</strong> $v(f) \leq c(S, \bar{S})$ for any flow $f$ and cut $(S, \bar{S})$.</p>

<hr />

<h2 id="max-flow--min-cut">Max-Flow = Min-Cut</h2>

<p><strong>Lemma 6.</strong> If $G_f$ has no $s$-$t$ path, there exists a cut $(S^<em>, \bar{S}^</em>)$ with $v(f) = c(S^<em>, \bar{S}^</em>)$.</p>

<p><strong>Proof.</strong> Let $S$ = vertices reachable from $s$ in $G_f$, $\bar{S} = V \setminus S$.</p>

<ul>
  <li>Each edge $e = \langle u,v \rangle$ leaving $S$ in $G$: since $v \notin S$, $\langle u,v \rangle \notin G_f$ → $f(e) = c_e$.</li>
  <li>Each edge $e = \langle u,v \rangle$ entering $S$ in $G$: since $u \notin S$, backward edge $\langle v,u \rangle \notin G_f$ → $f(e) = 0$.</li>
</ul>

<p>Therefore $v(f) = f^{out}(S) - f^{in}(S) = c(S, \bar{S})$. ∎</p>

<p><strong>Corollary 2 (Ford-Fulkerson returns max flow).</strong> The algorithm terminates when $G_f$ has no $s$-$t$ path, at which point Lemma 6 gives an equal-value cut, making $f$ optimal.</p>

<p><strong>Theorem 2 (Max-Flow Min-Cut).</strong> In every flow network, the value of the maximum $s$-$t$ flow equals the capacity of the minimum $s$-$t$ cut.</p>

<p><strong>Theorem 3 (Integrality).</strong> If all capacities are integers, there exists a maximum flow in which every edge carries integer flow.</p>

<p><strong>Proof.</strong> Ford-Fulkerson returns such a flow. ∎</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="network-flow" /><category term="ford-fulkerson" /><category term="max-flow-min-cut" /><category term="augmenting-path" /><category term="residual-graph" /><summary type="html"><![CDATA[Maximum Flow]]></summary></entry><entry><title type="html">Dynamic Programming: Hotel Scheduling, LNS, and Sequence Alignment</title><link href="https://programelot.github.io/algorithms/2026/06/03/dynamic-programming-intro/" rel="alternate" type="text/html" title="Dynamic Programming: Hotel Scheduling, LNS, and Sequence Alignment" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/dynamic-programming-intro</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/dynamic-programming-intro/"><![CDATA[<h1 id="dynamic-programming">Dynamic Programming</h1>

<h2 id="motivation-hotel-scheduling-problem">Motivation: Hotel Scheduling Problem</h2>

<blockquote>
  <p><strong>Problem 1.</strong> Given $n+1$ cities $c_0, \ldots, c_n$ with distances $d_i$ between consecutive cities, find a schedule that minimizes hotel costs while respecting a maximum daily travel distance $D$. The cost of staying at $c_i$ is $p_i$, and the cost at $c_n$ is always paid.</p>
</blockquote>

<hr />

<h2 id="approaches-and-pitfalls">Approaches and Pitfalls</h2>

<h3 id="approach-1-generate-all-cases-inefficient">Approach 1: Generate All Cases (Inefficient)</h3>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function GenerateSchedules(k):
    if k = 0: return {empty schedule}
    S ← GenerateSchedules(k-1)
    S' ← ∅
    for each schedule s ∈ S:
        add version "sleep at ck" to S'
        add version "don't sleep at ck" to S'
    return S'
</code></pre></div></div>

<p><strong>Problem:</strong> The amount of information passed between function calls grows exponentially.</p>

<h3 id="approach-2-recursion--split-flawed">Approach 2: Recursion + Split (Flawed)</h3>

<p>If we sleep at an intermediate city $c_i$, split the problem into $[c_0, \ldots, c_i]$ and $[c_i, \ldots, c_n]$.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function BestSchedule(c0, ..., cn):
    sol ← ∞
    if Σdi ≤ D: sol ← min(sol, pn)
    for i ← 1, ..., n-1:
        sol1 ← BestSchedule(c0, ..., ci)
        sol2 ← BestSchedule(ci, ..., cn)
        sol ← min(sol, sol1 + sol2)
    return sol
</code></pre></div></div>

<p><strong>Problem:</strong> Same subproblems are computed multiple times. For example, if the optimal solution sleeps at $c_2, c_4, c_6$, this solution is considered separately when $i=2$, $i=4$, and $i=6$.</p>

<h3 id="approach-3-simplified-recursion">Approach 3: Simplified Recursion</h3>

<p>Focus on “the last city slept at (excluding $c_n$).”</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function BestSchedule(n):
    if n = 0: return p0
    sol ← ∞
    for i ← 0, ..., n-1:
        if Σ(j=i+1 to n) dj ≤ D:
            sol ← min(sol, pn + BestSchedule(i))
    return sol
</code></pre></div></div>

<p><strong>Correctness:</strong> Clear. However, the number of recursive calls is exponential due to overlapping subproblems.</p>

<p><strong>Key observation:</strong> The return value depends only on argument $n$, so the same computation repeats every time the function is called with the same $n$.</p>

<hr />

<h2 id="memoization">Memoization</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function BestSchedule(n):
    if n = 0: return p0
    if M[n] is defined: return M[n]
    sol ← ∞
    for i ← 0, ..., n-1:
        if Σ(j=i+1 to n) dj ≤ D:
            sol ← min(sol, pn + BestSchedule(i))
    M[n] ← sol
    return sol

create array M
return BestSchedule(n)
</code></pre></div></div>

<p>The technique of caching return values to avoid redundant computation is called <strong>memoization</strong>.</p>

<p><strong>Time complexity analysis:</strong> The main logic (computing <code class="language-plaintext highlighter-rouge">sol</code>) executes at most once per value of $n$ (no circular dependencies). One execution: $O(n)$. Number of distinct inputs: $O(n)$. <strong>Overall: $O(n^2)$.</strong></p>

<hr />

<h2 id="iterative-dynamic-programming">Iterative Dynamic Programming</h2>

<p>Observing the recursive structure, we can fill a table in a simple order ($i = 0, 1, \ldots, n$):</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[0] ← 0
for i ← 1, ..., n do
    M[i] ← pi + min{ M[j] | 0 ≤ j ≤ i-1, Σ(k=j+1 to i) dk ≤ D }
return M[n]
</code></pre></div></div>

<p>($\min \emptyset := +\infty$)</p>

<hr />

<h2 id="core-principles-of-dynamic-programming">Core Principles of Dynamic Programming</h2>

<blockquote>
  <p><strong>Dynamic Programming:</strong></p>
  <ol>
    <li>First define a <strong>recurrence</strong> satisfied by the optimal solutions of subproblems.</li>
    <li>Fill a table by computing subproblems in order from smallest to largest.</li>
    <li>Determine the evaluation order to avoid circular dependencies.</li>
  </ol>
</blockquote>

<ul>
  <li><strong>Difference from divide-and-conquer:</strong> Subproblem sizes typically decrease by only a small amount.</li>
  <li>Why pass only small amounts of information between recursive calls: it determines the size of the subproblem table (= running time).</li>
  <li>Why <strong>“decoupling”</strong> matters: we must express the optimal solution to a large problem in terms of optimal solutions to subproblems.</li>
</ul>

<hr />

<h1 id="longest-nondecreasing-subsequence-lns">Longest Nondecreasing Subsequence (LNS)</h1>

<h2 id="problem-definition">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 2.</strong> Given a sequence of integers $a_1, \ldots, a_n$, find the longest nondecreasing subsequence (not necessarily contiguous).</p>
</blockquote>

<p><strong>Example:</strong> <code class="language-plaintext highlighter-rouge">40 41 2 41 7 8 15 1 3</code></p>

<hr />

<h2 id="subproblem-coupling-issue-and-resolution">Subproblem Coupling Issue and Resolution</h2>

<p>If we define $\text{OPT}(i)$ as the length of the longest nondecreasing subsequence of $a_1, \ldots, a_i$, a coupling problem arises: when expressing $\text{OPT}(9)$ in terms of $\text{OPT}(i)$ ($i \leq 8$), we cannot tell whether the last value of each subproblem’s solution is $\leq a_9 = 3$.</p>

<p><strong>Resolution — Subproblem Redefinition:</strong></p>

<blockquote>
  <p><strong>Definition.</strong> $\text{OPT}(i)$ = the length of the longest nondecreasing subsequence of $a_1, \ldots, a_i$ that <strong>must include $a_i$</strong></p>
</blockquote>

<p>“Undoing the last decision”: consider element $a_j$ ($j &lt; i$, $a_j \leq a_i$) directly before $a_i$.</p>

<hr />

<h2 id="recurrence">Recurrence</h2>

<p><strong>Lemma 1.</strong> For all $i \geq 1$:</p>

\[\text{OPT}(i) = \max \begin{cases} 1 \\ \displaystyle\max_{\substack{j:\ 1 \leq j \leq i-1 \\ a_j \leq a_i}} \text{OPT}(j) + 1 &amp; \text{(if such } j \text{ exists)} \end{cases}\]

<h3 id="two-requirements-for-recurrence-design">Two Requirements for Recurrence Design</h3>

<ol>
  <li><strong>Exhaustive:</strong> The optimal solution must fall into at least one of the cases.</li>
  <li><strong>Sound:</strong> Each case must correspond to a feasible solution.</li>
</ol>

<hr />

<h2 id="proof-of-lemma-1">Proof of Lemma 1</h2>

<p><strong>LHS ≤ RHS:</strong> Let $a_{k_1}, \ldots, a_{k_\ell}$ ($k_\ell = i$) be the longest nondecreasing subsequence containing $a_i$.</p>

<ul>
  <li>If $\ell = 1$: $\text{OPT}(i) = 1$, so $\text{LHS} \leq \text{RHS}$.</li>
  <li>If $\ell &gt; 1$: Let $j = k_{\ell-1}$. Since $a_{k_1}, \ldots, a_{k_{\ell-1}}$ ends at $a_j$, $\text{OPT}(j) \geq \text{OPT}(i) - 1$, giving $\text{LHS} \leq \text{RHS}$.</li>
</ul>

<p><strong>LHS ≥ RHS:</strong></p>

<ul>
  <li>$\text{OPT}(i) \geq 1$: The sequence $\lbrace a_i\rbrace$ itself is nondecreasing.</li>
  <li>For all $j$ with $a_j \leq a_i$: appending $a_i$ to the subsequence achieving $\text{OPT}(j)$ gives a nondecreasing subsequence of length $\text{OPT}(j)+1$, so $\text{OPT}(i) \geq \text{OPT}(j)+1$.</li>
</ul>

<p>Therefore $\text{LHS} \geq \text{RHS}$. ∎</p>

<hr />

<h2 id="algorithm">Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[0] ← 0  (define a0 := -∞)
for i ← 1, ..., n do
    M[i] ← max{ M[j] + 1 | 0 ≤ j ≤ i-1, aj ≤ ai }
return max(M[1], ..., M[n])
</code></pre></div></div>

<p><strong>Lemma 2.</strong> $M[i] = \text{OPT}(i)$.</p>

<p><strong>Proof.</strong> Defining $\text{OPT}(0) := 0$ and $a_0 := -\infty$ unifies the recurrence. Proved by induction on $i$. ∎</p>

<p><strong>Theorem 1.</strong> The algorithm correctly returns the LNS length in $O(n^2)$ time.</p>

<hr />

<h2 id="recovering-the-actual-sequence">Recovering the Actual Sequence</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[0] ← 0
for i ← 1, ..., n do
    M[i] ← max{ M[j]+1 | 0 ≤ j ≤ i-1, aj ≤ ai }
    prev[i] ← corresponding j (argmax)
last ← argmax{ M[i] | 1 ≤ i ≤ n }
while last ≠ 0:
    push a[last] to stack
    last ← prev[last]
pop everything  (output)
</code></pre></div></div>

<hr />

<h1 id="sequence-alignment">Sequence Alignment</h1>

<h2 id="problem-definition-1">Problem Definition</h2>

<p>A problem that quantifies how similar two strings are.</p>

<blockquote>
  <p><strong>Definition 2.</strong> A <strong>matching</strong> $M \subseteq \lbrace 1,\ldots,m\rbrace \times \lbrace 1,\ldots,n\rbrace$ is a set where each index appears at most once.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 3.</strong> A matching is an <strong>alignment</strong> if there are no crossing pairs: for $(i,j),(i’,j’) \in M$ with $i &lt; i’$, we have $j &lt; j’$.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 4.</strong> The <strong>cost</strong> of an alignment:</p>
  <ul>
    <li>Unmatched position (gap): penalty $\delta$</li>
    <li>$(i,j) \in M$ with $x_i \neq y_j$: mismatch cost $\alpha_{x_i y_j}$</li>
  </ul>
</blockquote>

<blockquote>
  <p><strong>Problem 3.</strong> Find the minimum-cost alignment of two strings $x_1 \cdots x_m$ and $y_1 \cdots y_n$.</p>
</blockquote>

<hr />

<h2 id="recurrence-1">Recurrence</h2>

<p>Let $\text{OPT}(i,j)$ be the minimum cost alignment of $x_1 \cdots x_i$ and $y_1 \cdots y_j$. The last decision is either $(i,j) \in M$, or at least one of $x_i$ or $y_j$ is unmatched.</p>

<p><strong>Lemma 3.</strong> For all $i \geq 1$, $j \geq 1$:</p>

\[\text{OPT}(i,j) = \min \begin{cases} \text{OPT}(i-1,j-1) + \alpha_{x_i y_j} \\ \text{OPT}(i-1,j) + \delta \\ \text{OPT}(i,j-1) + \delta \end{cases}\]

<p><strong>Boundary conditions:</strong> $\text{OPT}(0,k) = \text{OPT}(k,0) = k\delta$</p>

<hr />

<h2 id="algorithm-1">Algorithm</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M[i,0] ← iδ  for all i
M[0,j] ← jδ  for all j
for i ← 1, ..., m do
    for j ← 1, ..., n do
        M[i,j] ← min(
            M[i-1,j-1] + α(xi, yj),
            M[i-1,j] + δ,
            M[i,j-1] + δ
        )
return M[m,n]
</code></pre></div></div>

<p><strong>Theorem 2.</strong> The algorithm is correct (by induction on $i+j$) and runs in $O(mn)$ time.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="dynamic-programming" /><category term="memoization" /><category term="LNS" /><category term="sequence-alignment" /><category term="scheduling" /><summary type="html"><![CDATA[Dynamic Programming]]></summary></entry><entry><title type="html">Karatsuba Integer Multiplication</title><link href="https://programelot.github.io/algorithms/2026/06/03/karatsuba-integer-multiplication/" rel="alternate" type="text/html" title="Karatsuba Integer Multiplication" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/karatsuba-integer-multiplication</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/karatsuba-integer-multiplication/"><![CDATA[<h1 id="karatsuba-integer-multiplication">Karatsuba Integer Multiplication</h1>

<h2 id="problem">Problem</h2>

<blockquote>
  <p><strong>Problem.</strong> Given two positive $n$-bit integers $x$ and $y$, compute their product $xy$.</p>
</blockquote>

<p>The elementary school algorithm runs in $O(n^2)$. Our goal is a subquadratic algorithm.</p>

<blockquote>
  <p><strong>Note:</strong> Efficiency is measured with respect to the <strong>input size</strong> (number of bits). $O(PQ)$ is exponential in the input size.</p>
</blockquote>

<hr />

<h2 id="divide-and-conquer-approach">Divide-and-Conquer Approach</h2>

<p>Split $x = x_1 \cdot 2^{\lfloor n/2 \rfloor} + x_0$ and $y = y_1 \cdot 2^{\lfloor n/2 \rfloor} + y_0$:</p>

\[xy = x_1 y_1 \cdot 2^{2\lfloor n/2 \rfloor} + (x_1 y_0 + x_0 y_1) \cdot 2^{\lfloor n/2 \rfloor} + x_0 y_0\]

<p><strong>4-recursion version:</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>p1 ← Multiply(x1, y1)
p2 ← Multiply(x1, y0)
p3 ← Multiply(x0, y1)
p4 ← Multiply(x0, y0)
return p1·2^(2⌊n/2⌋) + (p2+p3)·2^(⌊n/2⌋) + p4
</code></pre></div></div>

<p><strong>Theorem 1.</strong> $T(n) = 4T(n/2) + \Theta(n) = \Theta(n^2)$ — no improvement over the elementary algorithm.</p>

<hr />

<h2 id="karatsuba-algorithm">Karatsuba Algorithm</h2>

<p><strong>Key observation:</strong> $x_1 y_0 + x_0 y_1 = (x_1 + x_0)(y_1 + y_0) - x_1 y_1 - x_0 y_0$</p>

<p>This reduces 4 multiplications to <strong>3</strong>.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function Multiply(x, y):
    if n ≤ 3: return elementary_school(x, y)
    p1 ← Multiply(x1+x0, y1+y0)
    p2 ← Multiply(x1, y1)
    p3 ← Multiply(x0, y0)
    return p2·2^(2⌊n/2⌋) + (p1-p2-p3)·2^(⌊n/2⌋) + p3
</code></pre></div></div>

<p><strong>Theorem 2 (Correctness).</strong> Proved by induction on $n$:</p>

\[p_2 \cdot 2^{2\lfloor n/2 \rfloor} + (p_1 - p_2 - p_3) \cdot 2^{\lfloor n/2 \rfloor} + p_3 = xy\]

<p><strong>Theorem 3 (Time complexity).</strong> The Karatsuba algorithm runs in $O(n^{\log_2 3})$ time.</p>

<p><strong>Proof.</strong> $T(n) = 3T(n/2) + cn$. By the Master Theorem:</p>

\[T(n) = \Theta(n^{\log_2 3}) = O(n^{1.59}) \qquad \square\]]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="divide-and-conquer" /><category term="karatsuba" /><category term="integer-multiplication" /><category term="master-theorem" /><summary type="html"><![CDATA[Karatsuba Integer Multiplication]]></summary></entry><entry><title type="html">Greedy Algorithms: Interval Scheduling and Minimizing Maximum Lateness</title><link href="https://programelot.github.io/algorithms/2026/06/03/greedy-interval-scheduling/" rel="alternate" type="text/html" title="Greedy Algorithms: Interval Scheduling and Minimizing Maximum Lateness" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/greedy-interval-scheduling</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/greedy-interval-scheduling/"><![CDATA[<h1 id="interval-scheduling">Interval Scheduling</h1>

<h2 id="problem-definition">Problem Definition</h2>

<p>The lecture hall scheduling problem: accept as many non-overlapping requests as possible.</p>

<blockquote>
  <p><strong>Problem 1.</strong> Given $n$ intervals $1, \ldots, n$, find the maximum subset of mutually compatible intervals.</p>
</blockquote>

<ul>
  <li>$s(i)$, $f(i)$: start and finish times of interval $i$ ($s(i) &lt; f(i)$)</li>
  <li>Intervals $i$ and $j$ are <strong>compatible</strong> if $f(i) \leq s(j)$ or $f(j) \leq s(i)$</li>
</ul>

<hr />

<h2 id="incorrect-attempt">Incorrect Attempt</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>let R = given intervals
sol ← ∅
while R ≠ ∅ do
    i ← interval in R with fewest overlaps with others
    sol ← sol ∪ {i}
    remove all intervals overlapping i from R
</code></pre></div></div>

<p><strong>Counterexample:</strong> Choosing the interval with the fewest conflicts first can fail to find the optimal solution.</p>

<hr />

<h2 id="correct-algorithm-select-earliest-finishing-interval">Correct Algorithm: Select Earliest-Finishing Interval</h2>

<p><strong>Intuition:</strong> Selecting the interval with the earliest finish time leaves the room free “as soon as possible.”</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>let R = given intervals
sol ← ∅
while R ≠ ∅ do
    i ← interval in R with earliest finish time
    sol ← sol ∪ {i}
    remove all intervals overlapping i from R (including i)
return sol
</code></pre></div></div>

<p><strong>Observation 1.</strong> The algorithm returns a set of compatible intervals (each selected interval’s conflicts are immediately removed).</p>

<hr />

<h2 id="correctness-proof-stay-ahead-argument">Correctness Proof (Stay-Ahead Argument)</h2>

<p><strong>Notation:</strong></p>
<ul>
  <li>Algorithm’s solution: $\text{sol} = \lbrace i_1, \ldots, i_k\rbrace$ ($i_\ell$ selected in iteration $\ell$)</li>
  <li>Optimal solution: $O = \lbrace j_1, \ldots, j_m\rbrace$ ($f(j_1) \leq \cdots \leq f(j_m)$)</li>
</ul>

<p><strong>Lemma 1.</strong> For all $1 \leq r \leq m$: $k \geq r$ and $f(i_r) \leq f(j_r)$.</p>

<p><strong>Proof.</strong> By induction on $r$.</p>

<ul>
  <li>
    <p><strong>Base case ($r=1$):</strong> The algorithm selects the earliest finish time, so $k \geq 1$ and $f(i_1) \leq f(j)$ for all $j \in I$.</p>
  </li>
  <li>
    <p><strong>Inductive step ($r \geq 2$):</strong> By hypothesis, $f(i_{r-1}) \leq f(j_{r-1})$. Since $O$ is compatible, $f(j_{r-1}) \leq s(j_r)$. Therefore:</p>
  </li>
</ul>

\[f(i_1) \leq \cdots \leq f(i_{r-1}) \leq f(j_{r-1}) \leq s(j_r)\]

<p>So $j_r$ is not removed before iteration $r$, meaning the $r$-th iteration exists and $f(i_r) \leq f(j_r)$. ∎</p>

<p><strong>Theorem 1.</strong> The algorithm is correct.</p>

<p><strong>Proof.</strong> By Lemma 1, $k \geq m$. Since the algorithm returns a compatible set, $k \leq m$. Therefore $k = m$. ∎</p>

<p><strong>Theorem 2.</strong> Time complexity: $O(n \log n)$ (sort by finish time, then scan once).</p>

<hr />

<h2 id="certifying-algorithm">Certifying Algorithm</h2>

<p>We can output a <strong>certificate</strong> of optimality alongside the solution.</p>

<p><strong>Observation 2.</strong> If $J \subseteq I$ is compatible, then $\text{OPT}(I) \geq \lvert J\rvert$.</p>

<p><strong>Lemma 2.</strong> If $I_1, \ldots, I_k$ partition $I$ and every pair of intervals within each $I_i$ overlaps, then $\text{OPT}(I) \leq k$.</p>

<p><strong>Proof.</strong> For the optimal solution $O$, $\lvert I_i \cap O\rvert \leq 1$ for each $I_i$. Therefore $\text{OPT}(I) = \lvert O\rvert \leq k$. ∎</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>let R = given intervals
sol ← ∅
k ← 0
while R ≠ ∅ do
    k ← k + 1
    ik ← interval in R with earliest finish time
    sol ← sol ∪ {ik}
    Ck ← all intervals in R overlapping ik (including ik)
    R ← R \ Ck
return sol
</code></pre></div></div>

<p>This algorithm outputs <code class="language-plaintext highlighter-rouge">sol</code> together with $C_1, \ldots, C_{k^<em>}$ as the optimality certificate. In each $C_k$, all intervals contain $f(i_k)$ and therefore mutually overlap. By Lemma 2, $\text{OPT}(I) \leq k^</em>$, and since the algorithm returns $k^*$ compatible intervals, the solution is optimal.</p>

<hr />

<h2 id="characteristics-of-greedy-algorithms">Characteristics of Greedy Algorithms</h2>

<p>The defining characteristic of a <strong>greedy algorithm</strong>: it makes a <strong>myopic</strong> choice at each step. While combinatorial choices are often coupled, myopic choices under a carefully selected criterion can yield an optimal algorithm for certain problems.</p>

<hr />

<h1 id="minimizing-maximum-lateness">Minimizing Maximum Lateness</h1>

<h2 id="problem-definition-1">Problem Definition</h2>

<blockquote>
  <p><strong>Problem 2.</strong> Given $n$ jobs with processing times $p_i$ and deadlines $d_i$, find a schedule that minimizes the maximum lateness $L := \max_{1 \leq i \leq n} \ell(i)$.</p>
</blockquote>

<p><strong>Definitions:</strong></p>
<ul>
  <li>$s(i)$: start time of job $i$, $f(i) := s(i) + p_i$: completion time</li>
  <li>$\ell(i) := \max(f(i) - d_i,\ 0)$: lateness of job $i$</li>
</ul>

<hr />

<h2 id="failed-approaches">Failed Approaches</h2>

<p><strong>Shortest job first:</strong> $p_1=1, d_1=100$, $p_2=2, d_2=2$ → doing job 1 first gives $L=1$, optimally job 2 first gives $L=0$.</p>

<p><strong>Smallest slack first ($d_i - p_i$):</strong> $p_1=1, d_1=2$, $p_2=10, d_2=10$ → counterexample exists.</p>

<hr />

<h2 id="edd-earliest-due-date-rule">EDD (Earliest Due Date) Rule</h2>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sort jobs so that d1 ≤ ... ≤ dn
f ← 0
for each job i = 1, ..., n do
    s(i) ← f
    f ← f + pi
</code></pre></div></div>

<hr />

<h2 id="correctness-proof">Correctness Proof</h2>

<p><strong>Definition 1.</strong> A schedule has <strong>idle time</strong> if the machine is empty at some time $t$ but has a job scheduled at some $t’ &gt; t$.</p>

<p><strong>Lemma 3.</strong> There exists an optimal schedule with no idle time. (Compressing idle time does not increase maximum lateness.)</p>

<p><strong>Definition 2.</strong> An <strong>inversion</strong> $(i,j)$ occurs if job $i$ is scheduled before $j$ but $d_i &gt; d_j$.</p>

<p><strong>Lemma 4.</strong> There exists an optimal schedule with no inversions and no idle time.</p>

<p><strong>Proof sketch.</strong> If an inversion $(i,j)$ exists, there is an adjacent inversion pair. Swapping them:</p>
<ul>
  <li>Jobs other than $i, j$: completion time unchanged.</li>
  <li>$j$: $\bar{s}(j) = s(i) &lt; s(j)$ so $\bar{\ell}(j) \leq \ell(j) \leq L$.</li>
  <li>$i$: $\bar{\ell}(i) = \max\lbrace 0, f(j) - d_i\rbrace \leq \max\lbrace 0, f(j) - d_j\rbrace = \ell(j) \leq L$ (since $d_i &gt; d_j$).</li>
</ul>

<p>Each swap reduces inversions by 1. After at most $\binom{n}{2}$ swaps, we reach an inversion-free schedule. ∎</p>

<p><strong>Lemma 5.</strong> All schedules with no inversions and no idle time have the same maximum lateness.</p>

<p><strong>Proof.</strong> In any inversion-free schedule, jobs with the same deadline are scheduled consecutively after all earlier-deadline jobs. The last completion time among jobs with deadline $d$ is independent of their internal order, so the maximum lateness is the same. ∎</p>

<p><strong>Theorem 3.</strong> The EDD algorithm is correct. (Follows from Lemmas 3, 4, and 5.)</p>

<p>Time complexity: $O(n \log n)$ (dominated by sorting).</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="greedy" /><category term="interval-scheduling" /><category term="EDD" /><category term="job-scheduling" /><category term="stay-ahead" /><category term="certifying-algorithm" /><summary type="html"><![CDATA[Interval Scheduling]]></summary></entry><entry><title type="html">3-SAT, Independent Set, and the P vs. NP Question</title><link href="https://programelot.github.io/algorithms/2026/06/03/3sat-independent-set-p-np/" rel="alternate" type="text/html" title="3-SAT, Independent Set, and the P vs. NP Question" /><published>2026-06-03T00:00:00-05:00</published><updated>2026-06-03T00:00:00-05:00</updated><id>https://programelot.github.io/algorithms/2026/06/03/3sat-independent-set-p-np</id><content type="html" xml:base="https://programelot.github.io/algorithms/2026/06/03/3sat-independent-set-p-np/"><![CDATA[<h1 id="3-sat-p-independent-set">3-SAT ≤P Independent Set</h1>

<h2 id="sat-problem">SAT Problem</h2>

<blockquote>
  <p><strong>Problem 1 (SAT).</strong> Given $k$ clauses over $n$ variables: does a satisfying truth assignment exist?</p>
</blockquote>

<blockquote>
  <p><strong>Problem 2 (3-SAT).</strong> SAT where each clause has exactly 3 literals.</p>
</blockquote>

<ul>
  <li><strong>Literal:</strong> a variable $x_i$ or its negation $\bar{x}_i$</li>
  <li><strong>Clause:</strong> a disjunction of literals $t_1 \vee \cdots \vee t_\ell$</li>
  <li><strong>Conflict:</strong> $x_i$ and $\bar{x}_i$ conflict with each other</li>
</ul>

<hr />

<h2 id="the-reduction">The Reduction</h2>

<p><strong>Theorem 1.</strong> 3-SAT $\leq_P$ Independent Set.</p>

<p><strong>Proof.</strong> From a 3-SAT instance with $k$ clauses, construct graph $G = (V,E)$:</p>

<ol>
  <li>Create $k$ <strong>triangles</strong>, one per clause. Each vertex of a triangle corresponds to one of the 3 literals in that clause.</li>
  <li>Connect pairs of vertices corresponding to <strong>conflicting</strong> literals with an edge.</li>
  <li>Set the target independent set size $k’ = k$.</li>
</ol>

<p><strong>Example:</strong> For $\lbrace (x_1 \vee \bar{x}_2 \vee x_3),\ (\bar{x}_1 \vee \bar{x}_2 \vee \bar{x}_4)\rbrace$, set $k’=2$.</p>

<p><strong>($\Rightarrow$)</strong> If 3-SAT is satisfiable: pick one true literal per clause to get $k$ vertices.</p>
<ul>
  <li>Exactly one selected per triangle → no edges within triangles.</li>
  <li>Two conflicting literals cannot both be true → no conflict edges.</li>
  <li>This is an independent set of size $k$.</li>
</ul>

<p><strong>($\Leftarrow$)</strong> If there is an independent set $S$ of size $\geq k$: since at most one vertex can be selected per triangle, exactly one is chosen per triangle. Assign true to the literals in $S$ (others arbitrary):</p>
<ul>
  <li>$S$ cannot contain conflicting vertices → the assignment is consistent.</li>
  <li>Exactly one vertex per triangle → all clauses are satisfied. ∎</li>
</ul>

<p>The small components used are called <strong>gadgets</strong>. Combined with previous reductions:</p>

\[\text{3-SAT} \leq_P \text{Independent Set} \leq_P \text{Vertex Cover} \leq_P \text{Set Cover}\]

<hr />

<h1 id="definitions-of-p-and-np">Definitions of P and NP</h1>

<blockquote>
  <p><strong>Definition 3.</strong> <strong>P</strong> = the set of decision problems solvable by an efficient (polynomial-time) algorithm.</p>
</blockquote>

<blockquote>
  <p><strong>Definition 4.</strong> An <strong>efficient certifier</strong> $B$ for problem $X$:</p>
  <ul>
    <li>Takes strings $s$ (instance) and $t$ (hint) as input.</li>
    <li>Runs in time polynomial in $\lvert s\rvert$ and $\lvert t\rvert$.</li>
    <li>For every yes-instance $s$: there exists $t$ with $\lvert t\rvert \leq p(\lvert s\rvert)$ such that $B(s,t) = \text{yes}$.</li>
    <li>For every no-instance $s$: $B(s,t) = \text{no}$ for all $t$.</li>
  </ul>
</blockquote>

<blockquote>
  <p><strong>Definition 5.</strong> <strong>NP</strong> = the set of decision problems for which an efficient certifier exists.</p>
</blockquote>

<p>Equivalently, <strong>NP</strong> is the class of problems solvable by a <strong>Nondeterministic</strong> polynomial-time algorithm (NP = Nondeterministic Polynomial).</p>

<p><strong>Theorem 2.</strong> $\text{P} \subseteq \text{NP}$.</p>

<p><strong>Proof.</strong> If $X \in \text{P}$ with algorithm $A$, define $B(s,t)$ = “ignore $t$ and run $A(s)$.” This is an efficient certifier. ∎</p>

<p><strong>Theorem 3.</strong> 3-SAT, Independent Set, Vertex Cover $\in$ NP.</p>]]></content><author><name>Programelot</name></author><category term="algorithms" /><category term="NP" /><category term="3-SAT" /><category term="independent-set" /><category term="gadget" /><category term="P-vs-NP" /><category term="reduction" /><summary type="html"><![CDATA[3-SAT ≤P Independent Set]]></summary></entry></feed>