Skip to content

Commit bac92ed

Browse files
jgclarkcursoragent
andcommitted
b1: Priority note-index cache to speed warm Priority section refreshes.
Bump to 2.5.0.b1. Index filenames of notes with open unscheduled priority items (tag-cache pattern); fall back to full scan and schedule background rebuild when missing. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 57b73ff commit bac92ed

13 files changed

Lines changed: 735 additions & 15 deletions

‎jgclark.Dashboard/CHANGELOG.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# What's changed in 🎛 Dashboard plugin?
22
For more details see the [plugin's documentation](https://github.com/NotePlan/plugins/tree/main/jgclark.Dashboard/).
33

4-
## [2.5.0] 2026-08-???
4+
## [2.5.0.b1] 2026-08-27
5+
- New: Priority note-index cache (`priorityNoteIndexCache.json`) so the Priority section can avoid a full vault scan on warm refreshes. On by default (`FFlag_UsePriorityCache`); falls back to full scan and schedules a background rebuild when the cache is missing. Commands: `generatePriorityNoteIndexCache` / `updatePriorityNoteIndexCache`.
56
- New: Interactive Processing on Active Projects and Projects to Review (`>> N` walks project rows; PROJACT allows back-navigate, PROJREVIEW forward-only; Start Reviews unchanged).
67
- Dev: Consolidate shared section-row CSS (`.sectionItemRow`, `.sectionItemContent`, `.itemIcon`) into `ItemRow.css`; remove duplicate `TaskItem.css` / `TasksFiltered.css`.
8+
- Dev: Bring Reminder edit / Convert to Task / `ITEM_ORIG_SECTION` refresh and related 2.4.2–2.4.3 work from main onto the 2.5.0 branch.
79

810
## [2.4.3] 2026-08-???
911
- New: Reminder edit dialog Actions row has "Convert to Task" (second control) to convert a reminder into an open task prepended to today's daily note, then delete the Apple Reminder. Carries notes (in parentheses), location as `@mention`, `>date`, `at HH:MM`, and priority markers when set.

‎jgclark.Dashboard/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ If the **Yesterday** section is turned off, Overdue also includes open tasks fro
276276
You can set the "Sort order for Tag/Mention and Overdue items": 'priority' shows the higher priority (from `>>`, `!!!`, `!!` and `!` markers), 'earliest' by earliest modified date of the note, or 'most recent' changed note.
277277

278278
### Priority section
279-
This finds all **unscheduled** open items with a priority set (with `>>`, `!!!`, `!!` and `!` markers). Items that already have a `>date` (or `>today`) are left out on purpose: they belong in the Calendar period sections and/or **Overdue**, not here. Priority is the undated raised-priority backlog. Note: _this is likely to be very slow to generate, as it can't use any of NotePlan's internal caches, and doesn't have a natural way to limit it, like the Overdue section._
279+
This finds all **unscheduled** open items with a priority set (with `>>`, `!!!`, `!!` and `!` markers). Items that already have a `>date` (or `>today`) are left out on purpose: they belong in the Calendar period sections and/or **Overdue**, not here. Priority is the undated raised-priority backlog. Note: _a full vault scan is slow because NotePlan has no Priority API like `listOverdueTasks`_. From **2.5.0.b1** Dashboard keeps a local **Priority note-index cache** (similar to the Tag cache) so warm refreshes only open candidate notes; the first run after install may still do a full scan and then build the cache in the background.
280280

281281
David's advice is: "Priority tasks float to the tops of their individual sections already. And I go through all overdue tasks and handle them so that section stays small after you bite the bullet and do it once."
282282

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
// @flow
2+
/* eslint-disable flowtype/require-valid-file-annotation */
3+
/* globals describe, it, expect */
4+
5+
import { noteHasOpenUnscheduledPriorityItems, noteMayContainPriorityMarkers } from '../src/priorityNoteIndexCache'
6+
import { asTNote } from '@mocks/index'
7+
8+
/**
9+
* Build a minimal TNote for membership tests.
10+
* @param {string} content
11+
* @param {Array<any>} paragraphs
12+
* @returns {TNote}
13+
*/
14+
function mockNote(content: string, paragraphs: Array<any> = []): TNote {
15+
return asTNote({
16+
type: 'Notes',
17+
filename: 'test.md',
18+
content,
19+
paragraphs,
20+
})
21+
}
22+
23+
/**
24+
* @param {string} content
25+
* @param {string} type
26+
* @returns {any}
27+
*/
28+
function mockPara(content: string, type: string = 'open'): any {
29+
return {
30+
type,
31+
content,
32+
rawContent: `* ${content}`,
33+
}
34+
}
35+
36+
describe('noteMayContainPriorityMarkers', () => {
37+
it('returns true when content has ! or >>', () => {
38+
expect(noteMayContainPriorityMarkers(mockNote('task ! later'))).toBe(true)
39+
expect(noteMayContainPriorityMarkers(mockNote('>> win'))).toBe(true)
40+
})
41+
42+
it('returns false when content has neither', () => {
43+
expect(noteMayContainPriorityMarkers(mockNote('plain open task'))).toBe(false)
44+
})
45+
})
46+
47+
describe('noteHasOpenUnscheduledPriorityItems', () => {
48+
it('returns true for open unscheduled priority para', () => {
49+
const note = mockNote('! important', [mockPara('! important', 'open')])
50+
expect(noteHasOpenUnscheduledPriorityItems(note)).toBe(true)
51+
})
52+
53+
it('returns false for done priority para', () => {
54+
const note = mockNote('! done', [mockPara('! done', 'done')])
55+
expect(noteHasOpenUnscheduledPriorityItems(note)).toBe(false)
56+
})
57+
58+
it('returns false for open para without priority marker', () => {
59+
const note = mockNote('no marker', [mockPara('no marker', 'open')])
60+
expect(noteHasOpenUnscheduledPriorityItems(note)).toBe(false)
61+
})
62+
63+
it('returns false for scheduled priority para (belongs in Calendar/Overdue)', () => {
64+
// isOpenNotScheduled rejects paras with >date when type is open
65+
const note = mockNote('! soon >2024-01-01', [
66+
{
67+
type: 'open',
68+
content: '! soon >2024-01-01',
69+
rawContent: '* ! soon >2024-01-01',
70+
},
71+
])
72+
expect(noteHasOpenUnscheduledPriorityItems(note)).toBe(false)
73+
})
74+
})

‎jgclark.Dashboard/docs/ARCHITECTURE-How_Stuff_Works.md‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -237,10 +237,45 @@ Additionally, synthetic sections (e.g. `WINS`) are stripped from pluginData in t
237237
### Refresh cadence
238238
239239
- **Update** (`updateTagMentionCache`): notes changed since last run (~**1 hour** threshold via `updateTagMentionCacheIfTooOld`).
240-
- **Full regenerate** (`generateTagMentionCache`): ~**24 hours** or when new wanted items appear / scheduled via preference `regenerateTagMentionCachePref`.
240+
- **Full regenerate** (`generateTagMentionCache`): ~**5 days** or when new wanted items appear / scheduled via preference `regenerateTagMentionCachePref`.
241241
- Plugin install/update may force a full rebuild (`index.js` `onUpdateOrInstall`).
242242
243243
244+
## Priority note-index cache (`priorityNoteIndexCache.js`)
245+
246+
The Priority section has no NotePlan API equivalent of `DataStore.listOverdueTasks()`, so discovery historically walked every non-special project note plus past calendar notes (~11s on a large vault for a few dozen hits). From **2.5.0.b1**, Dashboard keeps a **plugin-local filename index** of notes that contain at least one **open, unscheduled, priority>0** paragraph (same membership rule as `getOpenPriorityItems` in `dataGenerationPriority.js`).
247+
248+
Implementation: `src/priorityNoteIndexCache.js`. Consumed by `getRelevantPriorityTasks()` when Priority cache is enabled (default -- **on** unless `FFlag_UsePriorityCache: false` in top-level `dashboardSettings`).
249+
250+
### Cache file
251+
252+
| File | Role |
253+
|------|------|
254+
| `priorityNoteIndexCache.json` | Filenames only: `regularNotes`, `calendarNotes`, plus `generatedAt` / `lastUpdated` (ISO UTC) and `version` |
255+
256+
Paragraphs are **not** stored. On every Priority refresh the plugin re-opens candidate notes and re-runs `getOpenPriorityItems`, so false positives self-heal. False negatives are mitigated by incremental updates on changed notes and periodic full regenerate.
257+
258+
### Index vs query universe
259+
260+
| At index build / update | At Priority query |
261+
|-------------------------|-------------------|
262+
| All non-`@` regular notes | Existing folder filters (`includedFolders` / `excludedFolders`) via `filterParasByRelevantFolders` |
263+
| **All** calendar notes that match the membership rule | Restrict calendar candidates to `pastCalendarNotes()` (date start before today) |
264+
265+
Indexing calendar notes broadly avoids a midnight-only invalidation path when yesterday's note becomes "past." Folder filters are **not** baked into the index so one cache serves all perspectives.
266+
267+
### Cold / missing cache
268+
269+
If the cache file is missing or invalid, Priority falls back to the full vault scan (correct first paint) and **schedules** `generatePriorityNoteIndexCache` via preference. Scheduled rebuilds run fire-and-forget after `incrementallyRefreshSomeSections` / `batchReplaceSections` (same pattern as the tag cache), on the **main thread** with loading/banner feedback.
270+
271+
### Refresh cadence
272+
273+
- **Update** (`updatePriorityNoteIndexCache`): notes changed since last run (~**1 hour** via `updatePriorityNoteIndexCacheIfTooOld`; 5s debounce).
274+
- **Full regenerate** (`generatePriorityNoteIndexCache`): ~**5 days**, or when scheduled / commanded (`generatePriorityNoteIndexCache` / `updatePriorityNoteIndexCache`).
275+
276+
Diagnostics (`generateDiagnosticsFile`) include a **Priority Note-Index Cache** section.
277+
278+
244279
## Section refresh functions (`refreshClickHandlers.js`)
245280
246281
There are three related backend functions that re-generate a subset of Sections and push them to the React window. They differ in **merge vs replace**, **how many `setPluginData()` calls (and therefore React redraws)** they trigger, and **when they are the right tool**. Main Window Reload is not one of these: it calls `restartDashboard()` which re-runs `showDashboardReact()`.

‎jgclark.Dashboard/plugin.json‎

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@
77
"plugin.iconColor": "red-600",
88
"plugin.description": "A Dashboard for NotePlan, that in one place can show:\n- a compact list of open tasks and checklists from today's note, and scheduled ones from other notes.\n- similarly for yesterday's note, tomorrow's note, and the weekly, monthly and quarterly notes too (if used)\n- open Apple Reminders\n- any timeblocks for today\n- all overdue tasks\n- all open tasks and checklists that contain particular @tags or #mentions of your choosing\n- the next notes ready to review (if you use the 'Projects and Reviews' plugin).\nIt includes many other ways of speeding up managing your tasks: see the website for more details.",
99
"plugin.author": "@jgclark",
10-
"plugin.version": "2.4.2",
11-
"plugin.releaseStatus": "full",
10+
"plugin.version": "2.5.0.b1",
11+
"plugin.releaseStatus": "beta",
1212
"plugin.hidden": false,
13-
"plugin.lastUpdateInfo": "2.4.2: Further extending edit dialog and interactive processing for Reminders.\n2.4.1: New Edit dialog and Interactive Processing for Apple Reminders. Quicker response when changing Perspective. Fix IP/task dialog error banner when Update/Enter with unchanged content (#778). Demo timing and Timed Items display fixes.\n2.4.0: New 'Reminders' Section, and addition of Apple Reminders into other Sections.\nNew 'Active Projects' Section.\nNew 'Wins' Section.\nNew 'Spaces to Include' setting which controls which (Team)Spaces you wish to include, plus whether or not to include the Private 'Space' (all notes not in a Space)\nSignificant performance improvements on refresh and for Tag/Mention sections.\nDozens of other improvements and fixes.\n2.3.3: new 'Year' section available.\n2.3.2: fix display when there are no priority items shown.\n2.3.1: fix for possible loss of settings error when upgrading.\n2.3.0: Support for NotePlan (Team)Spaces. Can re-order display of Sections.New '/backupSettings' command. Added 'noteTags' feature. Speeded up Tag/Mention sections. Layout improvements. Lots of other small fixes and improvements.\n2.2.1: Add new sorting option for Tag and Overdue sections.\n2.2.0: Add 'Search' section. New keyboard shortcuts. Plus many small improvements, bug fixes and performance improvements. See documentation for details.",
13+
"plugin.lastUpdateInfo": "2.5.0.b1: Priority note-index cache speeds up the Priority section (avoids full vault scan on warm refreshes). Interactive Processing for Active/Review Projects. Shared section-row CSS.\n2.4.2: Further extending edit dialog and interactive processing for Reminders.\n2.4.1: New Edit dialog and Interactive Processing for Apple Reminders. Quicker response when changing Perspective. Fix IP/task dialog error banner when Update/Enter with unchanged content (#778). Demo timing and Timed Items display fixes.\n2.4.0: New 'Reminders' Section, and addition of Apple Reminders into other Sections.\nNew 'Active Projects' Section.\nNew 'Wins' Section.\nNew 'Spaces to Include' setting which controls which (Team)Spaces you wish to include, plus whether or not to include the Private 'Space' (all notes not in a Space)\nSignificant performance improvements on refresh and for Tag/Mention sections.\nDozens of other improvements and fixes.\n2.3.3: new 'Year' section available.\n2.3.2: fix display when there are no priority items shown.\n2.3.1: fix for possible loss of settings error when upgrading.\n2.3.0: Support for NotePlan (Team)Spaces. Can re-order display of Sections.New '/backupSettings' command. Added 'noteTags' feature. Speeded up Tag/Mention sections. Layout improvements. Lots of other small fixes and improvements.\n2.2.1: Add new sorting option for Tag and Overdue sections.\n2.2.0: Add 'Search' section. New keyboard shortcuts. Plus many small improvements, bug fixes and performance improvements. See documentation for details.",
1414
"plugin.dependencies": [],
1515
"plugin.requiredFiles": [
1616
"react.c.WebView.bundle.dev.js"
@@ -174,6 +174,15 @@
174174
"utc"
175175
]
176176
},
177+
{
178+
"hidden": false,
179+
"name": "generatePriorityNoteIndexCache",
180+
"description": "generate Priority note-index cache for Dashboard",
181+
"jsFunction": "generatePriorityNoteIndexCache",
182+
"alias": [
183+
"gpic"
184+
]
185+
},
177186
{
178187
"name": "Make Callback from Current Settings",
179188
"description": "Make a callback url or link for the current settings, and copy to the clipboard.",
@@ -284,6 +293,15 @@
284293
"alias": [
285294
"utc"
286295
]
296+
},
297+
{
298+
"hidden": false,
299+
"name": "updatePriorityNoteIndexCache",
300+
"description": "update Priority note-index cache for Dashboard",
301+
"jsFunction": "updatePriorityNoteIndexCache",
302+
"alias": [
303+
"upic"
304+
]
287305
}
288306
],
289307
"plugin.commands.unused": [
@@ -390,4 +408,4 @@
390408
"title": "All the other Dashboard Settings live in the Dashboard's settings screen, accessed by clicking the gear icon in the top right of the Dashboard."
391409
}
392410
]
393-
}
411+
}

‎jgclark.Dashboard/src/dashboardSettingsClean.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ export function isTagCacheEnabled(dashboardSettings: TDashboardSettingsIn): bool
1818
return dashboardSettings?.FFlag_UseTagCache !== false
1919
}
2020

21+
/** Priority note-index cache is used unless FFlag_UsePriorityCache is explicitly false in dashboardSettings. */
22+
export function isPriorityCacheEnabled(dashboardSettings: TDashboardSettingsIn): boolean {
23+
return dashboardSettings?.FFlag_UsePriorityCache !== false
24+
}
25+
2126
/**
2227
* Build strip patterns for keys that belong in top-level dashboardSettings only (not in perspective defs).
2328
* @param {boolean} deleteAllShowTagSections

‎jgclark.Dashboard/src/dataGenerationPriority.js‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ import {
1818
isWinItem,
1919
makeDashboardParas,
2020
} from './dashboardHelpers'
21+
import { isPriorityCacheEnabled } from './dashboardSettingsClean'
22+
import { getNotesFromPriorityNoteIndexCache, schedulePriorityNoteIndexCacheGeneration } from './priorityNoteIndexCache'
2123
import { stringListOrArrayToArray } from '@helpers/dataManipulation'
2224
import { clo, JSP, logDebug, logError, logInfo, logTimer, logWarn, timer } from '@helpers/dev'
2325
import { getRegularNotesFromFilteredFolders } from '@helpers/folders'
@@ -163,9 +165,25 @@ async function getRelevantPriorityTasks(config: TDashboardSettings): Promise<Arr
163165
// const includedFolders = config.includedFolders ? stringListOrArrayToArray(config.includedFolders, ',').map((folder) => folder.trim()) : []
164166
const excludedFolders = config.excludedFolders ? stringListOrArrayToArray(config.excludedFolders, ',') : []
165167
logInfo('getRelevantPriorityTasks', `excludedFolders: ${String(excludedFolders)}`)
166-
// Reduce list to all notes that are not blank or in @ folders or excludedFolders
167-
let notesToCheck = getRegularNotesFromFilteredFolders(excludedFolders, true).concat(pastCalendarNotes())
168-
logTimer('getRelevantPriorityTasks', thisStartTime, `- Reduced to ${String(notesToCheck.length)} non-special regular notes + past calendar notes to check`)
168+
169+
let notesToCheck: Array<TNote> = []
170+
const usePriorityCache = isPriorityCacheEnabled(config)
171+
if (usePriorityCache) {
172+
const cachedNotes = await getNotesFromPriorityNoteIndexCache()
173+
if (cachedNotes != null) {
174+
notesToCheck = cachedNotes
175+
logTimer('getRelevantPriorityTasks', thisStartTime, `- from PRIORITY CACHE: ${String(notesToCheck.length)} candidate notes`)
176+
} else {
177+
logInfo('getRelevantPriorityTasks', `- Priority cache unavailable; falling back to full vault scan (generation scheduled for later)`)
178+
schedulePriorityNoteIndexCacheGeneration()
179+
notesToCheck = getRegularNotesFromFilteredFolders(excludedFolders, true).concat(pastCalendarNotes())
180+
logTimer('getRelevantPriorityTasks', thisStartTime, `- Reduced to ${String(notesToCheck.length)} non-special regular notes + past calendar notes to check`)
181+
}
182+
} else {
183+
// Reduce list to all notes that are not blank or in @ folders or excludedFolders
184+
notesToCheck = getRegularNotesFromFilteredFolders(excludedFolders, true).concat(pastCalendarNotes())
185+
logTimer('getRelevantPriorityTasks', thisStartTime, `- Reduced to ${String(notesToCheck.length)} non-special regular notes + past calendar notes to check`)
186+
}
169187

170188
// Note: PDF and other non-notes are contained in the directories, and returned as 'notes' by `DataStore.projectNotes` (the call behind 'regularNotesFromFilteredFolders').
171189
// Some appear to have 'undefined' content length, but I had to find a different way to distinguish them.

‎jgclark.Dashboard/src/diagnosticGenerator.js‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import {
1414
import { logPerspectives, logPerspectiveNames, getActivePerspectiveName, loadPerspectiveDefsFromPluginSettings } from './perspectiveHelpers'
1515
import { getCurrentlyAllowedFolders } from './perspectivesShared'
1616
import { getTagMentionCacheDiagnosticsLines } from './tagMentionCache'
17+
import { getPriorityNoteIndexCacheDiagnosticsLines } from './priorityNoteIndexCache'
1718
import type { TPerspectiveDef } from './types'
1819
import { clo, JSP, logDebug, logError, logInfo, logTimer, logWarn, timer } from '@helpers/dev'
1920
import { createPrettyRunPluginLink } from '@helpers/general'
@@ -91,6 +92,9 @@ export async function generateDiagnosticsFile(refreshArg: string = '') {
9192
output.push('## Tag/Mention Cache')
9293
output.push(...getTagMentionCacheDiagnosticsLines(ds))
9394
output.push('')
95+
output.push('## Priority Note-Index Cache')
96+
output.push(...getPriorityNoteIndexCacheDiagnosticsLines(ds))
97+
output.push('')
9498
output.push(`## Perspectives`)
9599
output.push(`Current Perspective = **${getActivePerspectiveName(perspectiveDefs)}**`)
96100
output.push(`- Enabled sections: ${String(getListOfEnabledSections(ds)) || 'none?'}`)

‎jgclark.Dashboard/src/index.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ export {
7474
} from './NPHooks'
7575

7676
export { generateTagMentionCache, updateTagMentionCache } from './tagMentionCache'
77+
export { generatePriorityNoteIndexCache, updatePriorityNoteIndexCache } from './priorityNoteIndexCache'
7778

7879
export { updateDoneCountsFromChangedNotes, logDoneCounts } from './countDoneTasks'
7980

0 commit comments

Comments
 (0)