⛔ [DEPRECATED] Active at https://github.com/valentin-schwind/study-power-analysis

Study Design Planner

The Study Design and Statistical Test Planner is part of the HCI User Studies Toolkit. It helps students and novice researchers sketch study designs, estimate plausible sample sizes, and inspect suitable statistical procedures for common HCI study setups.

Running the Tool

The published version is available at hci-studies.org/study-design-planner.

The planner is now able to run its core estimation workflow directly in the browser. A static deployment on GitHub Pages or any local web server is sufficient for:

• initial sample size estimation,
• ANOVA power estimation for supported factorial nominal designs,
• regression power estimation for supported continuous-predictor scenarios.

The frontend uses:

• Bootstrap for layout and components,
• jQuery and jQuery UI for interactions,
• a local JavaScript power engine in _js/power-engine.js.

Statistical Approach

Update: the ANOVA engine now uses a Superpower-inspired analytic approximation instead of the older browser simulation path. It still remains an educational planning aid, not a numerically exact port of Superpower, because the planner UI does not expose full user-defined mu vectors or full correlation matrices.

The browser-side power engine is intentionally approximate. It is designed for educational planning support and preserves the planner’s existing UI flow rather than reproducing Superpower exactly.

Supported design classes

• between-subject factorial designs with nominal IVs,
• repeated-measures factorial designs with nominal IVs,
• mixed nominal designs with both between- and within-subject factors,
• regression scenarios with one or more non-nominal predictors and one continuous outcome.

ANOVA approximation

Current implementation note:

1. The planner builds a balanced factorial design from the current nominal IV setup.
2. It generates a linear cell-mean pattern in a fixed within-first priority order.
3. It adds a small orthogonal interaction component so higher-order terms are not forced to exactly zero.
4. It derives main and interaction components through projection matrices.
5. It computes Cohen's f, partial eta-squared, denominator degrees of freedom, and a conservative noncentrality parameter for each effect.
6. It approximates power from those quantities with a browser-side noncentral chi-square / F-test approximation.

The current implementation uses a balanced-design approximation for factorial effects. It is stable and monotonic enough for planning, but it is not intended to be a drop-in numerical replacement for Superpower or a full repeated-measures modelling package.

Regression approximation

The regression path uses an analytic multiple-regression approximation:

1. It derives a target effect size f² from the current delta and pooled SD controls using the planner's legacy f = d / sqrt(2) mapping.
2. It treats the current regression scenario as an overall model F-test with u = predictors and v = N - predictors - 1.
3. It estimates power with a browser-side noncentral F-test approximation for the overall model.

Sample size estimation

Current implementation note:

• Sample size is searched iteratively over balanced designs.
• The search targets about 80% power for the strongest main effect under the current cell-mean pattern.
• The result is refined with a binary search over balanced sample sizes.
• Regression sample size is searched iteratively against the same analytic overall-model F-test used for the reported regression power.

If a scenario contains both nominal factors and regression-style predictors, the planner takes the larger of the locally estimated sample sizes so that both guidance paths stay usable.

Limitations

• Results can still differ from the previous R/OpenCPU workflow and from Superpower, especially when a study would be better described by explicit user-provided cell means.
• The local engine is designed for plausible, stable educational estimates, not exact inferential planning.
• Mixed and repeated-measures designs are handled with a defensible approximation, not a full general linear mixed-model implementation.
• The planner assumes balanced designs for power estimation.
• The generated R analysis suggestions in the UI are guidance text and remain separate from the JavaScript estimation engine.

Repository Structure

The repository still contains the historical R package sources in R/ and R-Package/powerAnalysis/, but the planner no longer requires OpenCPU for its default core estimation workflow.

Relevant parts:

• index.html: main UI and interaction logic
• _js/power-engine.js: browser-side power estimation engine
• _css/, _img/, _fonts/: static site assets
• R-Package/powerAnalysis: legacy server-side package kept for reference and compatibility work

Local Development

Because the planner is now static for its core functionality, a simple local web server is enough. For example:

python -m http.server

or any comparable static file server.

Opening the page through a web server is still recommended so that browser security behavior matches the deployed site more closely.

Citation

If you use the toolkit in academic work, please cite:

@inproceedings{schwind2023a,
  author = {Schwind, Valentin and Resch, Stefan and Sehrt, Jessica},
  title = {The HCI User Studies Toolkit: Supporting Study Designing and Planning for Undergraduates and Novice Researchers in Human-Computer Interaction},
  year = {2023},
  isbn = {978-1-4503-9422-2/23/04},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3544549.3585890},
  doi = {10.1145/3544549.3585890},
  booktitle = {Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems},
  location = {Hamburg, Germany},
  series = {CHI EA '23}
}