Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -25,19 +25,19 @@ function getChartData(metric: DependencyMetric) {
---

<ComparisonBarChart
title="Metaframework Dependencies"
title="Core Package Dependencies"
data={getChartData('allDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
/>
<ComparisonBarChart
title="Metaframework Development Dependencies"
title="Core Package Development Dependencies"
data={getChartData('devDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
/>
<ComparisonBarChart
title="Metaframework Production Dependencies"
title="Core Package Production Dependencies"
data={getChartData('prodDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ const frameworkDepsColumns = [
---

<StatsTable
label="Metaframework package dependency counts"
label="Core package dependency counts"
columns={frameworkDepsColumns}
data={frameworkDepsStats}
/>
Original file line number Diff line number Diff line change
Expand Up @@ -28,25 +28,25 @@ function getDependencyChartData(metric: DependencyMetric) {
---

<ComparisonBarChart
title="Meta-framework Direct Package Dependencies"
title="First-Party Dependencies"
data={getDependencyChartData('allDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
/>
<ComparisonBarChart
title="Meta-framework Direct Package Development Dependencies"
title="First-Party Development Dependencies"
data={getDependencyChartData('devDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
/>
<ComparisonBarChart
title="Meta-framework Direct Package Production Dependencies"
title="First-Party Production Dependencies"
data={getDependencyChartData('prodDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
/>
<ComparisonBarChart
title="Meta-framework Direct Package Duplicate Dependencies"
title="First-Party Duplicate Dependencies"
data={getDependencyChartData('duplicateDependencies')}
valueFormat="count"
yAxisLabel="Dependency count"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const firstPartyDependencyColumns = [
---

<StatsTable
label="First-party direct package dependency counts by framework"
label="First-party dependency counts by meta-framework"
columns={firstPartyDependencyColumns}
data={firstPartyDepsStats}
/>
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const depsColumns = [
---

<StatsTable
label="Dependency counts by framework"
label="Dependency counts by starter project"
columns={depsColumns}
data={depsStats}
/>
Original file line number Diff line number Diff line change
@@ -1,4 +1,14 @@
Methodology: [Dependency Counts](/methodology/#dependency-counts).

These views separate a framework's core package, the direct packages maintained by
its ecosystem, and the complete default starter configuration.
Measuring a meta-framework's dependencies isn't as straightforward as you might think. You could:

- Measure just the meta-framework package itself
- Maybe a full project setup

Both approaches have flaws. Some meta-frameworks need many other packages to work. A full project setup can include more packages than necessary.

To provide a clearer picture, we run the dependency-count tests three times, starting small and expanding in scope. In this order, we:

1. Start with the meta-framework package itself, for example, `@sveltejs/kit`
2. We then expand this to the meta-framework and its first-party packages. Looking at `@sveltejs/kit` again, this means any package maintained by the meta-framework authors that's needed to run a project, for example, the `svelte` package.
3. Finally, we count the dependencies for the entire `starter-*` project. Their CLI or documentation recommends this setup by default, so this is the default most people will start with.
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Next, we look at the direct framework-maintained packages that each starter
declares. This excludes the starter's third-party setup packages.
Looking at a meta-framework’s first-party packages. For `@sveltejs/kit`, this means packages maintained by the meta-framework authors that are needed to run a project, for example, `svelte`.
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
First, we look at the direct dependency counts of the meta-framework package
itself.
The meta-framework package itself, for example, `@sveltejs/kit`.
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
import { Content as Description } from './FrameworkDescription.mdx'
import FrameworkDependencyStatsTable from '../../FrameworkDependencyStatsTable.astro'
import FrameworkDependencyCharts from '../../FrameworkDependencyCharts.astro'
import CorePackageDependencyStatsTable from '../../CorePackageDependencyStatsTable.astro'
import CorePackageDependencyCharts from '../../CorePackageDependencyCharts.astro'
---

<Description />
<FrameworkDependencyStatsTable />
<FrameworkDependencyCharts />
<CorePackageDependencyStatsTable />
<CorePackageDependencyCharts />
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
import { Content as Description } from './StarterDescription.mdx'
import DependencyStatsTable from '../../DependencyStatsTable.astro'
import DependencyCharts from '../../DependencyCharts.astro'
import StarterDependencyStatsTable from '../../StarterDependencyStatsTable.astro'
import StarterDependencyCharts from '../../StarterDependencyCharts.astro'
---

<Description />
<DependencyStatsTable />
<DependencyCharts />
<StarterDependencyStatsTable />
<StarterDependencyCharts />
9 changes: 5 additions & 4 deletions packages/docs/src/components/setup/DevTimeSetup.mdx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Each starter project follows its framework's recommended CLI or documented setup.
See [Default Benchmark Setup](/methodology/#default-benchmark-setup) for the
shared environment, build tools, and adapters, and [Project
Setups](/methodology/#project-setups) for the starter configuration details.
Dev Time benchmarks use each meta-framework’s `starter-*` package.

Each starter follows its framework’s CLI or documented setup, with the selected options and adjustments recorded in the [methodology](/methodology/#project-setups).

See [Default Benchmark Setup](/methodology/#default-benchmark-setup) for the shared benchmark environment.
6 changes: 3 additions & 3 deletions packages/docs/src/content/docs/all-frameworks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,15 @@ import CoreWebVitalsResults from '../../components/benchmarks/core-web-vitals/Re

<DependencyCountsDescription />

#### Meta-framework Direct Package Counts
#### Core Package Dependencies

<FrameworkDependencyResults />

#### Meta-framework Direct Packages
#### First-Party Package Dependencies

<FirstPartyDependencyResults />

#### Starter Projects
#### Starter Project Dependencies

<StarterDependencyResults />

Expand Down
6 changes: 3 additions & 3 deletions packages/docs/src/content/docs/dev-time.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ import MinimumNodeVersionCompare from '../../components/benchmarks/minimum-node-

<DependencyCountsDescription />

### Meta-framework Direct Package Counts
### Core Package Dependencies

<FrameworkDependencyResults />

### Meta-framework Direct Packages
### First-Party Package Dependencies

<FirstPartyDependencyResults />

### Starter Projects
### Starter Project Dependencies

<StarterDependencyResults />

Expand Down
10 changes: 5 additions & 5 deletions packages/docs/src/content/docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ If you find an issue with a test or have an idea for new ones, please open an is

## What we measure and how

We currently split our tests into two sections: Dev Time and Run Time. Dev Time tests use a starter project set up with the Metaframeworks CLI or Docs, following the default or recommended setup. Runtime tests use a small app project set up for the measurements. We currently measure:
We currently split our tests into two sections: Dev Time and Run Time. Dev Time tests use a starter project set up with the meta-framework’s CLI or documentation, following the default or recommended setup. Runtime tests use a small app project set up for the measurements. We currently measure:

### DevTime
### Dev Time

- Dependancy counts
- Dependency counts
- Node Modules Size
- Build and Install Times
- Dev Server Startup
- Core-JS Polyfills
- Browser Baseline
- Minimum Node Version

### RunTime
### Run Time

- Lighthouse Client-Side Rendered
- Lighthouse Server-Side Rendered
Expand Down Expand Up @@ -64,6 +64,6 @@ We measure the following frameworks using the versions listed below.

## Additional frameworks

We also track the frameworks below. Their results appear on the All Tests page, but are excluded from the DevTime and RunTime stats.
We also track the frameworks below. Their results appear on the All Tests page, but are excluded from the Dev Time and Run Time stats.

<AdditionalFrameworkCards />
5 changes: 4 additions & 1 deletion packages/docs/src/content/docs/methodology.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,10 +220,13 @@ Installed using the CLI with the following setup:

### Dependency Counts

- The core package view counts the direct production and development dependencies
declared by the meta-framework package itself.

- The starter project view counts every direct production and development
dependency in the starter's `package.json`, then reports its resolved graph,
duplicates, and install size.
- The meta-framework direct packages view uses the starter's
- The first-party package view uses the starter's
`first-party-dependencies.json`. It keeps only the direct packages maintained
by the framework's ecosystem, preserving their production/development section
and version from the starter's `package.json`. Its resolved graph and duplicate
Expand Down
6 changes: 3 additions & 3 deletions packages/docs/src/lib/metrics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ export const metricCatalogue = {
},
'frameworkDependencies.prodDependencies': {
id: 'frameworkDependencies.prodDependencies',
title: 'Metaframework \u00b7 Production Dependencies',
title: 'Core Package \u00b7 Production Dependencies',
valueFormat: 'count',
yAxisLabel: 'Dependencies',
},
'frameworkDependencies.devDependencies': {
id: 'frameworkDependencies.devDependencies',
title: 'Metaframework \u00b7 Development Dependencies',
title: 'Core Package \u00b7 Development Dependencies',
valueFormat: 'count',
yAxisLabel: 'Dependencies',
},
'frameworkDependencies.allDependencies': {
id: 'frameworkDependencies.allDependencies',
title: 'Metaframework \u00b7 All Dependencies',
title: 'Core Package \u00b7 All Dependencies',
valueFormat: 'count',
yAxisLabel: 'Dependencies',
},
Expand Down
Loading