08-data-visualization.Rmd

```{r eval = FALSE, include = FALSE}
source("setup.r")
```

# Data Visualization

```{r dataviz_setup, include=F}
knitr::opts_chunk$set(dev = "svg",
                      fig.align = "center")
```

Look at your data!

Yes, that means look at your raw data by producing subsets, printing to the console, and all the other things we've been doing. But it also means that we should visualize our data.

The potential purposes for doing so are legion, but we can name a few for starters:

- To detect errors (e.g., miscoded observations)
- To detect missing data
- To detect your own coding errors (e.g., miscoded derivative variables)
- To see the variation of single variables and the covariation between pairs of variables
- To summarize a lot of quantitative information so that it can be digested by the reader quickly and efficiently
- To reveal relationships between exposure and outcome
- To diagnose statistical or computational procedures (e.g., regressions, algorithms)
- To make principled decisions about how to analyze your data

Before we continue, though, we should keep one thing in mind: **data visualization is not about making pretty pictures**. Saying so does not discount the importance of aesthetics in your visualizations. However, the aesthetics you employ should be chosen to make the message of the figure clear to your audience. In some cases, we might want to highlight a particular contrast between two variables. In others, we might want to visualize how a particular variable trends over time. In yet others, we may simply want to visualize features in our raw dataset.

Each of these objectives implies the usefulness of certain visualizations and the impropriety of others. In other words, we want the purpose of the visualization to dictate the means by which we display relevant data.

In this introduction, we will focus on a couple of broad concepts and code examples in order to get you up and running as quickly as possible. However, I highly recommend two (freely available) resources in particular:

1. [Data Visualization: A Practical Introduction](https://socviz.co/) by Kieran Healy

2. [Top 50 ggplot2 Visualizations - The Master List (with Full R Code)](http://r-statistics.co/Top50-Ggplot2-Visualizations-MasterList-R-Code.html) by Selva Prabhakaran

Keeping these bookmarked will probably cover about 90%^[Source: Thin air] of your data visualization needs.

## Data set

All the examples below use the NHEFS data we used in the chapter on statistical analysis, so let's remind ourselves what the data look like.

```{r iris-preview}
nhefs <- readr::read_csv("data/nhefs.csv")
str(nhefs)
```

## Base R examples

You may find that working with plotting in base R is more efficient when you want quick-and-dirty visualizations---for instance, while you're conducting exploratory data analysis.

### Boxplots

```{r base-r-bp, out.width="75%"}
# using built-in iris dataset
boxplot(nhefs$sbp)
boxplot(sbp ~ diabetes, data = nhefs)
```

### Density Plots

```{r base-r-density, out.width="75%"}
## use na.omit() to drop missing values in the `sbp` variable
## R will throw an error otherwise
plot(density(na.omit(nhefs$sbp)))
```

### Scatterplots

```{r base-r-scatter, out.width="75%"}
plot(nhefs$sbp)
plot(nhefs$sbp ~ nhefs$age)
```

### Scatterplot Matrix

```{r base-r-scattermatrix, out.width="100%", fig.asp=1}
matrix_vars <- c("qsmk", "sbp", "dbp", "age", "smokeyrs", "wt71")
pairs(nhefs[, matrix_vars])
```

### Histograms

```{r base-r-hist, out.width="75%"}
hist(nhefs$smokeyrs)
```

### Linear Regression Diagnostics

Base R and some packages provide built-in plotting methods for certain types of objects. Here, feeding an object of the class `lm` will return a series of linear regression diagnostics.

```{r base-linear-reg-diag, out.width="85%"}
## run a regression on built-in dataset ChickWeight
fit <- lm(wt82_71 ~ smokeyrs, data = nhefs)
class(fit)

plot(fit)
```

If you run `plot(fit)` as above, you might find R irritating in that it will print these plots to your image device (image pane in RStudio) in succession, leaving you to flip between images. For a nicer output, you can output a grid of regression diagnostics.

```{r base-linear-reg-diag-grid, out.width="85%", fig.asp=1}
par(mfrow = c(2, 2))
plot(fit)
``` 

## ggplot2 Examples

Before we start, let's load a few of the packages we'll be using. 

```{r ggplot-loads}
library(ggplot2)  # plotting package
library(ggthemes) # a package that adds plot themes to ggplot2
library(magrittr) # use pipes "%>%"
```

Before, we get into the examples, I'm going to apply a universal theme for the subsequent plots:

```{r ggplot-themeset}
theme_set(theme_few(base_size = 16))
```

The `ggplot2` package in R is an excellent package for data visualization based on the _grammar of graphics_ concept.^[Leland Wilkinson. [The Grammar of Graphics](https://link.springer.com/book/10.1007/0-387-28695-0). Springer, 2005.] The basic idea is to divide plot features into conceptual layers defined by their function in the visualization. 

Here's a general schema for how this concept maps to plot components in `ggplot2`:

- **data:** the underlying data set and variables we wish to convert into plot aesthetics
- **plot layer:** geometric entities (_e.g._, shapes, lines), statistical transformations, positioning elements
- **scales:** scales applied to geoms to map additional data dimensions (_e.g._, a color gradient)
- **coordinates:** the coordinate system used to plot the data (_e.g._, Cartesian, polar)
- **facets:** specifications that control a grid of plots, showing the same plot within multiple discrete groupings

Note that I've assigned names to each element for the sake of introduction, but the descriptions themselves follow closely from Hadley Wickham's (`ggplot2` creator) own in the paper referenced here.^[Hadley Wickham (2010) A Layered Grammar of Graphics, _Journal of Computational and Graphical Statistics_, 19:1, 3-28, DOI: [10.1198/jcgs.2009.07098](https://doi.org/10.1198/jcgs.2009.07098)]

In the following examples I will focus on longform syntax with the `ggplot` function. Learning the ins and outs of `ggplot2` will give you more flexibility and control over your images, particularly when you need to produce publication-quality graphics. However, `ggplot2` provides the `qplot` function for quick visualizations, akin to the base R graphics depicted above. One nice thing about `qplot` the its default setting to choose a geometric mapping automatically (hereafter, "geom"). Below, the function automatically selects a scatterplot.

```{r ggplot-qplot}
qplot(x = age, y = sbp, data = nhefs)
```

However, we can specify the geom explicitly. Here we get a smoothed line:

```{r ggplot-qplot-geomspec}
qplot(x = age, y = sbp, data = nhefs, geom = "smooth")
```

The [`ggplot2` website](https://ggplot2.tidyverse.org/reference) contains extensive documentation on all of the package's functions, including many helpful worked examples.

<span class="marginnote warning">
**Strange Symbols**
Note the difference between the `magrittr` pipe `%>%`, which we use to feed our data set to the `ggplot()` function and the `+` symbol we use to bind together the lines describing plot elements. This distinction is mandatory, as `ggplot2` will not allow you to use the `magrittr` pipe to link plot layers.
</span>

### Boxplots

```{r ggplot-boxplot, fig.asp = 1}
nhefs %>%
  ggplot(aes(x = sbp)) +
  geom_boxplot()

nhefs %>%
  ggplot(aes(x = factor(diabetes), y = sbp)) +
  geom_boxplot()
```

### Density Plots

```{r ggplot-density-color, fig.asp = 0.8}
nhefs %>%
  ggplot(aes(x = sbp, color = factor(diabetes))) +
  geom_density()
```

### Scatterplots

If we want to see the relationship between age and systolic blood pressure, we can make a scatterplot.

```{r ggplot-scatterplot-marginal, fig.asp = 1, fig.width = 5}
nhefs %>%
  ggplot(aes(x = age, y = sbp)) +
  geom_point()
```

Perhaps this relationship varies by diabetes status. In that case we need to differentiate each point based on the `diabetes` variable. We have multiple options to do so. Two that make immediate sense would be to differentiate species by color or by shape.

```{r ggplot-scatterplot-grouped, fig.asp = 0.75, fig.width = 7}

nhefs %>%
  ggplot(
    aes(
      x = age,
      y = sbp,
      color = factor(diabetes)
    )
  ) +
  geom_point(size = 3)

nhefs %>%
  ggplot(
    aes(
      x = age,
      y = sbp,
      shape = factor(diabetes)
    )
  ) +
  geom_point(size = 3) +
  scale_shape_manual(values = c(21, 22, 23)) # choose some shapes (optional)
```

### Faceted Plots

Another option would have been to create a separate scatterplot for each species. In the plot below, each section of the grid is called a **facet**. We'll also save an initial (unfaceted) plot so we can play with the facet settings a bit.

```{r ggplot-scatter-facet, fig.asp = 0.5}

## labels to use for axis and facet plots
dblabels <- c("0" = "No", "1" = "Yes", "2" = "Missing")
sbplab <- "Systolic blood pressure"
agelab <- "Age"

sbp_facet <- nhefs %>%
  ggplot(aes(x = age, y = sbp)) +
  geom_point() +
  labs(x = agelab, y = sbplab)

sbp_facet +
  facet_wrap(
    vars(diabetes),
    labeller = labeller(diabetes = dblabels)
  )

```

If we wanted a single column, we could have asked `facet_wrap()` for one.

```{r ggplot-scatter-facet-1col, fig.asp = 3, fig.width = 3}
sbp_facet +
  facet_wrap(
    vars(diabetes),
    labeller = labeller(diabetes = dblabels),
    ncol = 1
  )
```

Finally, we also could have plotted the group-specific densities using facets instead of colors.

```{r ggplot-density-facet, fig.height = 3}
nhefs %>%
  ggplot(aes(x = sbp)) +
  geom_density() +
  facet_wrap(
    vars(diabetes),
    labeller = labeller(diabetes = dblabels)
  )
```

In practice, you will choose particular plot designs to illuminate the aspects of your data you want to highlight.


### Bar Plots

To demonstrate the difference between two functions, `geom_bar()` and `geom_col()`, we'll create a binary variable that encodes whether a given observation had a petal length less than 2 or at least 2.

```{r ggplot-bar}

library(dplyr)

### make discrete categories for Petal.Length
### only to demonstrate geom_bar and geom_col
nhefs_new <- nhefs %>%
  mutate(
    sbp.group = case_when(
      sbp >= 140 ~ "High",
      sbp < 140 ~ "Low",
      TRUE ~ NA_character_
    )
  )

head(nhefs_new)
```

The `geom_bar()` function will count up the number of members in each group. 

```{r ggplot-bar1, fig.asp = 1, fig.width = 5}

nhefs_new %>%
  ggplot(aes(x = sbp.group)) +
  geom_bar()
```

Let's make that a little prettier, shall we?

<span class="marginnote">The `NA` you see represented in the plots indicates missing values of systolic blood pressure.</span>

```{r ggplot-bar2, fig.asp = 1, fig.width = 5}

nhefs_new %>%
  ggplot(aes(x = sbp.group)) +
  geom_bar(
    fill = "lightgray",
    color = "black",
    width = 0.5
  )
```

Often you'll find that you'll be summarizing your data to make certain types of plots. Let's say we had already counted the number of members per `sbp.group` level ourselves, as so:

```{r ggplot-plgrp-counts}

nhefs_sum <- nhefs_new %>% count(sbp.group)
nhefs_sum
```

To create the preceding plot, we would use `geom_col()` instead of `geom_bar()`:

```{r ggplot-geom-col, fig.asp = 1, fig.width = 5}

nhefs_sum %>%
  ggplot(aes(x = sbp.group, y = n)) +
  geom_col(
    fill = "lightgray",
    color = "black",
    width = 0.5
  )
```

In many cases, you'll find more than one viable route to your chosen graphic.


### Scatterplot Matrix

Base R has a nice function to create scatterplot matrices. To do so with `ggplot2` we need to install another package called `GGally`. 

<span class="marginnote idea">
One wonderful thing about `ggplot2` is that it boasts a universe of extensions written by R users to achieve specific plotting goals. While learning the ins and outs of `ggplot2` is fun (for me, anyway), you may find [extensions](https://exts.ggplot2.tidyverse.org) useful.
</span>

```{r ggplot-ggallymatrix, message = F, warning = F, fig.asp = 1}
GGally::ggpairs(nhefs[, matrix_vars])
```

<span class="marginnote warning">
**Missing Data**
Note that I've suppressed a number of warnings produced by this `ggpairs` example. These warnings highlight that some of the variables used in the plot have varying numbers of missing values.
</span>

This visualization is nice because it gives you a lot of information in a small space. The bottom row even contains embedded facets!


## Special Topics
### Image Formats

Whether you're writing a manuscript, submitting final materials for publication, compiling internal reports, or developing materials for the web, you will need to pick an appropriate image format for your output.

The table below summarizes some of the more common image formats you'll come across and when they may be useful. Perhaps the most important distinction between formats is between **raster** and **vector** images. Raster formats represent images using pixels, while vector formats represent image components using paths. For practical purposes, the most important thing to remember is that vector formats are scalable without loss of resolution, while raster images aren't.

<span class="marginnote idea">
**Images in this Book**
The majority of images in this book are in SVG format, which means you can zoom in on them to your heart's content without the image becoming blurry! Right-click on any of the prior images on this page, open it in a new tab, and zoom in to see what I mean.
</span>

I try to use vector formats whenever possible, as **it is easy to convert a vector format to a raster format---_i.e.,_ to rasterize---but not the other way around**. Therefore, I usually find myself saving images in PDF or SVG format, depending on the type of document I'm working on. In some cases, particularly for the web, using a raster format such as PNG may be the best choice to reduce file size and increase page load speeds.

```{r image-format-table, echo = F}

pacman::p_load(data.table, magrittr, knitr, kableExtra)

rast <- data.table(
  Type = "Raster",
  Format = c("JPG", "PNG", "GIF", "TIFF"),
  Web = c("Yes", "Yes", "Yes", "No"),
  PDF = c("Yes", "Yes", "Yes", "Yes"),
  Word = c("Yes", "Yes", "Yes", "Yes"),
  Publication = c("No", "No", "No", "Yes")
)

vect <- data.table(
  Type = "Vector",
  Format = c("EPS", "PDF", "WMF", "SVG"),
  Web = c("No", "No", "No", "Yes"),
  PDF = c("Yes", "Yes", "No", "No"),
  Word = c("No", "No", "Yes", "No"),
  Publication = c("Yes", "Yes", "No", "No")
)

imgfmt <- rbind(rast, vect)

imgfmt[, -c("Type")] %>%
  kable(format = "html") %>%
  kable_paper(
    lightable_options = "hover",
    position = "left",
    full_width = F
  ) %>%
  add_header_above(
    header = c(" " = 1, "Suitable for" = 4),
    bold = T
  ) %>%
  pack_rows("Raster", 1, 4, italic = T) %>%
  pack_rows("Vector", 5, 8, italic = T)

```

Let's take a look at some examples to get an idea of the pros and cons of several formats.

First, let's make a plot to use in the following examples.

```{r fmt-base}
imgfmt_plot <- nhefs %>%
  ggplot(aes(x = sbp)) +
  geom_density(
    aes(color = factor(diabetes),
        fill = factor(diabetes)),
    alpha = 0.1
  ) +
  scale_color_viridis_d(name = "Diabetes", labels = dblabels) +
  scale_fill_viridis_d(name = "Diabetes", labels = dblabels) +
  guides(
    color = guide_legend(override.aes = list(linetype = 0))
  ) +
  labs(x = "\nSystolic blood pressure", y = "Density\n") +
  theme_few()
```

We'll proceed to save this plot in a few different formats and at different resolutions using the `ggsave()` function. You can open the images in a new tab to view them at their true size and resolution. After presenting to code to produce each one, I'll provide a table so you can easily download the relevant files. One nice thing about `ggsave()` is that it will automatically detect the export format based on the file extension we specify.

<span class="marginnote definition">
**Resolution**
Resolution refers either to print or screen resolution. DPI stands for "dots per inch", the measure journals will use to request publication-quality graphics for their print editions. PPI stands for "pixels per inch", a measure of the image's resolution on screen.
</span>

```{r png-72dpi, warning = F}
ggsave(
  filename = "images/fmt-example-png-72dpi.png",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in",
  dpi = 72
)

```

```{r png-300dpi, warning = F, fig.cap = "PNG, 6 x 4 inches, 300 dots per inch"}
ggsave(
  filename = "images/fmt-example-png-300dpi.png",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in",
  dpi = 300
)

```

```{r tif-1200dpi, warning = F}
ggsave(
  filename = "images/fmt-example-tiff-1200dpi.tiff",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in",
  dpi = 1200
)
```

For the PDF, EPS, and SVG formats, we do not need to specify the image resolution (`dpi`) because these are vector formats.

```{r pdf-fmt, warning = F}
ggsave(
  filename = "images/fmt-example-pdf.pdf",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in"
)
```

```{r eps-fmt, warning = F}
ggsave(
  filename = "images/fmt-example-eps.eps",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in"
)
```

```{r svg-fmt, warning = F}
ggsave(
  filename = "images/fmt-example-svg.svg",
  plot = imgfmt_plot,
  width = 6,
  height = 4,
  units = "in"
)
```

| Format | Resolution (dpi)               | Download                                             | Accepted by journals? |
|:-------|:-------------------------------|:-----------------------------------------------------|:----------------------|
| PNG    | 72 (web standard)              | [PNG 72dpi](images/fmt-example-png-72dpi.png)        | No                    |
| PNG    | 300                            | [PNG 300dpi](images/fmt-example-png-300dpi.png)      | No                    |
| TIFF   | 1200 (print quality for plots) | [TIFF 1200dpi](images/fmt-example-tiff-1200dpi.tiff) | Yes                   |
| PDF    | N/A                            | [PDF](images/fmt-example-pdf.pdf)                    | Yes (usually)         |
| EPS    | N/A                            | [EPS](images/fmt-example-eps.eps)                    | Yes                   |
| SVG    | N/A                            | [SVG](images/fmt-example-svg.svg)                    | No                    |

### Interactive Graphics

Most of the time you'll probably focus on creating static graphics for publication. However, you may find dynamic graphics fitting when sharing HTML notebooks or other web-based resources.

Luckily, the [`plotly` package](https://plotly.com/r/getting-started/#rendering-charts) makes creating dynamic graphics from `ggplot2` plots trivial.

Lets make an interactive graphic out of our static `sbp_facet` plot from earlier in this chapter. It's literally this simple!

```{r ggplotly-example, fig.asp = 0.5}
sbp_facet_plotly <- sbp_facet +
  facet_wrap(
    vars(diabetes),
    labeller = labeller(diabetes = dblabels)
  ) +
  ggtitle("Systolic blood pressure, by age and diabetes status") +
  theme_few() +
  theme(axis.title.x = element_text(margin = margin(t = 15)),
        axis.title.y = element_text(margin = margin(r = 15)),
        plot.margin = margin(t = 10, r = 10, b = 10, l = 20))

plotly::ggplotly(sbp_facet_plotly)

```

If you mouse over the points, you should see information about each observation (_i.e._, x and y variable values).

You could also use the `plot_ly()` function to make the plotly figure directly. 

```{r ggplotly-plot_ly, warning = FALSE, fig.fullwidth = TRUE}

plotly::plot_ly(
    data = nhefs,
    x = ~ age,
    y = ~ sbp,
    color = ~ factor(diabetes),
    type = "scatter",
    mode = "markers",
    colors = viridisLite::cividis(n = length(unique(nhefs$diabetes))),
    marker = list(colorscale = "Viridis"),
    width = 700
  ) %>%
  plotly::layout(
    xaxis = list(title = agelab),
    yaxis = list(title = sbplab),
    legend = list(title = list(text = "<b>Diabetes</b>")),
    margin = list(b = 75)
  )

```

If interactive graphics is an avenue you'd like to pursue further, look into the [`r2d3` library](https://rstudio.github.io/r2d3/index.html), which allows you to create beautiful visualizations using the powerful [`d3` Javascript library](https://d3js.org). 

<span class="marginnote">Plotly is actually [powered by d3](https://www.freecodecamp.org/news/how-and-why-i-used-plotly-instead-of-d3-to-visualize-my-lollapalooza-data-d48345e2ca68/#:~:text=To%20be%20fair,%20Plotly%20is,to%20know%20how%20Plotly%20works.).</span>

For R users, however, the `plotly` package provides a more immediate means to translate your data into interactive graphics. As we saw above, we can create a nice plot in `ggplot2` and export it for the web with `plotly::ggplotly`!