Writing a data dictionary

Writing a Data Dictionary

We’ll do this with a smaller dataset than what we’re working with in this workshop.

install.packages("palmerpenguins")
library(palmerpenguins)

penguins_data <- palmerpenguins::penguins

First, we build each piece of metadata we’d like, inculding:

• variable names
• data types
• data ranges
• observation counts

varNames <- names(penguins_data) # we've seen this before
dataClass <- sapply(penguins_data, class) # we haven't seen sapply before -- see the note box below
observationsWithValues <- sapply(penguins_data, function(x) sum(!is.na(x)))
# this might need a bit more of a break down. Basically,
# for the range of numbers 1:n, where n is the number of columns in the dataset
# pass that number into the anynmous function, which then passes that number into
# each index subset. See alos the note box below on matrix subsetting.
dataRanges <- sapply(1:length(penguins_data), function(x) ifelse(
        class(penguins_data[[x]]) == "factor",
        paste0(levels(penguins_data[[x]]), collapse = ", "),
        paste0(range(penguins_data[[x]], na.rm = TRUE), collapse = " : ")))
# this is similar to above, see if you can figure out what's happening!
dataType <- sapply(penguins_data, function(x) dplyr::case_when(
  class(x) == "factor" && is.ordered(x) == TRUE ~ "odered",
  class(x) == "factor" && is.ordered(x) == FALSE ~ "nominal",
  .default = typeof(x)
))

sapply is one member of the apply family of functions. These functions take a list, and apply a function to each element in that list. sapply returns a vector. In the above case, for example with dataClass, we pass the data set as the list, which is interpreted as each of its columns, and then runs class() on each of these columns.

We’ve been introduced to filter and select to subset data. Subsetting can also be done by index location with []. There is a slight difference between using [] and [[]]. The former preserves the original data structure, the latter drops those attributes. In this example, we don’t want the class of the tibble, but of the individual column, hence the need for [[]].

class(penguins_data[1]) # reports tbl_df, tbl, data.frame

## [1] "tbl_df"     "tbl"        "data.frame"

class(penguins_data[[1]]) # reports factor

## [1] "factor"

We can then wrap this in a call to data.frame() to stitch it together. This requires a few small tweaks to the syntax.

penguins_dd <- data.frame(
  varNames = names(penguins_data),
  dataClass = sapply(penguins_data, class),
  dataType = sapply(penguins_data, function(x) dplyr::case_when(
    class(x) == "factor" && is.ordered(x) == TRUE ~ "odered",
    class(x) == "factor" && is.ordered(x) == FALSE ~ "nominal",
    .default = typeof(x)
  )),
  dataRanges = sapply(1:length(penguins_data), function(x) ifelse(
          class(penguins_data[[x]]) == "factor",
          paste0(levels(penguins_data[[x]]), collapse = ", "),
          paste0(range(penguins_data[[x]], na.rm = TRUE), collapse = " : "))),
  observationsWithValues = sapply(penguins_data, function(x) sum(!is.na(x)))
)
rownames(penguins_dd) <- NULL

Lastly, we can generalize, building a function that we can pass any data set into.

create_datadictionary <- function(dataset){
  df <- data.frame(
  varNames = names(dataset),
  dataClass = sapply(dataset, class),
  dataType = sapply(dataset, function(x) dplyr::case_when(
    class(x) == "factor" && is.ordered(x) == TRUE ~ "odered",
    class(x) == "factor" && is.ordered(x) == FALSE ~ "nominal",
    .default = typeof(x)
  )),
  dataRanges = sapply(1:length(dataset), function(x) ifelse(
          class(dataset[[x]]) == "factor",
          paste0(levels(dataset[[x]]), collapse = ", "),
          paste0(range(dataset[[x]], na.rm = TRUE), collapse = " : "))),
  observationsWithValues = sapply(dataset, function(x) sum(!is.na(x)))
)
rownames(df) <- NULL
return(df)
}

Let’s test it out!

penguins_dd_2 <- create_datadictionary(penguins_data)
penguins_dd_2

varNames	dataClass	dataType	dataRanges	observationsWithValues
species	factor	nominal	Adelie, Chinstrap, Gentoo	344
island	factor	nominal	Biscoe, Dream, Torgersen	344
bill_length_mm	numeric	double	32.1 : 59.6	342
bill_depth_mm	numeric	double	13.1 : 21.5	342
flipper_length_mm	integer	integer	172 : 231	342
body_mass_g	integer	integer	2700 : 6300	342
sex	factor	nominal	female, male	333
year	integer	integer	2007 : 2009	344

---

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Built and rendered courtesy of RMarkdown from RStudio