Deep-dive · Data Types, Coercion and Factors

Optional companion to Module 1

Optional. Nothing in Module 2 assumes it. Read it when a column turns out to be the wrong type and you want to know why.

library(readr)
library(dplyr)

Attaching package: 'dplyr'
The following objects are masked from 'package:stats':

    filter, lag
The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union
abcd <- read_csv("data/abcd-synthetic.csv")
Rows: 580 Columns: 9
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (4): subject_id, visit, site, sex
dbl (5): age_months, total_problems, cognition_score, screen_time_hrs, sleep...

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

The types, properly

class(10.5)        # numeric  (specifically "double")
[1] "numeric"
class(10L)         # integer  -- the L suffix forces a whole number
[1] "integer"
class("baseline")  # character
[1] "character"
class(TRUE)        # logical
[1] "logical"
class(NA)          # logical, oddly -- see below
[1] "logical"

class() tells you what something is. is.numeric(), is.character() and friends ask yes/no questions, which is more useful inside code:

is.numeric(abcd$age_months)
[1] TRUE
is.character(abcd$visit)
[1] TRUE

Coercion, and why your numbers became text

R will not let a vector hold mixed types. When you try, it promotes everything to the most accommodating type, following this one-way street:

logical  →  integer  →  double  →  character

Each step can represent everything to its left, so nothing is lost — but the result may not be what you wanted:

c(TRUE, 1L)          # logical joins integer   -> integer
[1] 1 1
c(1L, 2.5)           # integer joins double    -> double
[1] 1.0 2.5
c(2.5, "hello")      # double joins character  -> character
[1] "2.5"   "hello"

That last one is the one that bites. A single stray text value anywhere in a spreadsheet column turns the entire column into text:

ages <- c(118, 121, "not recorded", 130)
class(ages)
[1] "character"

And once it is text, arithmetic fails:

mean(ages)
Warning in mean.default(ages): argument is not numeric or logical: returning NA
[1] NA

In real data the culprit is usually a placeholder someone typed into a numeric column — "n/a", "missing", "999", "<1", or a stray footnote marker.

Fixing it

as.numeric() converts back. Anything that is not a number becomes NA, with a warning telling you it happened:

ages_fixed <- as.numeric(ages)
Warning: NAs introduced by coercion
ages_fixed
[1] 118 121  NA 130
mean(ages_fixed, na.rm = TRUE)
[1] 123

That warning — NAs introduced by coercion — is useful, not noise. It is R saying “I threw something away”. If you see it unexpectedly, go and look at what got discarded before moving on.

Better still, tell read_csv() about the placeholders up front so the column arrives numeric in the first place:

read_csv("data/my-file.csv", na = c("", "NA", "n/a", "missing", "999"))

NA in more detail

NA is not zero, and not an empty string. It means unknown, and R propagates that unknown-ness through everything it touches:

NA + 1
[1] NA
NA > 5
[1] NA
NA == NA     # not TRUE -- two unknowns may or may not be equal
[1] NA

That last line is why you cannot test for missingness with ==:

x <- c(1, NA, 3)
x == NA        # useless
[1] NA NA NA
is.na(x)       # this is the way
[1] FALSE  TRUE FALSE

There is one exception worth knowing, because it looks inconsistent until you see the logic:

NA | TRUE      # TRUE  -- "unknown OR true" is true whatever the unknown is
[1] TRUE
NA & FALSE     # FALSE -- "unknown AND false" is false whatever the unknown is
[1] FALSE

Factors

A factor is a categorical variable with a fixed, known set of possible values, called its levels. It looks like text and behaves like a number underneath.

visit_f <- factor(abcd$visit)
levels(visit_f)
[1] "baseline" "year_1"   "year_2"

Two reasons to care.

Reason 1: controlling order

factor() sorts levels alphabetically by default. Sometimes that is wrong:

levels(factor(c("baseline", "year_1", "year_2")))
[1] "baseline" "year_1"   "year_2"

Here alphabetical happens to be correct — b before y. Now watch it fail with more realistic visit labels:

levels(factor(c("baseline", "2_year", "1_year")))
[1] "1_year"   "2_year"   "baseline"

"1_year" sorts before "2_year" before "baseline", so a plot would show the timeline backwards. Set the order explicitly:

visits <- c("baseline", "2_year", "1_year")
factor(visits, levels = c("baseline", "1_year", "2_year")) |> levels()
[1] "baseline" "1_year"   "2_year"

This is the reason factors matter in practice: ggplot2 and group_by() follow factor level order, so setting levels is how you control the order of bars, boxes and legend entries.

abcd |> 
  mutate(visit = factor(visit, levels = c("baseline", "year_1", "year_2"))) |> 
  count(visit)
# A tibble: 3 × 2
  visit        n
  <fct>    <int>
1 baseline   200
2 year_1     200
3 year_2     180

Reason 2: statistical models care

In Module 3, when you fit a model with a categorical predictor, R uses the first level as the reference group — every coefficient is expressed relative to it. Changing which level is first changes how the output reads. relevel() picks the reference:

sex_f <- factor(abcd$sex)
levels(sex_f)
[1] "F" "M"
levels(relevel(sex_f, ref = "M"))
[1] "M" "F"

The classic factor trap

Converting a factor straight to numeric gives you the level codes, not the values:

f <- factor(c("10", "20", "30"))
as.numeric(f)              # 1 2 3 -- the internal codes. Wrong.
[1] 1 2 3
as.numeric(as.character(f))  # 10 20 30 -- correct
[1] 10 20 30

Go via as.character() every time. This one has silently corrupted a great many analyses.

Checking a fresh dataset

A habit worth building. Before analysing anything:

glimpse(abcd)
Rows: 580
Columns: 9
$ subject_id      <chr> "sub-0001", "sub-0001", "sub-0001", "sub-0002", "sub-0…
$ visit           <chr> "baseline", "year_1", "year_2", "baseline", "year_1", …
$ site            <chr> "S01", "S01", "S01", "S02", "S02", "S02", "S05", "S05"…
$ age_months      <dbl> 111, 125, 136, 110, 123, 134, 130, 142, 153, 127, 137,…
$ sex             <chr> "F", "F", "F", "M", "M", "M", "M", "M", "M", "M", "M",…
$ total_problems  <dbl> NA, 16, 20, 37, 30, 30, 18, 21, 20, 31, 16, 9, 36, 35,…
$ cognition_score <dbl> 98, 106, NA, 94, 86, 91, 122, 123, 104, 107, 104, 97, …
$ screen_time_hrs <dbl> 0.7, 3.1, 2.8, 2.2, NA, 6.0, 1.9, NA, 3.7, 2.3, 5.3, 3…
$ sleep_hrs       <dbl> 9.8, 8.9, 9.2, 10.1, 9.4, 9.1, 8.8, 10.5, 8.1, 9.0, 9.…
# Any column that should be numeric but arrived as character?
abcd |> summarise(across(everything(), ~ class(.x)[1]))
# A tibble: 1 × 9
  subject_id visit     site      age_months sex   total_problems cognition_score
  <chr>      <chr>     <chr>     <chr>      <chr> <chr>          <chr>          
1 character  character character numeric    char… numeric        numeric        
# ℹ 2 more variables: screen_time_hrs <chr>, sleep_hrs <chr>

If a column you expect to be numeric shows up as character, stop and find the offending value before doing anything else.

Back to Module 1.