As we get further into the project we will learn more data wrangling skills, but there are two I can see are immediately relevant. They are reshaping data and taking random samples.

require(mosaic)
# install.packages("tidyr")
require(tidyr)
require(Stat2Data)

Reshaping data

Sometimes, the data you get is in the wrong “shape.” Recall that at the beginning of the semester we talked about “tidy” data– data where every row is an observation at the same observational level, and every column is a variable. I usually think about this as rectangular data, but people will also call tidy data “tall” and un-tidy data “wide.”

We actually have a great example of this in problem 5.24, on baseball data

data(FantasyBaseball)
head(FantasyBaseball)
##   Round DJ  AR  BK  JW TS  RL  DR  MF
## 1     1 42  35  49 104 15  40  26 101
## 2     2 84  26  65 101 17 143  43  16
## 3     3 21  95 115  53 66 103 113  88
## 4     4 99  41  66 123  6 144  16  79
## 5     5 25 129  53 144  6 162 113  48
## 6     6 89  62  80 247 17  55 369   2

Here, each row is a round and each variable is a person. That doesn’t make a ton of sense. We would prefer to think about each row as a person, with variables for the round and their time for that round. We can use the gather/spread functions from tidyr to fix this.

I usually don’t get this right on the first try, so I run a few variations in my Console and just keep the one that looks the way I want. To remember which function I want, I look at the documentation

?spread
?gather

The documentation for gather says it “takes multiple columns and collapses into key-value pairs.” That’s what we want here.

gather(FantasyBaseball, Player, Time, Round) #nope! That's even worse than before!
gather(FantasyBaseball, Time, Round) #almost right 
gather(FantasyBaseball, Player, Time, -Round) #what we want

Once I’ve identified the way I want my data to look, I can overwrite the original data with my gathered version,

FantasyBaseball = FantasyBaseball %>%
  gather(Player, Time, -Round) 
head(FantasyBaseball)
##   Round Player Time
## 1     1     DJ   42
## 2     2     DJ   84
## 3     3     DJ   21
## 4     4     DJ   99
## 5     5     DJ   25
## 6     6     DJ   89

Now it looks the way I would like!

Taking a sample of data

Another task many of you are doing for your project is taking a random sample of your data, either because you want to have testing and training data, or you just need a smaller dataset that works on your laptop. R is great at taking random samples, and the sample() function is what you need.

Again, we can read the documentation

?sample

Before we do any sampling, however, we should “set a seed” for our randomness, so our work is reproducible. If you don’t do this, the next time you or one of your group members runs the same code, you will get a different result.

Let’s just see what this looks like in a simple case

sample(1:10, size = 5)
## [1]  7  8  1  6 10
sample(1:10, size = 5)
## [1] 2 9 5 8 7

Versus if we set a seed

set.seed(123)
sample(1:10, size = 5)
## [1] 3 8 4 7 6
set.seed(123)
sample(1:10, size = 5)
## [1] 3 8 4 7 6

You can put any number in the set.seed() function. I typically use 123, 42, or my birthday. You can put whatever you like.

Now, let’s apply this to taking a sample from data. Let’s say that for some reason we wanted a sample from the FantasyBaseball data.

dim(FantasyBaseball)
## [1] 192   3

It has 192 rows, so maybe I want to sample 50% of them or 96 rows. I’m going to do the sampling in two steps: first, I’m going to find a list of rows to sample, and then I’m going to use that list to actually slice up my data.

set.seed(123)
rows = sample(1:192, size=96)
mini_baseball = FantasyBaseball %>%
  slice(rows)
dim(mini_baseball)
## [1] 96  3

Now, my sample has all the same variables as my original data, but it’s half the size. You can apply this same strategy to take 25%, 10%, or any other size of sample you want to do.