Starting with Data
What are data frames and tibbles?
Data frames are the de facto data structure for tabular data in R and what we use for data processing, statistics, and plotting.
A data frame is the representation of data in the format of a table where the columns are vectors that all have the same length. Data frames are analogous to the more familiar spreadsheet in programs such as Excel, with one key difference. Because columns are vectors, each column must contain a single type of data (e.g., characters, integers, factors).
A data frame can be created manually, but most commonly they are generated by the functions read_csv() or read_table() -in other words, when importing spreadsheets from your hard drive or the web.
The base R function read.csv() stores data as a data frame, where read_csv() from the tidyverse package stores data as a tibble. We prefer tibbles because they have nice printing properties among other desirable qualities. Read more about tibbles here.
Presentation of the SAFI Data
SAFI (StudyingAfrican Farmer-Led Irrigation) is a study looking at farming and irrigation methods in Tanzania and Mozambique. The survey data was collected through interviews conducted between November 2016 and June 2017. For this lesson, we will be using a subset of the available data.
We will be using a subset of the cleaned version of the dataset that was produced through cleaning in OpenRefine. Each row holds information for a single interview respondent, and the columns represent: instanceID Unique identifier for the form data submission
| column_name | description |
|---|---|
| key_id | Added to provide a unique Id for each observation. (The InstanceID field does this as well but it is not as convenient to use) |
| village | Village name |
| interview_date | Date of interview |
| no_membrs | How many members in the household? |
| years_liv | How many years have you been living in this village or neighboring village? |
| respondent_wall_type | What type of walls does their house have (from list) |
| rooms | How many rooms in the main house are used for sleeping? |
| memb_assoc | Are you a member of an irrigation association? |
| affect_conflicts | Have you been affected by conflicts with other irrigators in the area? |
| liv_count | Number of livestock owned. |
| items_owned | Which of the following items are owned by the household? (list) |
| no_meals | How many meals do people in your household normally eat in a day? |
| months_lack_food | Indicate which months, In the last 12 months have you faced a situation when you did not have enough food to feed the household? |
You are going to load the data in R’s memory using the function read_csv() from the readr package, which is part of the tidyverse. (Learn more about the tidyverse collection of packages here.) readr gets installed as part as the tidyverse installation. When you load the tidyverse (library(tidyverse)), the core packages (the packages used in most data analyses) get loaded, including readr.
Before we can use the read_csv() function, we need to load the package.Also, if you recall, the missing data is encoded as “NULL” in the dataset. We’ll tell R in the function to automatically convert all the “NULL” entries in the dataset into NA.
library(tidyverse)
interviews <- read_csv("data/SAFI_clean.csv", na = "NULL")
The second statement in the code above creates a data frame but doesn’t output any data because, as you might recall, assignments (<-) don’t display anything. If we want to check that our data has been loaded, we can see the contents of the data frame by typing its name: interviews in the console.
interviews
## Try also ## View(interviews)
## head(interviews)
# A tibble: 131 x 14
key_ID village interview_date no_membrs years_liv respondent_wall… rooms
<dbl> <chr> <dttm> <dbl> <dbl> <chr> <dbl>
1 1 God 2016-11-17 00:00:00 3 4 muddaub 1
2 1 God 2016-11-17 00:00:00 7 9 muddaub 1
3 3 God 2016-11-17 00:00:00 10 15 burntbricks 1
4 4 God 2016-11-17 00:00:00 7 6 burntbricks 1
5 5 God 2016-11-17 00:00:00 7 40 burntbricks 1
6 6 God 2016-11-17 00:00:00 3 3 muddaub 1
7 7 God 2016-11-17 00:00:00 6 38 muddaub 1
8 8 Chirod… 2016-11-16 00:00:00 12 70 burntbricks 3
9 9 Chirod… 2016-11-16 00:00:00 8 6 burntbricks 1 10 10 Chirod… 2016-12-16 00:00:00 12 23 burntbricks 5 # … with 121 more rows, and 7 more variables: memb_assoc <chr>, # affect_conflicts <chr>, liv_count <dbl>, items_owned <chr>, no_meals <dbl>, # months_lack_food <chr>, instanceID <chr>
Inspecting data frames
When calling a tbl_df object (like interviews here), there is already a lot of information about our data frame being displayed such as the number of rows, the number of columns, the names of the columns, and as we just saw the class of data stored in each column. However, there are functions to extract this information from data frames. Here is a non-exhaustive list of some of these functions. Let’s try them out!
Size:
dim(interviews)- returns a vector with the number of rows as the first element, and the number of columns as the second element (the dimensions of the object) .nrow(interviews)- returns the number of rowsncol(interviews)- returns the number of columns
Content:
head(interviews)- shows the first 6 rowstail(interviews)- shows the last 6 rows
Names:
names(interviews)- returns the column names (synonym ofcolnames()fordata.frameobjects)
Summary:
str(interviews- structure of the object and information about the class, length and content of each columnsummary(interviews)- summary statistics for each column
Note: Most of these functions are “generic,” i.e., they can be used on other types of objects besides data frames or tibbles.