1 Exploratory Data Analysis (Descriptive statistics)

1.1 Introduction

Exploratory Data Analysis or EDA is the critical process of performing initial investigations on data to discover patterns, spot anomalies, test hypotheses and check assumptions with the help of summary statistics and graphical representations.

The major objective of this session is to teach participants on basics of exploring datasets and develop an understanding about the datasets using descriptive statistics and summary tables. In the next session, we shall explore visualization methods.

1.2 Descriptive statistics

As in the previous session, lets look at some summary statistics using summary() function.

Code 
df |> summary()
         child_weight_cat  maternal_age   maternal_weight      race      
 Normal          :130     Min.   :14.00   Min.   : 80.0   Min.   :1.000  
 Low Birth Weight: 59     1st Qu.:19.00   1st Qu.:110.0   1st Qu.:1.000  
                          Median :23.00   Median :121.0   Median :1.000  
                          Mean   :23.24   Mean   :129.8   Mean   :1.847  
                          3rd Qu.:26.00   3rd Qu.:140.0   3rd Qu.:3.000  
                          Max.   :45.00   Max.   :250.0   Max.   :3.000  
     smoke        number_preterm    hypertension     uterine_irritability
 Min.   :0.0000   Min.   :0.0000   Min.   :0.00000   Min.   :0.0000      
 1st Qu.:0.0000   1st Qu.:0.0000   1st Qu.:0.00000   1st Qu.:0.0000      
 Median :0.0000   Median :0.0000   Median :0.00000   Median :0.0000      
 Mean   :0.3915   Mean   :0.1958   Mean   :0.06349   Mean   :0.1481      
 3rd Qu.:1.0000   3rd Qu.:0.0000   3rd Qu.:0.00000   3rd Qu.:0.0000      
 Max.   :1.0000   Max.   :3.0000   Max.   :1.00000   Max.   :1.0000      
 health_visits     birth_weight       smoke_cat    race_cat 
 Min.   :0.0000   Min.   : 709   Non-smoker:115   White:96  
 1st Qu.:0.0000   1st Qu.:2414   Smoker    : 74   Black:26  
 Median :0.0000   Median :2977                    Other:67  
 Mean   :0.7937   Mean   :2945                              
 3rd Qu.:1.0000   3rd Qu.:3487                              
 Max.   :6.0000   Max.   :4990

1.3 Calculating mean and sd of maternal age.

summarise() function helps in obtaining summary statistics. The output has a summary statistics as asked within this function. In this example, we have used mean and sd functions for the same.

Code 
df |> 
  select(maternal_age) |> 
  summarise(mean_age = mean(maternal_age),
            sd_age = sd(maternal_age))
# A tibble: 1 × 2
  mean_age sd_age
     <dbl>  <dbl>
1     23.2   5.30

1.4 Presentation tables for descriptive statistics

Important

Though we were able to get the summary statistics, in routine, it is very time consuming, frustrating and error prone to write again the results/ outputs obtained from statistical software into the writing and communication documents, be it an article/ manuscript or a dissertation or a thesis. In R, there are certain packages which enable you to create publication ready tables which can be incorporated into research dissemination documents directly or with minor modifications. This saves a lot of mundane and unnecessary work and provides more time for interpretation and domain expertise related work. gtsummary package creates presentation-ready tables summarizing data sets, regression models, and more. The code to create the tables is concise and highly customizable.

Maternal characteristics: An illustrative example.

The tbl_summary() function calculates descriptive statistics for continuous, categorical, and dichotomous variables in R, and presents the results in a beautiful, customizable summary table ready for publication. To introduce tbl_summary() we will show the most basic behaviour first, which actually produces a large and beautiful table. Then, we will examine in detail how to make adjustments and more tailored tables.The default behavior of tbl_summary() is quite incredible - it takes the columns you provide and creates a summary table in one command. The function prints statistics appropriate to the column class: median and inter-quartile range (IQR) for numeric columns, and counts (%) for categorical columns. Missing values are converted to ‘Unknown’.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary()
Characteristic N = 1891
maternal_age 23.0 (19.0, 26.0)
maternal_weight 121 (110, 140)
race_cat
    White 96 (51%)
    Black 26 (14%)
    Other 67 (35%)
smoke_cat
    Non-smoker 115 (61%)
    Smoker 74 (39%)
1 Median (IQR); n (%)

1.4.1 Customizing output of selected variables.

Use equations to specify which statistics to show and how to display them. There are two sides to the equation, separated by a tilde ~. On the right side, in quotes, is the statistical display desired, and on the left are the columns to which that display will apply.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary(
    statistic = maternal_age ~ "{mean}, {sd}"
  )
Characteristic N = 1891
maternal_age 23.2, 5.3
maternal_weight 121 (110, 140)
race_cat
    White 96 (51%)
    Black 26 (14%)
    Other 67 (35%)
smoke_cat
    Non-smoker 115 (61%)
    Smoker 74 (39%)
1 Mean, SD; Median (IQR); n (%)

1.4.2 Customizing output of selected variable type.

Use equations to specify which statistics to show and how to display them. There are two sides to the equation, separated by a tilde ~. On the right side, in quotes, is the statistical display desired, and on the left are the list of columns to which that display will apply.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary(
    statistic = all_continuous() ~ "{mean}, {sd}"
  )
Characteristic N = 1891
maternal_age 23.2, 5.3
maternal_weight 130, 31
race_cat
    White 96 (51%)
    Black 26 (14%)
    Other 67 (35%)
smoke_cat
    Non-smoker 115 (61%)
    Smoker 74 (39%)
1 Mean, SD; n (%)

1.4.3 Changing label of a single variable.

Adjust how the column name should be displayed. Provide the column name and its desired label separated by a tilde. The default is the column name. This is done with help of argument label = in tbl_summary function.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary(
    statistic = all_continuous() ~ "{mean}, {sd}",
    label = maternal_weight ~ "Maternal Weight"
  )
Characteristic N = 1891
maternal_age 23.2, 5.3
Maternal Weight 130, 31
race_cat
    White 96 (51%)
    Black 26 (14%)
    Other 67 (35%)
smoke_cat
    Non-smoker 115 (61%)
    Smoker 74 (39%)
1 Mean, SD; n (%)

1.4.4 Changing labels of multiple variables.

You can change labels of multiple variables by providing the labels as a list to the label argument.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary(
    statistic = all_continuous() ~ "{mean}, {sd}",
    label = list(maternal_weight ~ "Maternal Weight",
                 maternal_age ~ "Maternal Age")
  )
Characteristic N = 1891
Maternal Age 23.2, 5.3
Maternal Weight 130, 31
race_cat
    White 96 (51%)
    Black 26 (14%)
    Other 67 (35%)
smoke_cat
    Non-smoker 115 (61%)
    Smoker 74 (39%)
1 Mean, SD; n (%)

1.4.5 Comparing group summaries.

Code 
df |> 
  select(maternal_age, maternal_weight,
         race_cat, smoke_cat) |> 
  tbl_summary(
    statistic = all_continuous() ~ "{mean}, {sd}",
    label = list(maternal_weight ~ "Maternal Weight",
                 maternal_age ~ "Maternal Age",
                 smoke_cat ~ "Smoking History"),
    by = race_cat
  )
Characteristic White, N = 961 Black, N = 261 Other, N = 671
Maternal Age 24.3, 5.7 21.5, 5.1 22.4, 4.5
Maternal Weight 132, 29 147, 40 120, 25
Smoking History
    Non-smoker 44 (46%) 16 (62%) 55 (82%)
    Smoker 52 (54%) 10 (38%) 12 (18%)
1 Mean, SD; n (%)

1.5 Summary.

In this session, we looked at summarise function and basics of gtsummary’s tbl_summary function for EDA. We shall be using these basic concepts further during the analysis in subsequent sessions.