Data Science Fundamentals - Lab - Titanic EDA

About

For this lab, we’re going to take a look at the Titanic manifest. We’ll be exploring this data to see what we can learn regarding the survival rates of different groups of people.

Submission

Lab submissions should be made as per guidance from your instructor. Your submission will be in the form of a link to your Github fork of this repository.

Prework

Fork and clone this repo.

Step 1: Reading the data

In the notebook provided, do the following:

  1. Import pandas and matplotlib.pyplot
  2. Load train.csv as a pandas DataFrame.
  3. In each of the following sections, copy the question as a python comment, then answer the question with your own code.
  4. Refer to the Titanic Kaggle competition if you need an explanation for any of the columns.

Step 2: Cleaning the data

  1. Create a bar chart showing how many missing values are in each column
    • Bonus : Theres a good library for visualizing missing values called Missingno.
  2. Which column has the most NaN values? How many cells in that column are empty?
  3. Delete all rows where Embarked is empty
  4. Fill all empty cabins with ¯\(ツ)/¯

Note: NaN, empty, and missing are synonymous.

Step 3: Feature extraction

  1. There are two columns that pertain to how many family members are on the boat for a given person. Create a new column called FamilyCount which will be the sum of those two columns.
  2. Reverends have a special title in their name. Create a column called IsReverend: 1 if they’re a preacher, 0 if they’re not.
  3. In order to feed our training data into a classification algorithm, we need to convert our categories into 1’s and 0’s using pd.get_dummies
    • Familiarize yourself with the pd.get_dummies documentation
    • Create 3 columns: Embarked_C, Embarked_Q and Embarked_S. These columns will have 1’s and 0’s that correspond to the C, Q and S values in the Embarked column
    • Do the same thing for Sex
    • BONUS: Extract the title from everyone’s name and create dummy columns

Step 4: Exploratory analysis

df.groupby() may be very useful.

  1. What was the survival rate overall?
  2. Which gender fared the worst? What was their survival rate?
  3. What was the survival rate for each Pclass?
  4. Did any reverends survive? How many?
  5. What is the survival rate for cabins marked ¯\(ツ)/¯
  6. What is the survival rate for people whose Age is empty?
  7. What is the survival rate for each port of embarkation?
  8. What is the survival rate for children (under 12) in each Pclass?
  9. Did the captain of the ship survive? Is he on the list?
  10. Of all the people that died, who had the most expensive ticket? How much did it cost?
  11. Does having family on the boat help or hurt your chances of survival?

Step 5: Plotting

Using matplotlib and seaborn, create several charts showing the survival rates of different groups of people. It’s fine if a handful of charts are basic (Gender, Age, etc), but what we’re really looking for is something beneath the surface.

Important note

Please ignore the _layouts folder and the config.yml file in the root of this repo. They are needed to make sure the repo renders well; please don’t change them or delete them and your repo will work fine :)