{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Titanic Survival: The classic first Kaggle competition: who survived the Titanic, and why a one-line rule is hard to beat.\n",
    "\n",
    "In 1912 the Titanic sank and 1,502 of 2,224 people on board died. Survival was not random: who you were and where you stood on the ship mattered. You will predict survival from the passenger list and find out how much a model adds beyond a single common-sense rule.\n",
    "\n",
    "**Data:** Titanic passengers (891 people). Bundled from the open datasciencedojo/datasets repository, which redistributes the public Titanic passenger list used in the Kaggle tutorial competition.\n",
    "\n",
    "Put `titanic.csv` in a folder called `data` next to this notebook (or download your own copy from Kaggle: https://www.kaggle.com/competitions/titanic)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1. Meet the passengers\n",
    "\n",
    "One row per passenger, with the answer in Survived. Check how common each answer is before anything else."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "print(df.shape)\n",
    "print(df.head(3).T.to_string())\n",
    "print()\n",
    "print(\"Survived (1) or not (0):\")\n",
    "print(df[\"Survived\"].value_counts())\n",
    "print(\"Survival rate:\", round(df[\"Survived\"].mean() * 100, 1), \"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** 891 passengers, 12 columns, and 38.4% survived. If you always guessed 'died' you would be right about 62% of the time, so a model has to do better than that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2. Find what is missing, and what the gaps mean\n",
    "\n",
    "Age, Cabin and Embarked have holes. Before filling them, ask whether being missing tells you something."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "print(\"Missing values per column:\")\n",
    "print(df.isna().sum()[lambda s: s > 0])\n",
    "print()\n",
    "print(\"Share missing: Age\", round(df[\"Age\"].isna().mean() * 100), \"% | Cabin\", round(df[\"Cabin\"].isna().mean() * 100), \"%\")\n",
    "print()\n",
    "print(\"Survival rate when Cabin is known vs unknown:\")\n",
    "print((df.groupby(df[\"Cabin\"].notna())[\"Survived\"].mean() * 100).round(1).to_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** 20% of ages and 77% of cabins are missing. Passengers with a known cabin survived 66.7% of the time against 30.0% without one. Cabin was recorded mostly for first class, so 'missing' is a clue about wealth, not random noise.\n",
    "\n",
    "**Aha.** A gap is information. Dropping or filling it without looking can throw the clue away."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3. Look at who survived\n",
    "\n",
    "Group averages show the pattern a model will learn. Look at one factor at a time, then two together."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "print(\"Survival rate by sex:\")\n",
    "print((df.groupby(\"Sex\")[\"Survived\"].mean() * 100).round(1).to_string())\n",
    "print()\n",
    "print(\"Survival rate by class:\")\n",
    "print((df.groupby(\"Pclass\")[\"Survived\"].mean() * 100).round(1).to_string())\n",
    "print()\n",
    "print(\"Survival rate by sex AND class:\")\n",
    "print((df.groupby([\"Sex\", \"Pclass\"])[\"Survived\"].mean() * 100).round(1).unstack().to_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** Women survived 74.2% of the time, men 18.9%. First class survived 63.0%, third class 24.2%. Together they tell the story: first-class women survived 96.8% of the time, third-class men only 13.5%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4. Split fairly, then set two baselines\n",
    "\n",
    "Split into train, validation and test with the same share of survivors in each. Then write the dumbest rules and score them."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print(\"train / validation / test:\", len(train), len(valid), len(test))\n",
    "print()\n",
    "print(\"Baseline 1: nobody survives ->\", round(accuracy_score(valid[\"Survived\"], [0] * len(valid)), 3))\n",
    "rule = (valid[\"Sex\"] == \"female\").astype(int)\n",
    "print(\"Baseline 2: women survive, men do not ->\", round(accuracy_score(valid[\"Survived\"], rule), 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** 'Nobody survives' scores 0.618. 'Women survive, men do not' scores 0.798, one line of code and nearly 80%. Every model has to beat 0.798 to earn its complexity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5. Build better columns\n",
    "\n",
    "Turn raw columns into ones a model can use: a 0/1 for sex, family size, the title hidden in the name, and a flag for 'age was missing'."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "print(\"Median age used to fill gaps (from train only):\", age_fill)\n",
    "print(X_train.shape)\n",
    "print(X_train.head(4).T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** 14 columns, built from 7. The median age 29 is learned from train only. 'Master' (a young boy) and 'Mrs' will turn out to matter.\n",
    "\n",
    "**Aha.** The name column looks useless, yet its title carries age and status. Good features come from asking what a column means."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6. Train a model you can read\n",
    "\n",
    "Logistic regression gives each feature a weight. Positive pushes toward survival, negative away."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)\n",
    "print(\"Logistic regression accuracy on validation:\", round(accuracy_score(y_valid, model.predict(X_valid)), 3))\n",
    "weights = pd.Series(model[-1].coef_[0], index=X_train.columns).sort_values()\n",
    "print()\n",
    "print(\"What it leans on (negative = less likely to survive):\")\n",
    "print(weights.round(2).to_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** Validation accuracy is 0.837. The largest weights are female (+0.96) and Pclass (-0.93), then family_size and Age. The model learned what the group averages showed, which is reassuring."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7. Try a random forest\n",
    "\n",
    "A random forest averages many decision trees. Compare it fairly with the logistic model on the same validation rows."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "models = {\n",
    "    \"Logistic regression\": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),\n",
    "    \"Random forest\": RandomForestClassifier(n_estimators=200, max_depth=5, random_state=0),\n",
    "}\n",
    "for name, m in models.items():\n",
    "    m.fit(X_train, y_train)\n",
    "    print(f\"{name:20s} validation accuracy: {accuracy_score(y_valid, m.predict(X_valid)):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** The forest scores 0.854 against 0.837. A modest gain on only 178 passengers is within noise: with so few rows, one or two passengers change the score by a full percentage point."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 8. Find where it is wrong\n",
    "\n",
    "Slice accuracy by group, and check which kind of mistake is more common."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "forest = RandomForestClassifier(n_estimators=200, max_depth=5, random_state=0).fit(X_train, y_train)\n",
    "out = valid[[\"Sex\", \"Pclass\", \"Survived\"]].copy()\n",
    "out[\"predicted\"] = forest.predict(X_valid)\n",
    "out[\"right\"] = (out[\"predicted\"] == out[\"Survived\"]).astype(int)\n",
    "print(\"Accuracy by sex:\")\n",
    "print((out.groupby(\"Sex\")[\"right\"].mean() * 100).round(1).to_string())\n",
    "print()\n",
    "print(\"Accuracy by class:\")\n",
    "print((out.groupby(\"Pclass\")[\"right\"].mean() * 100).round(1).to_string())\n",
    "print()\n",
    "print(\"Share of mistakes that are 'predicted died, actually survived':\",\n",
    "      round(((out[\"predicted\"] == 0) & (out[\"Survived\"] == 1)).sum() / (out[\"right\"] == 0).sum() * 100), \"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** Accuracy is similar for women (85.9%) and men (85.1%). Second class is easiest (94.4%) and third class hardest (82.8%). About 62% of the mistakes are survivors the model said would die.\n",
    "\n",
    "**Aha.** Overall accuracy hides which mistakes you make. Here, most are people who beat the odds."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 9. Open the test set once\n",
    "\n",
    "Choose the forest, refit on train plus validation, and score the 179 test passengers one time. Compare with the one-line rule."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "import pandas as pd\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "final = RandomForestClassifier(n_estimators=200, max_depth=5, random_state=0)\n",
    "final.fit(pd.concat([X_train, X_valid]), pd.concat([y_train, y_valid]))\n",
    "guess = final.predict(X_test)\n",
    "print(\"FINAL test result (look once)\")\n",
    "print(\"Accuracy:\", round(accuracy_score(y_test, guess), 3), \"| passengers scored:\", len(y_test))\n",
    "print(\"Women-survive rule on the same passengers:\", round(accuracy_score(y_test, (test[\"Sex\"] == \"female\").astype(int)), 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** Test accuracy is 0.821 against 0.777 for the women-survive rule. The model beats the rule by about four points, and the gap is smaller than what the validation numbers suggested. That shrinkage is normal and is exactly why the test set exists."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 10. Write your submission file\n",
    "\n",
    "Kaggle wants PassengerId and Survived, nothing else. Build exactly that and download it."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data/titanic.csv\")\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rest, test = train_test_split(df, test_size=0.2, stratify=df[\"Survived\"], random_state=42)\n",
    "train, valid = train_test_split(rest, test_size=0.25, stratify=rest[\"Survived\"], random_state=42)\n",
    "\n",
    "def prepare(frame, age_fill):\n",
    "    X = pd.DataFrame(index=frame.index)\n",
    "    X[\"Pclass\"] = frame[\"Pclass\"]\n",
    "    X[\"female\"] = (frame[\"Sex\"] == \"female\").astype(int)\n",
    "    X[\"Age\"] = frame[\"Age\"].fillna(age_fill)\n",
    "    X[\"age_was_missing\"] = frame[\"Age\"].isna().astype(int)\n",
    "    X[\"family_size\"] = frame[\"SibSp\"] + frame[\"Parch\"] + 1\n",
    "    X[\"Fare\"] = frame[\"Fare\"]\n",
    "    X[\"title\"] = frame[\"Name\"].str.extract(r\", (\\w+)\\.\", expand=False)\n",
    "    X[\"title\"] = X[\"title\"].where(X[\"title\"].isin([\"Mr\", \"Mrs\", \"Miss\", \"Master\"]), \"Other\")\n",
    "    X[\"port\"] = frame[\"Embarked\"].fillna(\"S\")\n",
    "    return pd.get_dummies(X, columns=[\"title\", \"port\"], dtype=float)\n",
    "\n",
    "age_fill = train[\"Age\"].median()\n",
    "X_train, y_train = prepare(train, age_fill), train[\"Survived\"]\n",
    "X_valid, y_valid = prepare(valid, age_fill).reindex(columns=X_train.columns, fill_value=0.0), valid[\"Survived\"]\n",
    "X_test, y_test = prepare(test, age_fill).reindex(columns=X_train.columns, fill_value=0.0), test[\"Survived\"]\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "final = RandomForestClassifier(n_estimators=200, max_depth=5, random_state=0)\n",
    "final.fit(pd.concat([X_train, X_valid]), pd.concat([y_train, y_valid]))\n",
    "submission = pd.DataFrame({\"PassengerId\": test[\"PassengerId\"].values, \"Survived\": final.predict(X_test)})\n",
    "submission_csv = submission.to_csv(index=False)\n",
    "print(\"Your submission file has\", len(submission), \"rows. First rows:\")\n",
    "print(submission.head(5).to_string(index=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What to notice.** A 179-row file of 0s and 1s. On Kaggle itself you would train on all of train.csv and predict the competition's own test.csv."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Limits to state out loud\n",
    "\n",
    "- This is a small, historical data set. A model that works here says little about modern risk.\n",
    "- 'Women and children first' shows up in the data, so the model partly encodes who was given priority, not who was fittest.\n",
    "- With only 891 passengers, small score differences are noise. Do not chase them.\n",
    "- Prediction of survival is not an explanation. Do not infer that being in third class caused death.\n",
    "\n",
    "## Try next\n",
    "\n",
    "- Try max_depth values from 2 to 8 in the forest and watch validation and training accuracy.\n",
    "- Add a column for 'travelling alone' and see if it helps.\n",
    "- Upload Kaggle's train.csv and compare your results with the numbers here."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}