exploration/5-explore-walmart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring the Walmart Triptype Dataset\n",
    "\n",
    "\n",
    "Walmart has generated a list of trip types for which they want to classify all transactions. The trip types were generated\n",
    "by Walmart data scientists over many years, but now Wal-mart would like a ML algorithm that will automatically classify\n",
    "sales transactional data with trip types.\n",
    "\n",
    "### Source\n",
    "\n",
    "This data source originally came from a [Kaggle Competition](https://www.kaggle.com/c/walmart-recruiting-trip-type-classification).\n",
    "\n",
    "### Transaction\n",
    "\n",
    "Note that this is a transactional data.  Before running our analysis, we may want to perform a pivot or rollup of the\n",
    "data to get all items fora given transation in a single row.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import re \n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn\n",
    "%matplotlib inline\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the data into Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#data = pd.read_csv(\"/data/walmart-triptype/walmart-triptype-train.csv.gz\")\n",
    "data = pd.read_csv(\"https://elephantscale-public.s3.amazonaws.com/data/walmart-triptype/walmart-triptype-train.csv.gz\")\n",
    "w_test_data = pd.read_csv(\"https://elephantscale-public.s3.amazonaws.com/data/walmart-triptype/walmart-triptype-test.csv.gz\")\n",
    "#w_test_data = pd.read_csv(\"/data/walmart-triptype/walmart-triptype-test.csv.gz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the Data\n",
    "\n",
    "Here's a brief guide to the triptypes.  Note that this isn't \"official\" from walmart, so it's mostly a guide that we can use to help describe the merchandise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "triptypes = pd.read_csv(\"https://elephantscale-public.s3.amazonaws.com/data/walmart-triptype/triptypes.csv\")\n",
    "triptypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examining the data\n",
    "\n",
    "Note that the data here is transactional.  That means that each row in the table is a single transaction rather than an entity.\n",
    "\n",
    "The row is listed on a per-item-bought basis.  Unlike what one might expect, this isn't given on a per sale basis, so one customer trip will have a number of rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(data[data.TripType == 8].VisitNumber.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Columns\n",
    "\n",
    " * TripType - a categorical id representing the type of shopping trip the customer made. This is the ground truth that you are predicting. TripType_999 is an \"other\" category. \n",
    "\n",
    " * VisitNumber - an id corresponding to a single trip by a single customer\n",
    " \n",
    " * Weekday - the weekday of the trip\n",
    "\n",
    " * Upc - the UPC number of the product purchased\n",
    " \n",
    " * ScanCount - the number of the given item that was purchased. A negative value indicates a product return.\n",
    " \n",
    " * DepartmentDescription - a high-level description of the item's department<br>\n",
    "\n",
    " * FinelineNumber - a more refined category for each of the products, created by Walmart<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How many rows are there of the data?  \n",
    "\n",
    "Are there any missing / NA values?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "647,054 rows of this data. We can see that the only columns with missing data are: <br>\n",
    "Upc (~4,000 missing values)<br>\n",
    "Fineline Number (same number of missing values as Upc)<br>\n",
    "Department Description (~1,500 missing values)<br>\n",
    "<br>\n",
    "Preliminary thoughts: <br>\n",
    "4,000 rows represents a very small portion of the test data (0.6%), so I think it will be safe to simply remove any rows with missing data from our dataframe.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some info about the triptypes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.TripType.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(data.TripType.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, 38 unique Trip Types. We will need to understand what 999 represents (could be missing information. Would be interesting to do some preliminary visual exploration of this data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get some info about the visit numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(data.VisitNumber.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.VisitNumber.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.VisitNumber.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data contains 94,247 unique store trips, as each visit number is the ID for a trip, and will be repeated for every item that is purchased on that trip."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What do visit numbers represent?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Days of the Week"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.Weekday.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nothing weird for days of the week, we should probably change them to numerals though. We can numerate Monday to Sunday as 1 to 7."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### UPCs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.Upc.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.Upc.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.Upc.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Good, no negative Upc numbers numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look at scan Counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.ScanCount.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not a lot of variation of scan counts.  There are a few large outliers though like 51, 71, and -12.  These outliers might skew analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look at Department Descriptions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.DepartmentDescription.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(data.DepartmentDescription.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Department Descriptions look fairly clean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look at Fineline numbers\n",
    "\n",
    "Finelines are product categories defined by walmart.  No \"key\" to finelines is included in the dataset, unlike triptype there are a very large number of them.\n",
    "\n",
    "It remains to be seen if this will prove to be a useful features for classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(data.FinelineNumber.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.FinelineNumber.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.FinelineNumber.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number zero for fineline is probably an \"unknown\" value of some sort."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[data.FinelineNumber == 0].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fineline_is_zero = data[data.FinelineNumber == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fineline_is_zero[fineline_is_zero.ScanCount == 1].count()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fineline_is_zero[fineline_is_zero.ScanCount == -1].count()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Almost all the fineline = 0 occurences are when either 1 item was purchased or 1 item was returned. Not sure if this means anything because this could be consistent with the number of 1 or -1 occurrences anyway."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding the Fineline Numbers "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_fineline_department = data[[\"DepartmentDescription\", \"FinelineNumber\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fineline_financial = data_fineline_department[data_fineline_department.DepartmentDescription == \"FINANCIAL SERVICES\"].FinelineNumber.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fineline_financial.plot(kind=\"bar\", rot=45, title=\"Type 4 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning Data\n",
    "\n",
    "Before loading the data for analysis, let's do a full cleanup items: get rid of NAs, change days of the week to numeric, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Dropping rows with missing values\n",
    "\n",
    "data = data.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Enumerate days of the week\n",
    "\n",
    "data = data.replace(\"Monday\", 1)\n",
    "data = data.replace(\"Tuesday\", 2)\n",
    "data = data.replace(\"Wednesday\", 3)\n",
    "data = data.replace(\"Thursday\", 4)\n",
    "data = data.replace(\"Friday\", 5)\n",
    "data = data.replace(\"Saturday\", 6)\n",
    "data = data.replace(\"Sunday\", 7)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Analysis\n",
    "\n",
    "Let's look at all the trip types one by one and see what kinds of insights we can get.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = data.TripType.unique()\n",
    "np.sort(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_triptypes = data.drop_duplicates(\"VisitNumber\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = data_triptypes[\"TripType\"]\n",
    "x = x.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph = x.plot(kind=\"bar\", figsize=(10, 5), color=\"midnightblue\")\n",
    "graph.set_title(\"Number of Occurences by trip type\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interesting - occurrences of different trip types are not evenly distributed. In fact, most trip types are categorized by just a handful of codes. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thinking about types of trips:<br>\n",
    "- Types of items purchased<br>\n",
    "- Weekday vs. weekend, or day of week<br>\n",
    "- returns vs. purchasing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Takeaways from types of trips:<br>\n",
    "    Many are focused on product category<br>\n",
    "    The only confusing trips were revolving around groceries or included groceries<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_3 = data[data.TripType == 3]\n",
    "type_3_items = type_3[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_3_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                        title=\"Type 3 - Financial Services\", color=\"midnightblue\")\n",
    "plt.xticks(fontsize=18)\n",
    "plt.ylabel('ylabel', fontsize=16)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "type_3_fineline = type_3.FinelineNumber.value_counts()\n",
    "type_3_fineline_finance = type_3[type_3.DepartmentDescription == \"FINANCIAL SERVICES\"].FinelineNumber.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_3_fineline_finance.plot(kind=\"bar\", rot=45, title=\"Financial Servies Fineline Numbers for Type 3\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_3_fineline.head(13).plot(kind=\"bar\", rot=45, title=\"Financial Servies Fineline Numbers for Type 3\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "It correlates as I suspected! The most frequent financial services fineline numbers - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_4 = data[data.TripType == 4]\n",
    "type_4_items = type_4[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_4_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, title=\"Type 4 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_5 = data[data.TripType == 5]\n",
    "type_5_items = type_5[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_5_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 5 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "type 5 trips = also pharmacy over the counter<br>\n",
    "they must differ from type 4 trips by another metric, like day of week or number of purchases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_6 = data[data.TripType == 6]\n",
    "type_6_items = type_6[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_6_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 6 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type 6 trips are about the booze. Alochol + candy/tobacco/cookies + grocery/impulse merchandise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_7 = data[data.TripType == 7]\n",
    "type_7_items = type_7[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_7_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 7 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type 7 trips are clearly grocery runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_8 = data[data.TripType == 8]\n",
    "type_8_items = type_8[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_8_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 8 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type 8 trips, the most frequent, seem like all purpose trips focused on grocery but also strong in personal care and impulse merchandise. They are not limited to grocery like type 7 trips and are twice as frequent. I wonder if the day, or number of items purchased, differs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_9 = data[data.TripType == 9]\n",
    "type_9_items = type_9[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_9_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 9 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All over the place here! Seems like this is a man shopping - men's clothing + automotive'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_12 = data[data.TripType == 12]\n",
    "type_12_items = type_12[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_12_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 12 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All over the place again "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_14 = data[data.TripType == 14]\n",
    "type_14_items = type_14[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_14_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 14 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "fabrics and crafts trip - very infrequent - I call this the \"Michael's\" trip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_15 = data[data.TripType == 15]\n",
    "type_15_items = type_15[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_15_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 15 Trips\", color=\"midnightblue\")\n",
    "#x.text(3,2000,\"lalalala\", size=15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PARTY trips!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "type_18 = data[data.TripType == 18]\n",
    "type_18_items = type_18[[\"TripType\",\"DepartmentDescription\"]]\n",
    "type_18_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 18 Trips\", color=\"midnightblue\")\n",
    "plt.ylabel=('Items Purchased')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TOYS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_19 = data[data.TripType == 19]\n",
    "type_19_items = type_19[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_19_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 19 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Electronics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_20 = data[data.TripType == 20]\n",
    "type_20_items = type_20[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_20_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Trip Type #20\", color=\"midnightblue\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Automotive!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_21 = data[data.TripType == 21]\n",
    "type_21_items = type_21[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_21_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 21 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Office supplies and fabrics/crafts - probably important which fabrics/crafts are being purchased"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_22 = data[data.TripType == 22]\n",
    "type_22_items = type_22[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_22_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 22 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Electronics + media and gaming -- probably different electronics than trip type 19, but that's the closest comparable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_23 = data[data.TripType == 23]\n",
    "type_23_items = type_23[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_23_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 23 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "players and electronics + media/gaming - how are players/electronics different than just electronics?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_24 = data[data.TripType == 24]\n",
    "type_24_items = type_24[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_24_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 24 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cook and dine - are these kitchen items? Looks like a best buy type trip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_25 = data[data.TripType == 25]\n",
    "type_25_items = type_25[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_25_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 25 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clothes trip, with more men's clothing being purchased"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_26 = data[data.TripType == 26]\n",
    "type_26_items = type_26[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_26_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 26 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "hardware trip - home depot type trip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_27 = data[data.TripType == 27]\n",
    "type_27_items = type_27[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_27_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 27 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lawn and garden + horticulture - home depot lawn and garden trip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_28 = data[data.TripType == 28]\n",
    "type_28_items = type_28[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_28_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 28 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sporting goods trip!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_29 = data[data.TripType == 29]\n",
    "type_29_items = type_29[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_29_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 29 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Toys and sporting goods! Probably a kids trip, maybe focused on boys or a different age group?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_30 = data[data.TripType == 30]\n",
    "type_30_items = type_30[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_30_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 30 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Shoe and jewelry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_31 = data[data.TripType == 31]\n",
    "type_31_items = type_31[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_31_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 31 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wireless technology (cellphones?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_32 = data[data.TripType == 32]\n",
    "type_32_items = type_32[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_32_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 32 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Baby products "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_33 = data[data.TripType == 33]\n",
    "type_33_items = type_33[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_33_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 33 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "household chemical supplies + paper goods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_34 = data[data.TripType == 34]\n",
    "type_34_items = type_34[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_34_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 34 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pet trip!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_35 = data[data.TripType == 35]\n",
    "type_35_items = type_35[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_35_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 35 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DSD Groceries (Direct store deliery) - focus on brands?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_36 = data[data.TripType == 36]\n",
    "type_36_items = type_36[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_36_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 36 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "personal care + beauty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_37 = data[data.TripType == 37]\n",
    "type_37_items = type_37[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_37_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 37 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Produce trips - another type of grocery trip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_38 = data[data.TripType == 38]\n",
    "type_38_items = type_38[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_38_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 38 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another grocery trip, with a focus on dairy (probably milk)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_39 = data[data.TripType == 39]\n",
    "type_39_items = type_39[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_39_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 39 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another grocery trip "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_40 = data[data.TripType == 40]\n",
    "type_40_items = type_40[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_40_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 40 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another grocery trip "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_41 = data[data.TripType == 41]\n",
    "type_41_items = type_41[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_41_items.DepartmentDescription.value_counts().head(10).plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 41 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A mix - could this be a return trip? Also, not very frequent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_42 = data[data.TripType == 42]\n",
    "type_42_items = type_42[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_42_items.DepartmentDescription.value_counts().head(10).plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 42 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another mix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_43 = data[data.TripType == 43]\n",
    "type_43_items = type_43[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_43_items.DepartmentDescription.value_counts().head(10).plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 43 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another mix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_44 = data[data.TripType == 44]\n",
    "type_44_items = type_44[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_44_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 44 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "mix of typical stuff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_999 = data[data.TripType == 999]\n",
    "type_999_items = type_999[[\"TripType\",\"DepartmentDescription\"]]\n",
    "x = type_999_items.DepartmentDescription.value_counts().head().plot(kind=\"bar\", rot=45, \n",
    "                                                              title=\"Type 999 Trips\", color=\"midnightblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"Others\" are often financial services related"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}