This wouldn't be a proper data blog unless we spend a vast majority of our time talking about cleaning data. Chances are if you're pursuing analysis that's groundbreaking (or worthwhile), we're probably starting with some ugly, untapped information. It turns out Mapbox has an API specifically for this purpose: the Mapbox Geocoding API.
Geocoding is a blanket term for turning
First things first, 10 points to anyone who understands what the title is referring to (that's either Charles Atlas' workout philosophy, or one of core tenants of Bokononism in Kurt Vonnegut's standout Cat's Cradle).
Now, with my obscure reference quota filled for the day, let's assume that you've been working as an analyst for some time now. As stated in
If you've been following along, we discussed in the last several posts of this series how, if you're not working in a very "tech forward" organization (like my two compatriots on his site), but you have the same title, you're probably obtaining your data from another department (or it might be a sentient sponge, or a gang of squirrels with
In my last post, we explored the organizational structure of many large companies and how this pertains to one's duties as a fledgling data analyst. I highly recommend you go back and read the first post on "dirty" data, but just in case you're one of those rebels who thinks that they're too cool to read part 1, here's a
In my last post, we discussed what separates a true analyst (read: technical) from a project manager wearing the mask of an analyst like some Scott Snyder era Joker (I figure that there's a solid overlap between fans of comic books and fans of the real world application of data. Note that this is a study with an N = 1
If you've been reading along, over the last several posts you've learned the two major skills that any self-respecting Excel jockey counts as their go-tos: the ability to lookup (remember, I'm partial to index-match, but if you learned VH lookup, ride that until you crash your system) and the ability to pivot.
Now here's something really interesting: until we pierce
During the last discussion, you've (hopefully) learned how to generate a pivot table, and learned about the four "buckets" that can house your columns:
I'm also going to make the wild assumption that you've played around with your newly birthed pivot table, taking your column headings from your "raw" data (in the lingua franca of the
In my last entry, we discussed how to write a formula, and you've been armed with what each piece of the formula represents (the command, the variables, and the definition of an array). With this knowledge, you've actually been armed with the keys to the kingdom, and you're finally going to learn how to do something fancy.
There comes a
You've heard the cliché before: it is often cited that roughly %80~ of a data scientist's role is dedicated to cleaning data sets. I Personally haven't looked in to the papers or clinical trials which prove this number (that was a joke), but the idea holds true: in the data profession, we find ourselves doing away with blatantly corrupt or
Let's say you have two obscenely large sets of data.
These sets of data contain information on a similar topic, such as customers. Dataset #1 might contain a high-level view of all customers of a business, while Datatset #2 contains a lifetime history of orders for a company. Unsurprisingly, the customers in Dataset #1 appear in
