Hackers and Slackers: Data Science for Badasses

Python

43 Posts
The language, the legend, the liberator.

The End-to-End Guide to Building Forms in Flask

The subtle art of consenually capturing personal data

Happy Tuesday everybody! To start this week hot, let's talk about something that everybody hates: forms. The only thing more painful than filling out a web form is creating one, much less a functional one with feedback. Listen, if you're into creating pleasant form UI experiences, you're probably into some freaky shit. Call me.

Flask's youth is an advantage in one regard, in that there are only so many libraries available to handle any given task. In this case, there's

Flask Author imageTodd Birchard August 15
Read

Importing Excel Datetimes Into Pandas

Pandas & Excel, Part 1

Different file formats are different!  For all kinds of reasons!

A few months back, I had to import some Excel files into a database. In this process I learned so much about the delightfully unique way Excel stores dates & times!  

The basic datetime will be a decimal number, like 43324.909907407404.  The number before the decimal is the day, the number afterwards is the time.  So far, so good - this is pretty common for computers.  The date is

Pandas Author imageMatthew Alhonte August 13
Read

Using MongoDB Atlas as your Flask Database

Since we’re fans of Python, I’ll assume we both prefer enjoyable dev.

I’ve been messing with MongoDB’s cloud offering, MongoDB Atlas, for a few months now. From my perspective, the jury is still out on how Atlas and Stitch will fit into the picture of next generation cloud services, I can vouch that they’re fun to use. Since we’re fans of Python and Flask already, I’ll assume you prefer enjoyable development too.

We're going to set up a simple app where users can submit URLs via a

Flask Author imageTodd Birchard July 31
Read

All That Is Solid Melts Into Graphs

Reshaping Pandas dataframes with a real-life example, and graphing it with Altair

Last few Code Snippet Corners were about using Pandas as an easy way to handle input and output between files & databases.  Let's shift gears a little bit!  Among other reasons, because earlier today I discovered a package that exclusively does that, which means I can stop importing the massive Pandas package when all I really wanted to do with it was take advantage of its I/O modules.  Check it out!

So, rather than the entrances & exits, let's

Python Author imageMatthew Alhonte July 30
Read

Automagically Turn JSON into Pandas DataFrames

Let pandas do the heavy lifting for you when turning JSON into a DataFrame.

In his post about extracting data from APIs, Todd demonstrated a nice way to massage JSON into a pandas DataFrame. This method works great when our JSON response is flat, because dict.keys() only gets the keys on the first "level" of a dictionary. It gets a little trickier when our JSON starts to become nested though, as I experienced when working with Spotify's API via the Spotipy library. For example, take a look at a response from their https:

Python Author imageGraham Beckley July 28
Read

Trash Pandas: Messy, Convenient DB Operations via Pandas

(And a way to clean it up with SQLAlchemy)

Let's say you were continuing our task from last week: Taking a bunch of inconsistent Excel files and CSVs, and putting them into a database.

Let's say you've been given a new CSV that conflicts with some rows you've already entered, and you're told that these rows are the correct values.

Explain why Pandas' built-in method wouldn't be good

Pandas' built-in to_sql DataFrame method won't be useful here.  Remember, it writes as a block - if you set the

Pandas Author imageMatthew Alhonte July 23
Read

Serving Frontend Templates and Assets in Flask

When Python developers manage presentation layers.

If you're familiar with Django (or Python to any extent), you've probably accepted the concept of what Python guys refer to as "static assets" without much consideration. At some point in Django's conception, a design decision was made: any code, styles, or assets were tucked away into their own respective uninteresting corners of the framework. Thus it was known: anything that isn't Python is simply a "static" asset to be dealt with either locally or via a CDN somewhere. Basically,

Flask Author imageTodd Birchard July 23
Read

A Dirty Way of Cleaning Data (ft. Pandas & SQL)

Code Snippet Corner ft. Pandas & SQL

Warning The following is FANTASTICALLY not-secure.  Do not put this in a script that's going to be running unsupervised.  This is for interactive sessions where you're prototyping the data cleaning methods that you're going to use, and/or just manually entering stuff.  Especially if there's any chance there could be something malicious hiding in the data to be uploaded.  We're going to be executing formatted strings of SQL unsanitized code.  Also, this will lead to LOTS of silent failures, which

Pandas Author imageMatthew Alhonte July 16
Read