Python

Let us feed your endless Python addiction! Regardless of where you stand as a Pythonista, our team of pros are constantly teaching and sharing pythonic gold.
Python
21 Apr 2019

Getting Started With Wagtail CMS

Setting up Wagtail CMS: The cleanest and friendliest CMS available for Django
Getting Started With Wagtail CMS

Last time we spoke of Django, we installed Django CMS: perhaps the most popular out-of-the-box CMS available for Django. Now that we've explored the most popular CMS, let's move on to the best CMS.

I realize I'm speaking of something subjective as though it were fact. As much as I'd like to agree that this is wrong, doing so would make for a boring read. I'll stick to my guns here.

Installing Wagtail

To makes things easy, we'll start with the obligatory server updates and Python3 setup. This way, the copy-and-paste crew will have no problem following along. It's important

Continue Reading
Data Science
14 Apr 2019

Upgrading to JupyterLab on Ubuntu

Improve your Jupyter workflow with JupyterLab's powerful interface & customizations.
Upgrading to JupyterLab on Ubuntu

Last we chatted, we walked through the process of getting started with Jupyter notebooks on a Ubuntu server. The classic Jupyter notebook interface is certainly well-suited to the job done. That said, it only takes some time of getting lost in the interface to start thinking we can do better. That's where Jupyter Lab comes in.

Why JupyterLab?

JupyterLab is sometimes referred to as "the next generation of Jupyter notebooks," which is a statement I can attest to. JupyterLab offers an improved interface for Jupyter notebooks which is both available out of the box, as well as highly customizable for

Continue Reading
Flask
04 Apr 2019

Using Flask-Login to Handle User Accounts

Add user authentication to your Flask app with Flask-Login
Using Flask-Login to Handle User Accounts

We’ve covered a lot of Flask goodness in this series thus far. We fully understand how to structure a sensible application; we can serve up complex page templates, and have dived into interacting with databases using Flask-SQLAlchemy. For our next challenge, we’re going to need all of the knowledge we've acquired thus far and much, much more. Welcome to the Super Bowl of Flask development. This. Is. Flask-Login.

Flask-Login is a dope library which handles all aspects of user management, including vital nuances you might not expect. Some noteworthy features include securing parts of our app behind login

Continue Reading
Data Engineering
28 Mar 2019

Building an ETL Pipeline: From JIRA to SQL

An example data pipeline which extracts data from the JIRA Cloud API and loads it to a SQL database.
Building an ETL Pipeline: From JIRA to SQL

Something we haven't done just yet on this site is walking through the humble process of creating data pipelines: the art of taking a bunch of data, changing said data, and putting it somewhere else. It's kind of a weird thing to be into, hence why the MoMA has been rejecting my submissions of Github repositories. Don't worry; I'll keep at it.

Something you don't see every day are people sharing their pipelines, which is understandable. Presumably, the other people who do this kind of stuff do it for work; nobody is happily building stupid pipelines in their free time

Continue Reading
Data Vis
28 Feb 2019

Drawing Mapbox Route Objects via the Directions API

Using the Mapbox Directions API to visually draw routes.
Drawing Mapbox Route Objects via the Directions API

If you've been here before, you probably already know our affinity for Mapbox and the visualization tools it provides data scientists and analysts. In the past, we've covered encoding location data from raw addresses, as well as an exploration of Mapbox Studio for those getting acquainted with the tool. Today we're going a step further: drawing directions on a map.

It sounds simple enough: we already know how to geocode addresses, so all we need to do is literally go from point A to point B. That said, things always tend to get tricky, and if you've never worked with

Continue Reading
Python
18 Feb 2019

S3 File Management With The Boto3 Python SDK

Modify and manipulate thousands of files in your S3 (or DigitalOcean) Bucket.
S3 File Management With The Boto3 Python SDK

It's incredible the things human beings can adapt to in life-or-death circumstances, isn't it? In this particular case it wasn't my personal life in danger, but rather the life of this very blog. I will allow for a brief pause while the audience shares gasps of disbelief. We must stay strong and collect ourselves from such distress.

Like most things I despise, the source of this unnecessary headache was a SaaS product. I won't name any names here, but it was Cloudinary. Yep, totally them. We'd been using their (supposedly) free service for hosting our blog's images for about a

Continue Reading
Flask
06 Feb 2019

Manage Database Models with Flask-SQLAlchemy

Connect your Flask app to a database using Flask-SQLAlchemy.
Manage Database Models with Flask-SQLAlchemy

By now you're surely familiar with the benefits of Python's SQLAlchemy library: the all-in-one solution for basically anything database related. Like most major Python libraries, SQLAlchemy has been ported into a version specifically compatible with Flask, aptly named Flask-SQLAlchemy.

Similar to the core SQLAlchemy package, Flask-SQLAlchemy provides an ORM for us to modify application data by easily creating defined models. Regardless of what your database of choice might be, Flask-SQLAlchemy will ensure that the models we create in Python will translate to the syntax of our chosen database. Given the ease-of-use and one-size-fits-all  nature of Flask-SQLAlchemy, it's no wonder that

Continue Reading
Big Data
02 Feb 2019

Google BigQuery's Python SDK: Creating Tables Programmatically

Explore the benefits of Google BigQuery and use the Python SDK to programmatically create tables.
Google BigQuery's Python SDK: Creating Tables Programmatically

GCP is on the rise, and it's getting harder and harder to have conversations around data warehousing without addressing the new 500-pound gorilla on the block: Google BigQuery. By this point, most enterprises have comfortably settled into their choice of "big data" storage, whether that be Amazon Redshift, Hadoop, or what-have-you. BigQuery is quickly disrupting the way we think about big data stacks by redefining how we use and ultimately pay for such services.

The benefits of BigQuery likely aren't enough to force enterprises to throw the baby out with the bathwater. That said, companies building their infrastructure from the

Continue Reading
Pandas
28 Jan 2019

Downcast Numerical Data Types with Pandas

Using an Example Where We Downcast Numerical Columns.
Downcast Numerical Data Types with Pandas

Recently, I had to find a way to reduce the memory footprint of a Pandas DataFrame in order to actually do operations on it.  Here's a trick that came in handy!

By default, if you read a DataFrame from a file, it'll cast all the numerical columns as the float64 type.  This is in keeping with the philosophy behind Pandas and NumPy - by using strict types (instead of normal Python "duck typing"), you can do things a lot faster.  The float64 is the most flexible numerical type - it can handle fractions, as well as turning missing values into

Continue Reading
Big Data
23 Jan 2019

From CSVs to Tables: Infer Data Types From Raw Spreadsheets

The quest to never explicitly set a table schema ever again.
From CSVs to Tables: Infer Data Types From Raw Spreadsheets

Back in August of last year (roughly 8 months ago), I hunched over my desk at 4 am desperate to fire off a post before boarding a flight the next morning. The article was titled Creating Database Schemas: a Job for Robots, or Perhaps Pandas. It was my intent at the time to solve a common annoyance: creating database tables out of raw data, without the obnoxious process of explicitly setting each column's datatype. I had a few leads that led me to believe I had the answer... boy was I wrong.

The task seems somewhat reasonable from the surface.

Continue Reading
Tableau
17 Jan 2019

The Hostile Extraction of Tableau Server Data

Say no to proprietary software constraints. Say no to vendor lock. Say yes to freedom.
The Hostile Extraction of Tableau Server Data

I try my best not to hate on Tableau. It was the software’s combination of power and ease-of-use that drove me to purchase a license in the first place. Ever since then, I’m finding new and exciting ways Tableau intentionally locks users out of their data.

I gave the Tableau Server Client Python library a spin recently in hopes of finding something useful. I decided to (sigh, once more) allow Tableau the benefit of the doubt: after pushing four updates in a single month, maybe things had changed. On the contrary, the Tableau business strategy stands strong: to

Continue Reading
PostgreSQL
15 Jan 2019

Psycopg2: PostgreSQL & Python the Old Fashioned Way

Manage PostgreSQL database interactions in Python with the Psycopg2 library.
Psycopg2: PostgreSQL & Python the Old Fashioned Way

Last time we met, we joyfully shared a little tirade about missing out on functionality provided to us by libraries such as SQLAlchemy, and the advantages of interacting with databases where ORMs are involved. I stand by that sentiment, but I’ll now directly contradict myself by sharing some tips on using vanilla Psycopg2 to interact with databases.

We never know when we’ll be stranded on a desert island without access to SQLAlchemy, but a lonesome Psycopg2 washes up on shore. Either that or perhaps you’re part of a development team stuck in a certain way of doing

Continue Reading
SQL
09 Jan 2019

Pythonic Database Management with SQLAlchemy

The iconic Python library for handling any conceivable database interaction.
Pythonic Database Management with SQLAlchemy

Something we've taken for granted thus far on Hackers and Slackers is a library most data professionals have accepted as standard: SQLAlchemy.

In the past, we've covered database connection management and querying using libraries such as PyMySQL and Psycopg2, both of which do an excellent job of interacting with databases just as we'd expect them to. The nature of opening/closing DB connections and working with cursors hasn't changed much in the past few decades. While boilerplate is boring, at least it has remained consistent, one might figure. That may have been the case, but the philosophical boom of MVC

Continue Reading
Python
08 Jan 2019

Poetically Packaging Your Python Project

Manage your projects with Poetry to handle dependencies, envs, packaging, etc.
Poetically Packaging Your Python Project

It wasn't long ago that we Hackers were singing the praises of Pipenv: Python's seemingly superior dependency manager at the time. While we hold much love in hearts, sometimes there is love to go around. We just so happen to be fair weather fans, which reminds me: what has Pipenv done for me lately?

As you've probably guessed (considering its a piece of software), nothing much. Well, there was that time when pip upgraded from v.18 to v.18.1, which broke Pipenv entirely with almost minimal acknowledgment (for all I know this might still be broken). As our

Continue Reading