data science Archives - Jeff Everhart

Parsing HTML/XML with NodeJS and cheerio

Eventually, this post will be a small part of a long post-mortem describing a ton of unexpected challenges I’ve faced recently trying to wrangle some historical geospatial data. However, this post is going to look at just one piece of that long process where I used the NodeJS library cheerio to help clean-up a GeoJSON […]

Read More

Retrieve Rows from Google Spreadsheet with Google Apps Script

Lots of the Google Apps Script projects that people are working on start with a Google Spreadsheet as a base. Sometimes a Google Form writes data to a spreadsheet that triggers other events, or you have a Sheet you’ve imported that you want to use to coordinate a larger workflow. This post will describe different […]

Read More

Using the Right Tools: Indexing in MySQL

As I’m sure this is with most people, my deepest learning moments come from random walks down different avenues. In preparation for a GIS focused project I might get to work on, I spent a few minutes reviewing the geospatial features available in MySQL, which lead me down some side research into database indexes. You […]

Read More

Reflections on Data Visualization in Observable

Continuing my exploration of data visualization tools, I circled back to where it all started for me: D3. For those of you involved in developing data-driven apps for the web, you most likely have a love/hate relationship with D3. You look at all of the cool examples and think, “Wow! That looks great,” but then […]

Read More

Using AWS for Data Analysis

I’m not really sure when this happened, but over the last several years, I’ve started to spend a lot of my personal and professional time working on building out data visualization tools and workflows. In a recent meeting, someone referred to us as data scientists, and we’ve had a good running joke ever since. While […]

Read More

Query Timeout in MySQL Workbench | Error Code: 2013. Lost connection to MySQL server during query

This is kind of a silly and duplicative post, but I spent too much time searching for the right answer, so maybe this will help the right course of action bubble to the top faster in the future. The Issue I was trying to run a query on my local SQL install (whatever MAMP manages […]

Read More

Visualizing Geographic Data with D3

I’m not sure the power of visualizing data can be understated in the modern world. After all, we have more data than we’ve every had before, and we at least seem to understand the world we live in less and less. For example, this image below displays population density in Virginia census tracts, but what […]

Read More

Command Line: Download Files with cURL Command

I’m not sure how I didn’t know about this command sooner since I use cURL for a few other tasks. I’ve been using WGET to download remote files, but I recently stumbled across this new little shortcut that should save me a few seconds here or there. If you’d rather just watch a short video, […]

Read More

Building A Simple Heat Map with D3

Heat maps are a great way to analyze frequency in a data set. Beautiful and creative things happen when you are stuck in an airport for four hours. I’ve been wanting to explore heat map visualizations for awhile now, so when I had some time to kill flying back from SXSW, I decided to take […]

Read More

Tech Tip: Randomizing Data in Microsoft Excel

Excel is a powerful data analysis tool. In this blog post, we will look at how to use the =RAND() function in Microsoft Excel to randomize a column of data. Randomizing data in Microsoft Excel could be important for a number of reasons, especially if you are trying to do any type of anonymous analysis. […]

Read More