All Gambill Data Blogs
Automating Data Pipelines with Python Classes and Functions
Handling new CSV files manually can be time-consuming and prone to errors. If you've ever had to inspect a CSV file, determine its schema, create a table, and then load the data manually, you know how tedious it can be. In this post, I’ll walk you through a Python class that automates this entire process—from reading a CSV file to dynamically creating a staging table and loading the data into a database.
This solution is great for data engineers and analysts who need a flexible, reusable approach to handling structured data.
Prioritizing DEI in Tech: The Importance of Diversity
Tech is built by all of us, for all of us—but are we doing enough to ensure everyone’s voices are included?
Diversity isn’t just about fairness—it’s about innovation, better decision-making, and stronger teams. Studies show that diverse teams solve problems faster and build better products, yet women and underrepresented groups still face barriers in tech.
November Data Challenge: Optimizing SQL Queries for Year-End Reports
As we approach the end of the year, data engineers are often tasked with preparing large-scale reports that provide insights on business performance. These reports can involve complex queries, requiring efficiency to avoid long processing times. This month’s code challenge focuses on optimizing SQL queries to handle high data volumes effectively.
