Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
adf
7 Posts
Is My dbt Ready?
Set up activities in Azure Data Factory to track when your containerized dbt project has finished running
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
Pulling Azure Container Instance Logs into Azure Data Factory
A description of how to include Azure Container Instance logs into your Data Factory pipeline.
Running Scripts using Azure Data Factory and Batch, Part II
An example on how to run scripts within an Azure Data Factory pipeline
Running Scripts using Azure Data Factory and Batch, Part I
An example on how to connect Azure Batch and Azure Storage accounts to Azure Data Factory
Testing Expressions in Azure Data Factory
Testing expression is a simple trick in Azure Data Factory, so that you are able to see what is being passed on to your conditional activities.
airflow
2 Posts
My Airflow Journey
Starting from scratch is impractical, Google Cloud Platform's Composer is expensive, and debugging Airflow on Windows is challenging
Syncing a Cloud Source Repository to your Composer Airflow Environment
And thus starting your CI/CD pipeline
azure
9 Posts
Cleaning Up Deprecated Models using dbt in Azure Synapse
Use custom macros to remove deprecated tables and views from your target schema
Creating Date Dimensions using dbt in Azure Synapse: Custom Macros
A custom macro to create date dimensions when using the dbt-synapse adapter
Creating Tables with LOBs using dbt in Azure Synapse: The Columnstore Index Error
A work-around to the default settings when creating tables using the dbt-synapse adapter
Taking snapshots using dbt in Azure Synapse: The Hash Distribution Issue
Learn how to troubleshoot and resolve the "Hash Distributed Table" error when taking snapshots using dbt in Azure Synapse Dedicated Pool
Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
Is My dbt Ready?
Set up activities in Azure Data Factory to track when your containerized dbt project has finished running
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
Pulling Azure Container Instance Logs into Azure Data Factory
A description of how to include Azure Container Instance logs into your Data Factory pipeline.
A simple Dockerfile for a dbt Synapse
A sample Dockerfile to containerize a dbt project with Azure Synapse dependencies
big-query
4 Posts
Pausing dbt Resources after Pipeline Retirement
A summary of three ideas on pausing dbt resources: leveraging the exclusion flag, disabling resources, and transforming models into analyses
Cool dbt Feature: Project Variables
A brief introduction to project variables and their benefit when modeling in dbt
Understanding DAGs in dbt: Threads, Errors and Failing Fast
Three scenarios show us the implications of running a Directed Acyclic Graph (DAG) in multi-threaded dbt environments, shedding light on how errors and the "fail fast" argument impact model execution
Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
databricks
2 Posts
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
Saving and Loading Scalers using MLflow and Databricks with Python
Chances are, you will be needing scalers to make predictions on new data sets
dataset
2 Posts
Berlin Marathon Data: 1974-2023
Berlin Marathon results among runners from 1974 to 2023, including data from Kaggle for the years 1974-2019 and additional results from 2021-2023
Mapping of Postal and District Codes
A mapping from Austrian 4-digit postal codes to Austrian 3-digit district codes
dax
1 Posts
Formatting DAX Code
Keeping your code neat and tidy
dbt
16 Posts
Working with dbt in VS Code
A brief description of features from CSV Rainbow and dbt Power User, both extensions for VS Code
Cleaning Up Deprecated Models using dbt in Azure Synapse
Use custom macros to remove deprecated tables and views from your target schema
Pausing dbt Resources after Pipeline Retirement
A summary of three ideas on pausing dbt resources: leveraging the exclusion flag, disabling resources, and transforming models into analyses
Creating Date Dimensions using dbt in Azure Synapse: Custom Macros
A custom macro to create date dimensions when using the dbt-synapse adapter
Cool dbt Feature: Exposures
A brief introduction to exposures and why they should be in every dbt project
Cool dbt Feature: Project Variables
A brief introduction to project variables and their benefit when modeling in dbt
Creating Tables with LOBs using dbt in Azure Synapse: The Columnstore Index Error
A work-around to the default settings when creating tables using the dbt-synapse adapter
Understanding DAGs in dbt: Threads, Errors and Failing Fast
Three scenarios show us the implications of running a Directed Acyclic Graph (DAG) in multi-threaded dbt environments, shedding light on how errors and the "fail fast" argument impact model execution
Taking snapshots using dbt in Azure Synapse: The Hash Distribution Issue
Learn how to troubleshoot and resolve the "Hash Distributed Table" error when taking snapshots using dbt in Azure Synapse Dedicated Pool
Understanding DAGs in dbt: Erring Models and Failing Tests
Three scenarios shows us what happens downstream when our models and/or tests do not succeed
dbt seed: Data Warehousing and CSV Files
The following are a few points I have learned while working with seeds in my dbt project
Effortless Documentation with dbt: Streamlining Data Warehouse Documentation
A step-by-step approach to documenting tables and columns, leveraging YAML and Markdown files. Learn how to create comprehensive documentation, including tests, with the help of Jinja scripts. Enhance collaboration, improve understanding, and ensure the clarity of your data models with dbt's powerful documentation capabilities
Is My dbt Ready?
Set up activities in Azure Data Factory to track when your containerized dbt project has finished running
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
A simple Dockerfile for a dbt Synapse
A sample Dockerfile to containerize a dbt project with Azure Synapse dependencies
dbt run commands for model development
A compilation of a few commands when developing new models in dbt
docker
1 Posts
A simple Dockerfile for a dbt Synapse
A sample Dockerfile to containerize a dbt project with Azure Synapse dependencies
dwh
17 Posts
Working with dbt in VS Code
A brief description of features from CSV Rainbow and dbt Power User, both extensions for VS Code
Cleaning Up Deprecated Models using dbt in Azure Synapse
Use custom macros to remove deprecated tables and views from your target schema
Pausing dbt Resources after Pipeline Retirement
A summary of three ideas on pausing dbt resources: leveraging the exclusion flag, disabling resources, and transforming models into analyses
Creating Date Dimensions using dbt in Azure Synapse: Custom Macros
A custom macro to create date dimensions when using the dbt-synapse adapter
Cool dbt Feature: Exposures
A brief introduction to exposures and why they should be in every dbt project
Cool dbt Feature: Project Variables
A brief introduction to project variables and their benefit when modeling in dbt
Creating Tables with LOBs using dbt in Azure Synapse: The Columnstore Index Error
A work-around to the default settings when creating tables using the dbt-synapse adapter
Understanding DAGs in dbt: Threads, Errors and Failing Fast
Three scenarios show us the implications of running a Directed Acyclic Graph (DAG) in multi-threaded dbt environments, shedding light on how errors and the "fail fast" argument impact model execution
Taking snapshots using dbt in Azure Synapse: The Hash Distribution Issue
Learn how to troubleshoot and resolve the "Hash Distributed Table" error when taking snapshots using dbt in Azure Synapse Dedicated Pool
Understanding DAGs in dbt: Erring Models and Failing Tests
Three scenarios shows us what happens downstream when our models and/or tests do not succeed
dbt seed: Data Warehousing and CSV Files
The following are a few points I have learned while working with seeds in my dbt project
Effortless Documentation with dbt: Streamlining Data Warehouse Documentation
A step-by-step approach to documenting tables and columns, leveraging YAML and Markdown files. Learn how to create comprehensive documentation, including tests, with the help of Jinja scripts. Enhance collaboration, improve understanding, and ensure the clarity of your data models with dbt's powerful documentation capabilities
Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
Is My dbt Ready?
Set up activities in Azure Data Factory to track when your containerized dbt project has finished running
Running Scripts using Azure Data Factory and Batch, Part II
An example on how to run scripts within an Azure Data Factory pipeline
Running Scripts using Azure Data Factory and Batch, Part I
An example on how to connect Azure Batch and Azure Storage accounts to Azure Data Factory
Testing Expressions in Azure Data Factory
Testing expression is a simple trick in Azure Data Factory, so that you are able to see what is being passed on to your conditional activities.
fonts
1 Posts
Using Google Fonts with Plots in R
A short example on how to use Google Fonts within ggplot graphs
gcp
4 Posts
Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
Syncing a Cloud Source Repository to your Composer Airflow Environment
And thus starting your CI/CD pipeline
Gathering Questions about GCP
A sample of questions about GCP used as a study-guide for Google's Associate Cloud Engineer Certification
Preventing Unexpected Billing in GCP
A few tips on how to avoid surprises at the end of the month
machine-learning
4 Posts
Applying Scalers using DataFrameMapper()
An example on how to scale and hot-encode variables while preprocessing your data frame.
Creating Sample Datasets in Python
A script to create sample pandas data frames with randomly generated data.
Deriving Meaning from Date Features for Prediction in Python
Some ideas on new features that can be created based on date features
Saving and Loading Scalers using MLflow and Databricks with Python
Chances are, you will be needing scalers to make predictions on new data sets
maps
3 Posts
Creating Minimalist Map Posters using R
Levering R and OpenStreetMaps to create custom minimalist posters
Mapping of Postal and District Codes
A mapping from Austrian 4-digit postal codes to Austrian 3-digit district codes
OpenStreetMap Features and Tags in R
Some tips to find the right features and tags for the elements you are mapping in R
metabase
3 Posts
Mapping of Postal and District Codes
A mapping from Austrian 4-digit postal codes to Austrian 3-digit district codes
Using Number Filters in Metabase
A work around to slider-like filters in Metabase
Using Custom Maps in Metabase
All you need is the GeoJSON file of the map you would like to plot on
onedrive
1 Posts
Importing Excel Files from a Personal OneDrive in Python
I saw multiple people struggling with this topic on Stack Overflow. Here is simple and quick way of importing excel data located in a Personal OneDrive into Python
power-bi
17 Posts
Nudging The End User
In PBI, you can nudge the user towards selecting the data they need first, before it is actually displayed in the report
Keeping Table Highlights when Exporting to PDF in Power BI
Highlights in Power BI are a great presentation tool which can unfortunately act oddly when exporting to PDF
Promoting Data Marts in Power BI
And make them stand out from all other datasets in your account
On the Reasons to Create Data Marts in Power BI
Saving you and your team priceless time when creating reports for your organization
Datasets vs Reports in Power BI
And the importance of distinguish the one from the other.
Using PowerPoint Backgrounds in Power BI
Keep your logo and page layout standard across the your organization
Using Calendar Weeks in Power BI
Allowing months and years to be composed of exact weeks.
Creating a Custom Theme in Power BI
Save time and make your Power BI reports look professional by creating custom themes.
Go to Column in Power Query
A simple trick to make your Power Query work more efficient
Formatting DAX Code
Keeping your code neat and tidy
Display the Last Refresh Date and Time in Power BI
Give your stakeholders the ability to know when your data was last refreshed precisely
Connecting to Excel Files in OneDrive Personal from Power BI
A common but not intuitive task.
Create Calendar Tables in Power BI
Tips on creating basic calendar tables.
Getting to Know Power BI
A brief description of the main components of Power BI as a tool
Getting to Know the Services in Power BI
A brief description of the three main services offered by Power BI
Getting a Power BI Account
A description of the different types of accounts
Reason to Choose Power BI
Thoughts after working with the tool for about two years
python
8 Posts
Surf, Work, and Explore: A Web Scraping Exercise
I scrape data using python to find new countries with surf spots where you can work remotely from, ceteris paribus
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
Applying Scalers using DataFrameMapper()
An example on how to scale and hot-encode variables while preprocessing your data frame.
Creating Sample Datasets in Python
A script to create sample pandas data frames with randomly generated data.
Matching First Names and Gender using gender-guesser
gender-guesser is a python package that allows you to match a first names and name origin to a gender
Deriving Meaning from Date Features for Prediction in Python
Some ideas on new features that can be created based on date features
Saving and Loading Scalers using MLflow and Databricks with Python
Chances are, you will be needing scalers to make predictions on new data sets
Importing Excel Files from a Personal OneDrive in Python
I saw multiple people struggling with this topic on Stack Overflow. Here is simple and quick way of importing excel data located in a Personal OneDrive into Python
r
3 Posts
Creating Minimalist Map Posters using R
Levering R and OpenStreetMaps to create custom minimalist posters
OpenStreetMap Features and Tags in R
Some tips to find the right features and tags for the elements you are mapping in R
Using Google Fonts with Plots in R
A short example on how to use Google Fonts within ggplot graphs
reading
4 Posts
Top 10 Books Read in 2023
Selected quotes from what I considered were the best books I read/listened to in 2023.
Top 10 Books Read in 2022
Selected quotes from what I considered were the best books I read/listened to in 2022.
Top 10: Books Read in 2021
Selected quotes from what I considered were the best books I read/listened to in 2021.
Top 10: Books Read in 2020
A few thoughts on what I considered were the best books I read/listened to in 2020.
remote-work
5 Posts
The Nomad Work
A list of spots where to do some remote work
Surf, Work, and Explore: A Web Scraping Exercise
I scrape data using python to find new countries with surf spots where you can work remotely from, ceteris paribus
Working from Taghazout, Morocco
The beauty is found in the mess
Working from Santa Teresa, Costa Rica
Strong waves (surf, not WiFi)
Working from Corralejo, Fuerteventura
Some learnings from my travels in the nomad island
running
11 Posts
Running the Transgrancanaria Classic Ultra
An account of unexpected challenges and forging unlikely friendships during the grueling trail ultramarathon across Gran Canaria
Berlin Marathon Data: 1974-2023
Berlin Marathon results among runners from 1974 to 2023, including data from Kaggle for the years 1974-2019 and additional results from 2021-2023
A Runner's Tale of Plantar Fasciitis
The story of a rollercoaster ride dealing with foot inflammation and my journey back to running
Running my First 100-miler
My experience running the Berlin Wall, my first 100-mile foot race
4 x 4 x 48
Going through with the running challenge popularized by David Goggins
Challenge November 2020: My First 100k Run
Running alongside the Mauerweg on a cold November day
Running in the Tatras
I insist, the color of the trail marker has nothing to do with the trail's difficulty!
Challenge July 2020: My First 50-Miler
Discovering the power of alcohol-free beer on a warm July 4th
The Foundation Program
My experience following Eric Orton’s running plan, the Foundation Program
Running my First Ultra, Part II
A tale of my achievements at the Balaton Supermarathon 2020
Running my First Ultra, Part I
A tale of my struggles in the Balaton Supermarathon 2020
synapse
4 Posts
Cleaning Up Deprecated Models using dbt in Azure Synapse
Use custom macros to remove deprecated tables and views from your target schema
Creating Date Dimensions using dbt in Azure Synapse: Custom Macros
A custom macro to create date dimensions when using the dbt-synapse adapter
Creating Tables with LOBs using dbt in Azure Synapse: The Columnstore Index Error
A work-around to the default settings when creating tables using the dbt-synapse adapter
Taking snapshots using dbt in Azure Synapse: The Hash Distribution Issue
Learn how to troubleshoot and resolve the "Hash Distributed Table" error when taking snapshots using dbt in Azure Synapse Dedicated Pool
vscode
1 Posts
Working with dbt in VS Code
A brief description of features from CSV Rainbow and dbt Power User, both extensions for VS Code