Surf, Work, and Explore: A Web Scraping Exercise
I scrape data using python to find new countries with surf spots where you can work remotely from, ceteris paribus
Hi there! My name is Paulo and I am an analytics/data engineer and endurance runner based in Berlin. Welcome to my blog.
When I am not building data pipelines, you will find me training for my next endurance race. I write about both.
Currently preparing for Spartathlon 2026 and documenting the journey in my 246 series.
Effortless Documentation with dbt: Streamlining Data Warehouse Documentation
A step-by-step approach to documenting tables and columns, leveraging YAML and Markdown files. Learn how to create comprehensive documentation, including tests, with the help of Jinja scripts. Enhance collaboration, improve understanding, and ensure the clarity of your data models with dbt's powerful documentation capabilities
My Airflow Journey
Starting from scratch is impractical, Google Cloud Platform's Composer is expensive, and debugging Airflow on Windows is challenging
Linking to BigQuery in Azure Data Factory
In this article, we go over the steps, requirements and recommendations when creating a BigQuery linked service in Azure Data Factory
Nudging The End User
In PBI, you can nudge the user towards selecting the data they need first, before it is actually displayed in the report
Is My dbt Ready?
Set up activities in Azure Data Factory to track when your containerized dbt project has finished running
Keeping Table Highlights when Exporting to PDF in Power BI
Highlights in Power BI are a great presentation tool which can unfortunately act oddly when exporting to PDF
Alerting for dbt Errors in Azure Data Factory Using Databricks
Stay on top of errors that may come up when running your dbt build
Pulling Azure Container Instance Logs into Azure Data Factory
A description of how to include Azure Container Instance logs into your Data Factory pipeline.