Week 7 Final Project: Build an End-to-end Data Pipeline for Reporting CO2 and other greenhouse gases

March 30, 2023 in Data Engineering, Analytics Engineering

Climate change is one of the most challenges of our time. I use the techniques and tools learned in the course to build an end-to-end data pipeline and CO2 dashboard.

Create a Free Custom Domain-based email address

March 25, 2023 in Personal Website, Email

Once I’ve established my personal website, the next step is to create a professional email address. The good news, you can create a custom email address with Zoho for free, which takes only 5 minutes to complete.

Take A Break and Continue

March 11, 2023 in Life, Continuous Learning

I took a break to reflect and recharge. ChatGPT is a powerful tool. But it can’t solve every problem. The panic I experienced was genuine, but taking a break allows me to move forward with renewed energy.

Week 4 Data Engineering Zoomcamp Notes: Analytics Engineering and dbt

February 18, 2023 in Data Engineering, Analytics Engineering

I had completed the analytics engineering bootcamp in Udemy. So data modelling and dbt are not new to me. In week 4, some interesting take-aways for me are the usage of dbt cloud and the software engineering best practices like modularity, portability, CI/CD, and documentation. Great course as usual from dezoomcamp!

Week 3 Data Engineering Zoomcamp Notes: Data Warehouse and BigQuery

February 11, 2023 in Data Engineering

In week 3 we learned data warehouse and google Big Query. Some best practices like creating external table, partition and clustering tables are introduced. We can even do machine learning in Big Query.

Week 2 Data Engineering Zoomcamp Notes: Prefect

February 2, 2023 in Data Engineering

In week 2 we learned Prefect to manage our data flow. I like Prefect very much because it allows to use python to define tasks and flows. Very interesting topic to learn!