Adding currency exchange rates into your Data Warehouse.

How to add exchange rates pipeline into BigQuery or Snowflake.

Photo by Austin Distel on Unsplash

Have you ever wondered how to add currency exchange rates into your Data Warehouse ?

There is a number of online resources where you can get exchange rate data and currency conversion on a daily basis in JSON format.

In this tutorial, we’ll walk through setting up our own curency exchange rates pipeline to Data Warehouse . Our example will compare how it works in BigQuery and Snowflake .

We will see what it takes to handle openexchangerates JSON data, how to UNNEST and/or FLATTEN in order to represent it in columnar view ready to be used with your other tables .

Currency exchange rates data

For example, I use Lambda Function to pull exchange rates on a daily basis and save it in my S3 bucket. This is how currency exchange rates look after you hit Openexchangerates API.

What I need to have in my Data Warehouse is something like this: